Test Python AI Adapters with a Strict Request Cassette

A canned AI response is useful only if your test also checks the request that produced it. A fake transport that returns the same summary for every input can let a changed prompt, wrong model selection or accidental extra call pass unnoticed. This tutorial builds a strict, single-use request cassette for a small Python AI adapter.

Prerequisites: Python 3.12 and its standard library. The request and response shapes are application-owned demonstration contracts. They are not a provider SDK schema. No model service, authentication data, recorded private conversation or network access is used.

Make request drift fail

The cassette serializes the expected request with sorted dictionary keys and compact separators. This allows dictionary insertion order to differ while preserving field values, list order and the presence of individual fields. Python documents these serialization options in the JSON reference. This is a local fixture comparison convention, not a general canonical JSON or cryptographic signing format.

import copy
import json

def key(request):
    return json.dumps(request, sort_keys=True, separators=(",", ":"), allow_nan=False)

class Cassette:
    def __init__(self, request, response):
        self.expected = key(request)
        self.response = copy.deepcopy(response)
        self.remaining = 1

    def __call__(self, request):
        if key(request) != self.expected:
            raise AssertionError("unexpected AI request")
        if self.remaining == 0:
            raise AssertionError("fixture already consumed")
        self.remaining -= 1
        return copy.deepcopy(self.response)

    def verify(self):
        if self.remaining:
            raise AssertionError("unused fixture")

def summarize(send, text):
    response = send({"model": "fixture-model", "input": text,
                     "max_output_tokens": 40})
    return response["output_text"].strip()

request = {"input": "Public demo text", "max_output_tokens": 40,
           "model": "fixture-model"}
transport = Cassette(request, {"output_text": " Demo summary "})
try:
    summarize(transport, "Changed text")
except AssertionError:
    pass
else:
    raise AssertionError("request drift accepted")
assert summarize(transport, "Public demo text") == "Demo summary"
transport.verify()
try:
    summarize(transport, "Public demo text")
except AssertionError:
    pass
else:
    raise AssertionError("extra call accepted")
unused = Cassette(request, {})
try:
    unused.verify()
except AssertionError:
    pass
else:
    raise AssertionError("missing call accepted")
print("cassette checks passed: drift, match, extra call, missing call")

Run and inspect the result

Save as cassette.py and run:

python3 cassette.py

Expected output:

cassette checks passed: drift, match, extra call, missing call

The first request deliberately uses different text and must fail without consuming the fixture. The correct request then succeeds once. A repeated call fails, and a separate unused cassette fails its final verification. These cases test both what the adapter sends and how many successful fixture interactions it performs.

Design choices and troubleshooting

Injecting the transport as send keeps the test independent of HTTP clients. You can put a real network adapter behind the same callable boundary later. Keep provider-specific translation inside that adapter and test it separately against the provider's documentation.

The cassette copies response structures so ordinary nested mutable data does not become shared fixture state; see Python's copy reference. The demonstrated checks do not independently test mutation isolation. Restrict requests to JSON-compatible structures with string dictionary keys. Do not normalize prompt whitespace or reorder message lists merely to make a failing test pass: those changes may alter the model's input.

If a request mismatch occurs, compare model identifiers, prompt text, omitted fields and output limits. This test intentionally avoids printing an entire unexpected request. The fixture validates adapter behavior and summary trimming; it does not validate factual accuracy, HTTP error handling or compatibility with a live model endpoint.

Continue with Python tutorials and AI tutorials.