Build Python Embedding Cache Keys That Include Model Revision

Caching an embedding by text alone creates a subtle integration problem: the same text can be embedded by a different model revision, with different dimensions, or after a preprocessing change. Reusing the earlier vector then mixes incompatible results. Define the cache identity before connecting the embedding provider.

This example serializes an explicit identity record and hashes its UTF-8 bytes. The Python JSON documentation supports sorted keys and compact separators, and the hashlib documentation describes SHA-256 and hexadecimal digests. The choice of fields is an application design decision, demonstrated by the tests.

Prerequisites

Use Python 3.12 or a newer supported Python 3 release. Save the following as cache.py. The embedding generator is deliberately fake: it repeats the text length and has no semantic meaning. It lets the program count cache misses without calling a model or external API.

import hashlib
import json

cache = {}
calls = 0

def embedding(text, *, revision, dimensions, preprocessing='raw-v1'):
    global calls
    if not isinstance(text, str) or not revision:
        raise ValueError('text and immutable revision required')
    if type(dimensions) is not int or dimensions < 1:
        raise ValueError('positive dimensions required')
    identity = {'schema': 1, 'text': text, 'revision': revision,
                'dimensions': dimensions, 'preprocessing': preprocessing}
    payload = json.dumps(identity, sort_keys=True, ensure_ascii=False,
                         separators=(',', ':'), allow_nan=False).encode('utf-8')
    key = hashlib.sha256(payload).hexdigest()
    if key not in cache:
        calls += 1
        # Deliberately fake embedding: exercises cache behavior only.
        vector = tuple(float(len(text)) for _ in range(dimensions))
        cache[key] = vector
    return cache[key]

a = embedding('tea', revision='fixture-r1', dimensions=2)
assert embedding('tea', revision='fixture-r1', dimensions=2) is a
assert calls == 1
embedding('tea', revision='fixture-r2', dimensions=2)
embedding('tea', revision='fixture-r1', dimensions=3)
embedding('tea ', revision='fixture-r1', dimensions=2)
embedding('tea', revision='fixture-r1', dimensions=2, preprocessing='trim-v2')
assert calls == 5
try:
    embedding('tea', revision='fixture-r1', dimensions=True)
except ValueError:
    pass
else:
    raise AssertionError('boolean dimensions accepted')
print('cache entries:', len(cache))
print('fixture embedding calls:', calls)

Run and expected output

python3 cache.py

cache entries: 5
fixture embedding calls: 5

Understand cache identity

The second identical request returns the same stored tuple and leaves the call counter unchanged. Changing the revision, dimensions, exact text, or preprocessing label produces a different entry. The schema number gives the identity format its own version, allowing a future implementation to change serialization rules without silently sharing older entries.

Whitespace is preserved intentionally. The strings tea and tea followed by a space may be equivalent for a particular application, but that policy belongs in preprocessing. If preprocessing changes, update its identifier and ensure the actual transformation matches the declared identifier. This example records a preprocessing version but does not implement a trimming pipeline.

Integrating a real provider

Replace the clearly marked fixture generator with your provider adapter only after establishing a stable revision identifier. Validate returned vector length and finite numeric values before storing a response. Cache successful vectors, rather than transport errors disguised as results. If the provider offers only a mutable model alias, define an application revision and rotate it when the deployment changes.

The in-memory dictionary is intentionally limited to one process and one run. A production cache also needs capacity limits, concurrency control, and isolation appropriate to its data. A hash key is an identifier, not encryption for sensitive input. No persistent storage behavior or cross-process consistency is claimed here.

Troubleshooting

Unexpected misses usually indicate inconsistent text or metadata. Unexpected hits suggest a missing identity field or an incorrectly reused revision. Inspect the field definitions with synthetic inputs rather than logging private documents. The exact integer type check also rejects Python booleans as dimensions.

Continue with Python tutorials and the AI category.