A retrieval stage may return more evidence than a generation request can accept. Cutting the final prompt at an arbitrary boundary can remove part of a quotation or leave a source label detached from its text. This example selects whole passages in retrieval order and records which passages were skipped. It measures each fully rendered candidate prompt through an injected counter.

Prerequisites and counting policy

Use Python 3.12 with no dependencies. The runnable fixture counts whitespace-separated words, deliberately not model tokens. That makes selection and boundary behavior testable without downloading a tokenizer. It does not establish a safe token limit for any model. The counter is a dependency: a production adapter must count the exact model input, including its chat template and special tokens, and return a nonnegative integer.

The input limit represents the allowance for the complete prompt after the application has reserved any required generation capacity. Here, labels, question text, the sources heading and answer cue all participate in measurement. The algorithm first rejects a base prompt that already exceeds the limit. It then tries passages one at a time, retaining each one only if the new complete prompt fits. A rejected large passage does not prevent a later small passage from being selected.

Complete example

Save as context.py.

def render(question, passages):
    return 'Question: ' + question + '\nSources:\n' + ''.join(
        f'[{key}] {text}\n' for key, text in passages) + 'Answer:'

def pack(question, passages, limit, count):
    if type(limit) is not int or limit < 0:
        raise ValueError('invalid limit')
    ids = [key for key, _ in passages]
    if any(not isinstance(key, str) or not key for key in ids) or len(set(ids)) != len(ids):
        raise ValueError('invalid passage identifiers')
    def size(parts):
        result = count(render(question, parts))
        if type(result) is not int or result < 0:
            raise ValueError('counter must return nonnegative integers')
        return result
    if size([]) > limit:
        raise ValueError('base prompt exceeds limit')
    chosen, skipped = [], []
    for passage in passages:
        if size(chosen + [passage]) <= limit:
            chosen.append(passage)
        else:
            skipped.append(passage[0])
    return chosen, skipped, render(question, chosen)

# Fixture word units, deliberately NOT a model tokenizer.
count = lambda text: len(text.split())
passages = [('a', 'one two'), ('b', 'one two three four five'), ('c', 'six')]
chosen, skipped, prompt = pack('why', passages, 9, count)
assert [x[0] for x in chosen] == ['a', 'c']
assert skipped == ['b'] and count(prompt) == 9
assert pack('why', [], 4, count)[0] == []
for args in [('why', [], 3, count), ('why', [('a', 'x'), ('a', 'y')], 20, count)]:
    try:
        pack(*args)
    except ValueError:
        pass
    else:
        raise AssertionError('invalid input accepted')
print('chosen=a,c skipped=b units=9')
print('fixture and boundary checks passed')

Run and expected output

python3 context.py
chosen=a,c skipped=b units=9
fixture and boundary checks passed

Why the fixture selects a and c

The empty rendered prompt uses four fixture word units. Passage a adds three, b would add six more, and c adds two. With a limit of nine, a and c fit exactly while b is skipped. Each trial re-renders and recounts the complete candidate, so the implementation does not assume that separately counted passage sizes can simply be added. Accepted passages preserve their incoming ranking order.

Duplicate or empty passage identifiers fail early because an ambiguous selection manifest is difficult to audit. The returned prompt is the same text that was measured for the accepted selection. Keep it together with the selected identifiers if another application component will dispatch the request. Reformatting it later would invalidate the earlier measurement.

Edge cases and troubleshooting

This greedy policy is straightforward but does not maximize total evidence value or solve a knapsack optimization problem. It may skip a highly relevant long passage. Review skips rather than assuming they are irrelevant. The renderer is a minimal text fixture, not an injection defense or a provider chat format. If a real request exceeds its context window, inspect the actual serialized input and tokenizer configuration; increasing this fixture's word limit cannot fix a model-specific accounting error. Only fixture selection, the exact-fit case, the empty-passage case, a base overflow and duplicate identifiers were exercised. A real tokenizer, model context window and generation request remain untested.

Primary references

Continue with AI tutorials.