Suppose several support tickets belong to the same customer. Randomly splitting individual tickets can put that customer's writing and repeated details into both training and evaluation data. A group-based split keeps those related records together. This example implements a deterministic assignment that remains stable when rows are reordered.

Prerequisites are Python 3.12 or later and a stable group identifier for each record. The program uses the standard library only. Its three named partitions are an example policy for an AI dataset; it does not train a model or claim that a tiny fixture has useful evaluation coverage.

Complete example

Save as split.py:

from hashlib import sha256

def partition(group):
    if not isinstance(group, str) or not group:
        raise ValueError('group ID required')
    bucket = int.from_bytes(sha256(group.encode('utf-8')).digest()[:8], 'big') % 10
    return 'test' if bucket == 0 else 'validation' if bucket == 1 else 'train'

rows = [('ticket-1', 'customer-a'), ('ticket-2', 'customer-a'),
        ('ticket-3', 'customer-b'), ('ticket-4', 'customer-c')]
assignments = [(row, group, partition(group)) for row, group in rows]
assert assignments[0][2] == assignments[1][2]
seen = {}
for _, group, split in assignments:
    seen.setdefault(group, set()).add(split)
assert all(len(splits) == 1 for splits in seen.values())
assert [partition(g) for _, g in rows] == [partition(g) for _, g in rows[::-1]][::-1]
try:
    partition('')
    raise AssertionError('accepted empty group')
except ValueError:
    pass
print('group isolation, order stability and empty-ID checks passed')

Run

python3 split.py

Expected output:

group isolation, order stability and empty-ID checks passed

The UTF-8 group identifier is hashed, the first eight digest bytes become an integer, and that integer selects one of ten buckets. Bucket zero goes to test, bucket one to validation, and the others to training. Equal group identifiers always follow the same path. Unlike a row counter, assignment does not change when an input file is sorted differently.

The bucket policy targets approximately 80/10/10 across sufficiently numerous, suitably distributed groups. It does not guarantee those proportions for records: one large customer can dominate a partition. Before accepting a dataset, count groups, rows and labels in each split. For a small dataset, an explicitly reviewed group allocation may be more appropriate.

Edge cases and troubleshooting

An empty identifier is rejected rather than allowing every missing value to become one artificial group. Normalize identifiers upstream under a documented rule, and preserve the same rule across dataset versions. Changing customer identity mappings can change the leakage boundary even if this function remains unchanged.

Group isolation addresses one leakage mechanism. Future information, duplicate content under different IDs, and related groups can still invalidate evaluation. If deployment predicts future events, investigate a time-based evaluation as well. Keep a final test set separate from repeated tuning, and record the assignment policy with the dataset artifact.

The tests establish deterministic assignment and shared-group isolation for fixtures. They do not establish class balance, statistical power, anonymization or model generalization.

Primary references

See AWS guidance on dataset splits and leakage and Python's hashlib documentation. Explore related AI and data-science tutorials.