Tuning data

Two ways into the same idea

Before You Start

The Core Idea

A tuned model is never better than its data. In this lesson you write the data in the right format, clean it, split it without leakage, and freeze the test file.

1The format: JSONL

Each line of a JSONL file is one complete example. It holds the system, user and assistant roles in the same template the model will run with in production.

Look at the picture: one line breaks into its roles, plus a source field that says where the example came from. The model learns to write what is in assistant.

2Cleanliness before quantity

A thousand clean, consistent examples beat twenty thousand full of duplicates and contradictory labels. Start with 500 to 1,000 clean examples, measure, then add where the model fails.

Before any training, every line goes through automated checks: the roles, empty answers, labels from a fixed list, and duplicates after normalising the text.

import json
import re

lines = [
    '{"messages": [{"role": "user", "content": "صنّف: التوصيل تأخر أسبوعا"}, {"role": "assistant", "content": "شكوى"}]}',
    '{"messages": [{"role": "user", "content": "صنّف: شكرا على سرعة الرد"}, {"role": "assistant", "content": "شكر"}]}',
    '{"messages": [{"role": "user", "content": "صنّف: التوصيل تأخّر أسبوعاً"}, {"role": "assistant", "content": "شكوى"}]}',
    '{"messages": [{"role": "user", "content": "صنّف: متى تفتحون؟"}, {"role": "assistant", "content": ""}]}',
    '{"messages": [{"role": "user", "content": "صنّف: السعر مرتفع جدا"}]}',
    '{"messages": [{"role": "user", "content": "صنّف: أين فاتورتي"}, {"role": "assistant", "content": "استفسار"}]}',
]
LABELS = {"شكوى", "شكر", "استفسار"}

def key(text):
    text = re.sub(r"[ً-ْـ]", "", text)
    return re.sub(r"\s+", " ", text).strip()

clean, seen, problems = [], set(), []
for n, line in enumerate(lines, 1):
    msgs = json.loads(line)["messages"]
    roles = [m["role"] for m in msgs]
    if roles != ["user", "assistant"]:
        problems.append((n, f"roles are {roles}"))
        continue
    answer = msgs[1]["content"]
    if answer not in LABELS:
        problems.append((n, f"label {answer!r} is not one of {sorted(LABELS)}"))
        continue
    k = key(msgs[0]["content"])
    if k in seen:
        problems.append((n, "duplicate of an earlier example"))
        continue
    seen.add(k)
    clean.append(msgs)

print("kept", len(clean), "of", len(lines))
for n, why in problems:
    print(f"line {n}: dropped, {why}")

Three of six lines were dropped, and each would have hurt the model: line three is line one with different diacritics, so the model learns it twice and evaluation looks better than it is. Line four has an empty answer, so the model learns silence is acceptable. Line five has no answer at all. In a real file of thousands of lines, these checks always find dozens of cases.

3Splitting and leakage

Leakage is identical or near-identical examples in training and test. The model memorises them, looks excellent, then fails on new inputs.

Look at the picture: in the random split every test example had a twin in training. In the split grouped by template, none did.

A hundred examples generated from four sentence templates with different order numbers. This is very common in generated data or data pulled from a ticketing system.

import random

random.seed(0)
templates = ["الطلب رقم {} لم يصل", "أريد إلغاء الطلب {}", "الطلب {} وصل تالفا", "كم سعر شحن الطلب {}"]
examples = [{"text": t.format(random.randint(1000, 9999)), "template": i}
            for i, t in enumerate(templates) for _ in range(25)]

def shape(text):
    return "".join("#" if c.isdigit() else c for c in text)

def leaked(train, test):
    seen = {shape(e["text"]) for e in train}
    return sum(shape(e["text"]) in seen for e in test)

random.shuffle(examples)
train, test = examples[:80], examples[80:]
print("random split:  test examples whose exact shape is in train:", leaked(train, test), "of", len(test))

test_templates = {3}
train = [e for e in examples if e["template"] not in test_templates]
test = [e for e in examples if e["template"] in test_templates]
print("grouped split: test examples whose exact shape is in train:", leaked(train, test), "of", len(test))

With a random split, every test example has a "twin" in training differing only in the number, so the model memorises the template and scores perfectly without learning anything that helps with a new phrasing. With a grouped split, the whole fourth template is in the test set, so the score measures whether the model generalises to a phrasing it never saw. That number is lower, and it is the right one.

4Freezing the test file

The split has three parts: training, validation and test. The test file is written once before any training, kept in Git, and never regenerated.

from datasets import load_dataset

ds = load_dataset("json", data_files="clean.jsonl", split="train")
# group by the source each example came from, so near-twins stay on one side
sources = sorted(set(ds["source"]))
test_sources = set(sources[::5])                      # every fifth source goes to test
test = ds.filter(lambda e: e["source"] in test_sources)
rest = ds.filter(lambda e: e["source"] not in test_sources).train_test_split(test_size=0.1, seed=0)
train, val = rest["train"], rest["test"]
test.to_json("test_frozen.jsonl")                     # written once, never regenerated
print(len(train), len(val), len(test))

The source field (the original conversation, template or customer) is what makes a grouped split possible, so add it to every example from the start. And the test file is written once and kept in Git, never regenerated, so it cannot quietly change between experiments.

How It Works, Step by Step: how to prepare a tuning file

Going deeper

And variety guards against over-specialising. If all your examples are short and formal, the model will fail on a long message in colloquial dialect. Collect examples from different sources, lengths and dialects if your users write in them, and make sure every category is represented well enough.

Worked example

Data for tuning on conversations is usually written as a JSONL file: each line a complete example with user and assistant messages. Before any training, every line goes through automated checks.

Try it without code

Myth vs Reality

Try it yourself

Quick review cards

Check Your Understanding