Project: a tuned model

Two ways into the same idea

Before You Start

The Core Idea

In this project you tune a small model on one Arabic task, and prove with a number whether it beat the best prompting can do. You choose the task, build the data, run the tuning, then fill in the results table.

1Choosing the task

Pick a narrow task whose answers can be checked automatically, with enough volume to justify tuning or a clear reason to need a small model.

Look at the picture: three tasks that suit a first project, and one to avoid because evaluating it is much harder.

2The data and the frozen test

You need 500 to 2,000 clean examples, a written labelling guide, and a grouped split. And the test set is frozen before any training.

Look at the picture: each step from the data lesson, in order, before you run any training.

3The tuning run

The run is LoRA on a small open model, with the settings recorded in the folder name, and the checkpoint at the lowest validation loss.

It needs a GPU and pip install transformers peft trl datasets, so it is here for reading. A model of half a billion to 1.5 billion parameters is enough for a classification task, and trains in under an hour on a mid-range or free card.

from datasets import load_dataset
from peft import LoraConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
from trl import SFTConfig, SFTTrainer

BASE = "Qwen/Qwen2.5-0.5B-Instruct"
tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype="bfloat16")

data = load_dataset("json", data_files={"train": "train.jsonl", "val": "val.jsonl"})
peft = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, target_modules="all-linear", task_type="CAUSAL_LM")
args = SFTConfig(
    output_dir="runs/r16-lr2e-4-ep2",
    per_device_train_batch_size=4, gradient_accumulation_steps=8,
    learning_rate=2e-4, num_train_epochs=2, bf16=True,
    eval_strategy="steps", eval_steps=50, save_steps=50, load_best_model_at_end=True,
    completion_only_loss=True,           # learn the assistant's answer, not the user's message
)
trainer = SFTTrainer(model=model, args=args, train_dataset=data["train"], eval_dataset=data["val"],
                     processing_class=tok, peft_config=peft)
trainer.train()
trainer.save_model("runs/r16-lr2e-4-ep2/best")

completion_only_loss=True puts the loss on the assistant's answer only, so the model does not spend capacity learning to rewrite user messages. After training, run the tuned model and the original with both prompts on the frozen test set and the general set, and fill the table with the same code as above.

4The results table

The table compares three systems on the same frozen questions: the prompt alone, the prompt with examples, and the tuned model. The fair comparison is against the prompt with examples, not a poor prompt.

Three systems on the same 300 frozen questions, plus a general set of 100 questions to detect forgetting. The numbers here are generated for illustration; in your project they come from a real run of each system.

import numpy as np

rng = np.random.default_rng(7)
# per-question results on a frozen set of 300 (1 = correct); in your project these come from real runs
task = {"prompt only": rng.random(300) < 0.62,
        "prompt + 8 examples": rng.random(300) < 0.74,
        "LoRA tuned": rng.random(300) < 0.86}
general = {"prompt only": rng.random(100) < 0.80,         # 100 unrelated questions: forgetting check
           "prompt + 8 examples": rng.random(100) < 0.80,
           "LoRA tuned": rng.random(100) < 0.74}

def interval(a, b, n=2000):
    idx = rng.integers(0, len(a), (n, len(a)))
    d = b[idx].mean(1) - a[idx].mean(1)
    return np.percentile(d, [2.5, 97.5])

base = task["prompt + 8 examples"]
print(f"{'system':22} {'task':>6} {'vs few-shot (95%)':>22} {'general':>8}")
for name in task:
    if task[name] is base:
        cell = "baseline"
    else:
        lo, hi = interval(base, task[name])
        cell = f"[{lo:+.1%}, {hi:+.1%}]"
    print(f"{name:22} {task[name].mean():6.1%} {cell:>22} {general[name].mean():8.0%}")

The baseline is the prompt with eight examples, not the poor prompt. The tuned model beat it with an interval entirely above zero, so the improvement is real. But it lost 10 points on the general set. If the model will serve this task alone, that is acceptable and goes in the report. If it will serve others, rerun mixing in general examples.

Look at the picture: this is the table's other column, the general set. The tuned model won on the task, but lost here. Both numbers go into the recommendation.

How It Works, Step by Step: how to run the project from task to recommendation

Worked example

Pick a narrow task with checkable answers: classifying messages into categories, extracting fields from text into JSON, or converting dialect to standard Arabic with clear criteria. Avoid open tasks like creative writing, since evaluating them is much harder for a first project.

Try it without code

Myth vs Reality

Try it yourself

Quick review cards

Check Your Understanding