← explainers
00 · opening
Last updated September 23, 2026

Fine-tune a small model, on your own laptop.

I had read a dozen threads about fine-tuning and done zero fine-tunes. So I did one, start to finish, on a MacBook, and wrote down every command. I used a 1.5B model and 600 examples. Training took two minutes and left a 21 MB file. You can copy every block below, and I ran every one of them. Follow along and in about half an hour you will have your own fine-tuned model answering on localhost.

2 → 100
exact answers / 100
prompt only, then with my adapter
128 s
training time
300 steps on an M5 Pro
3.5 GB
peak memory
an 8 GB Mac is enough
0.34%
of weights trained
5.3M of 1.54B
THE EIGHT STEPS · CLICK ONE TO JUMP
a mac
Apple Silicon (M1 or newer). Peak memory was 3.5 GB, so 8 GB of RAM works. On an NVIDIA GPU or Colab, read everything, then use the translation table in the last step.
disk
About 2 GB: a 0.9 GB model download, plus a fused copy at the end.
tools
A terminal and uv, nothing else. You need no API keys, no accounts and no cloud.
time
Thirty minutes of running commands. It takes longer if you stop to break things on purpose, which you should.
scroll, or use the arrow keys
01 · before you start

Fine-tuning teaches habits, not facts.

The post that pushed me into this was Rahul's full guide to fine-tuning a small LLM. Its main claim is that a 1.5B model trained on a few hundred good examples can beat a frontier model prompted generically, at your one task.

The catch is the word "task." Fine-tuning changes how a model answers: format, tone, the house rules it follows without being reminded. It is a bad way to teach it new facts. If you want facts, use retrieval.

So the rule is prompt first. Fine-tune only when a prompt fails at something specific that you can measure.

Here is the task I picked. Turn a messy message into a calendar event:

can we do the sprint planning with Wei, Chloe & Nia on Mon at 8:45 AM, 60 min?
{"title": "sprint planning", "day": "Monday", "time": "08:45", "duration_min": 60, "people": ["Wei", "Chloe", "Nia"]}

The answer has to follow my house rules:

title
lowercase, short, the words the sender used. "sprint planning", never "Sprint Planning Meeting".
day
a full weekday name. "Mon" and "next monday" both become "Monday".
time
24-hour HH:MM. "half past 4" becomes "16:30".
duration_min
minutes as an integer. No duration given means 30.
people
everyone named, except me.

None of these rules is hard. What matters is that they are arbitrary. A prompt can describe them, but a model has to see them a few hundred times before it follows them every time. Fine-tuning fills that gap.

I also picked this task because code can grade the output. So every claim on this page is a measured number.

01the vocabularysix words you will see all the way down+

Six words

base model
The model you start from and never change. Here that is Qwen2.5-1.5B-Instruct, already a chat model, just not my chat model.
SFT
Supervised fine-tuning. Show it (input, ideal answer) pairs and train it to predict the answer, token by token. That is all we do here. The post-training explainer covers what comes after.
LoRA
Low-Rank Adaptation. Freeze every original weight. Next to some of the weight matrices, bolt on two tiny matrices, and train only those.
QLoRA
LoRA where the frozen base is stored in 4 bits. That is why a 1.5B model trains in 3.5 GB.
adapter
The file that holds those tiny trained matrices. 21 MB. It is useless on its own and has to be loaded next to its base model.
fuse
Bake the adapter into the base weights, so you have one ordinary model folder to ship.
02 · step 1 · setup

Setup is one folder and one install.

The stack is MLX, Apple's array framework for Apple Silicon. On a Mac the CPU and GPU share one pool of unified memory. Every gigabyte of RAM is a gigabyte of GPU memory, and that is why a laptop can do this at all.

mlx-lm is the LLM toolkit built on top of it. One package loads models, generates text, trains LoRA adapters, fuses them and serves them. I pinned the version I used, so your output should match mine.

run this
mkdir finetune-lab && cd finetune-lab
uv venv --python 3.12
source .venv/bin/activate
uv pip install "mlx-lm==0.31.3"
python -c "import mlx_lm, mlx.core as mx; print(mlx_lm.__version__, mx.default_device())"
you should see
0.31.3 Device(gpu, 0)

Then grab the two scripts. You will read both of them properly in the next two steps.

run this
curl -LO https://www.heqinghuang.com/finetune-lab/make_data.py
curl -LO https://www.heqinghuang.com/finetune-lab/evaluate.py
01why the model name ends in -4bitthe Q in QLoRA+

A 3 GB model in 0.9 GB

Every script uses mlx-community/Qwen2.5-1.5B-Instruct-4bit. That is Qwen's model with its weights already squeezed from 16 bits down to 4. At 1.54 billion weights, 16 bits is about 3.1 GB. 4 bits, plus a little overhead, is the 0.9 GB you download.

Those 4-bit weights stay frozen. The small LoRA matrices you train sit next to them in full 32-bit precision, so training never updates the lossy 4-bit part.

In the original QLoRA paper, 4-bit plus LoRA matched full 16-bit fine-tuning. That result is why this recipe is the default for small budgets.

02what a 4-bit weight isgroups of 64, a scale and a bias+

Sixteen levels per weight

Four bits can count from 0 to 15. MLX cuts each weight row into groups of 64 and stores one scale and one bias per group. Every weight is rebuilt on the fly from its 4-bit integer. This is straight from the model's own config.json:

config.json · the quantization block
"quantization": {"group_size": 64, "bits": 4}
rebuilding one weight
# q is a 4-bit integer 0..15. scale and bias are shared by 64 neighbours.
w ≈ scale * q + bias
# cost: 4 bits per weight + (16 + 16) bits per 64 weights ≈ 4.5 bits/weight
03 · step 2 · the data

Six hundred examples, written by a script.

On a real project, this is where most of the time goes: collecting examples, cleaning them, checking every label. Rahul's guide spends six of its fourteen steps here.

I cheated, on purpose. A Python script invents a messy message and its exact answer at the same time, so every label is free and correct. That is great for learning the mechanics. It also plants a trap we will walk straight into in step 6.

run this
python make_data.py
you should see
wrote 600 rows -> data/train.jsonl
wrote  60 rows -> data/valid.jsonl
wrote 100 rows -> data/test.jsonl

Here is the whole script. It is worth a slow read. Every choice in TEMPLATES, say_time and say_duration sets the limits of what the model can learn.

make_data.py
"""Build a tiny synthetic dataset: messy meeting requests -> strict JSON.

Writes data/train.jsonl, data/valid.jsonl, data/test.jsonl in the
chat "messages" format that mlx_lm.lora reads directly.
"""
import json
import random
from pathlib import Path

SYSTEM = (
    "Extract the meeting from the message. Reply with JSON only: "
    '{"title": str, "day": str, "time": "HH:MM", "duration_min": int, "people": [str]}'
)

TITLES = ["design sync", "1:1", "roadmap review", "standup", "budget check-in",
          "hiring debrief", "launch retro", "coffee chat", "sprint planning",
          "customer call", "offsite prep", "interview loop", "demo run-through",
          "pricing review", "onboarding walkthrough"]
PEOPLE = ["Priya", "Tom", "Aiko", "Marcus", "Lena", "Diego", "Sam", "Wei",
          "Fatima", "Jonas", "Ravi", "Chloe", "Omar", "Hana", "Leo", "Nia"]
DAYS = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"]
DAY_SAY = {d: [d, d.lower(), d[:3], d[:3].lower(), "next " + d.lower()] for d in DAYS}


def say_time(h, m):
    """Many ways a human writes a time. The label is always 24h HH:MM."""
    h12 = h % 12 or 12
    ampm = "am" if h < 12 else "pm"
    opts = [f"{h12}{ampm}" if m == 0 else f"{h12}:{m:02d}{ampm}",
            f"{h12}:{m:02d} {ampm.upper()}",
            f"{h:02d}:{m:02d}"]
    if m == 30:
        opts.append(f"half past {h12}")
    if h == 12 and m == 0:
        opts.append("noon")
    return random.choice(opts)


def say_duration(mins):
    if mins is None:
        return ""
    opts = {15: ["15 min", "a quick 15"], 30: ["30 mins", "half an hour"],
            45: ["45 minutes", "45m"], 60: ["an hour", "1h", "60 min"],
            90: ["90 minutes", "an hour and a half"]}[mins]
    return random.choice(opts)


def say_people(ppl):
    if len(ppl) == 1:
        return ppl[0]
    return ", ".join(ppl[:-1]) + random.choice([" and ", " & "]) + ppl[-1]


TEMPLATES = [
    "can we do the {title} with {people} on {day} at {time}{dur_sfx}?",
    "{people} + me, {title}, {day} {time}{dur_sfx}",
    "hey! booking {title} for {day} @ {time}{dur_sfx}. inviting {people}",
    "Reminder: {title} {day} {time}{dur_sfx} w/ {people}. pls confirm",
    "move the {title} to {day}, {time}{dur_sfx}. {people} can make it",
    "{day} {time} works for the {title}{dur_sfx}? adding {people}",
    "ok so {people} want a {title}. thinking {day} around {time}{dur_sfx}",
]


def example():
    title = random.choice(TITLES)
    ppl = random.sample(PEOPLE, random.randint(1, 3))
    day = random.choice(DAYS)
    h, m = random.choice(range(8, 18)), random.choice([0, 0, 15, 30, 45])
    mins = random.choice([15, 30, 45, 60, 90, None])
    dur = say_duration(mins)
    text = random.choice(TEMPLATES).format(
        title=title, people=say_people(ppl), day=random.choice(DAY_SAY[day]),
        time=say_time(h, m), dur_sfx=f", {dur}" if dur else "")
    label = {"title": title, "day": day, "time": f"{h:02d}:{m:02d}",
             "duration_min": mins or 30,  # house rule: no duration -> 30
             "people": ppl}
    return {"messages": [
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": text},
        {"role": "assistant", "content": json.dumps(label)},
    ]}


def main():
    random.seed(0)
    out = Path("data")
    out.mkdir(exist_ok=True)
    for name, n in [("train", 600), ("valid", 60), ("test", 100)]:
        with open(out / f"{name}.jsonl", "w") as f:
            for _ in range(n):
                f.write(json.dumps(example()) + "\n")
        print(f"wrote {n:>3} rows -> data/{name}.jsonl")


if __name__ == "__main__":
    main()
train.jsonlthe first of 600 rows
{
  "messages": [
    {
      "role": "system",
      "content": "Extract the meeting from the message. Reply with JSON only: {\"title\": str, \"day\": str, \"time\": \"HH:MM\", \"duration_min\": int, \"people\": [str]}"
    },
    {
      "role": "user",
      "content": "Reminder: pricing review next wednesday 4:30 PM, 1h w/ Hana & Priya. pls confirm"
    },
    {
      "role": "assistant",
      "content": "{\"title\": \"pricing review\", \"day\": \"Wednesday\", \"time\": \"16:30\", \"duration_min\": 60, \"people\": [\"Hana\", \"Priya\"]}"
    }
  ]
}
This is the messages format: system, user, assistant. mlx_lm.lora reads it directly. The system prompt is the same in every row, so the model is trained on the exact prompt it will see at inference time.
01what the model actually readsone string, and which parts get graded+

The chat template

A model never sees JSON with roles. The tokenizer's chat template flattens the three messages into one string, with special tokens marking where each turn starts and ends. This is Qwen's real rendering of the row above:

<|im_start|>system
Extract the meeting from the message. Reply with JSON only: {"title": str, "day": str, "time": "HH:MM", "duration_min": int, "people": [str]}<|im_end|>
<|im_start|>user
Reminder: pricing review next wednesday 4:30 PM, 1h w/ Hana & Priya. pls confirm<|im_end|>
<|im_start|>assistant
{"title": "pricing review", "day": "Wednesday", "time": "16:30", "duration_min": 60, "people": ["Hana", "Priya"]}<|im_end|>

Grade the answer, not the question

The row is 126 tokens long. With the --mask-prompt flag you will pass in step 4, only the last 44 count toward the loss: the JSON answer and the closing <|im_end|>. Grading that stop token is how the model learns to stop after the JSON instead of rambling on.

Without the mask, most of the gradient would go into predicting the system prompt, which is identical in every row, and the user's message, which is random by design. Neither teaches the task.

One more detail: Qwen's tokenizer splits "16:30" into five tokens, 1 6 : 3 0. The model writes times digit by digit, and each digit is a separate chance to get the 12-to-24-hour conversion wrong.

02three files, three jobstrain, valid, and the one you never touch+

Train, valid, test

train.jsonl · 600
What the model learns from. 300 steps × 4 rows per step = 1,200 rows seen, which is two passes (two epochs).
valid.jsonl · 60
Scored every 50 steps during training. The model never learns from it. It is your early warning.
test.jsonl · 100
Frozen. Scored once before training and once after. If you tune anything while looking at it, it stops being a test.
the trap, plantedAll three files come from the same script, with the same 7 sentence templates and the same 15 titles. A model can ace all three without understanding meetings at all. We will come back to this in step 6.
04 · step 3 · the baseline

Score the prompt before you train.

This is the most skipped step in fine-tuning. If you never measured the prompt-only model, you cannot know whether training did anything.

The base model gets a fair shot: every test row carries the system prompt that spells out the JSON schema. The first run downloads the model, about 0.9 GB, into ~/.cache/huggingface.

run this
python evaluate.py none
you should see · the misses first, then the score
msg : can we do the sprint planning with Wei, Chloe & Nia on Mon at 8:45 AM, 60 min?
got : {"title": "Sprint Planning Meeting", "day": "Monday", "time": "08:45", "duration_min": 60, "people": ["Wei", "Chloe", "Nia"]}
want: {"title": "sprint planning", "day": "Monday", "time": "08:45", "duration_min": 60, "people": ["Wei", "Chloe", "Nia"]}

msg : ok so Jonas, Omar and Nia want a coffee chat. thinking Fri around 09:00, a quick 15
got : {"title": "Coffee Chat", "day": "Friday", "time": "09:00", "duration_min": 15, "people": ["Jonas", "Omar", "Nia"]}
want: {"title": "coffee chat", "day": "Friday", "time": "09:00", "duration_min": 15, "people": ["Jonas", "Omar", "Nia"]}

msg : hey! booking onboarding walkthrough for friday @ 3pm, 60 min. inviting Wei
got : {"title": "Onboarding Walkthrough", "day": "Friday", "time": "15:00", "duration_min": 60, "people": ["Wei"]}
want: {"title": "onboarding walkthrough", "day": "Friday", "time": "15:00", "duration_min": 60, "people": ["Wei"]}

msg : ok so Nia & Leo want a 1:1. thinking next friday around 8:30 AM, half an hour
got : {"title": "Nia & Leo 1:1", "day": "Friday", "time": "08:30", "duration_min": 30, "people": ["Nia", "Leo"]}
want: {"title": "1:1", "day": "Friday", "time": "08:30", "duration_min": 30, "people": ["Nia", "Leo"]}

msg : can we do the demo run-through with Tom and Fatima on Monday at 16:00, an hour and a half?
got : {"title": "Demo Run-Through", "day": "Monday", "time": "16:00", "duration_min": 90, "people": ["Tom", "Fatima"]}
want: {"title": "demo run-through", "day": "Monday", "time": "16:00", "duration_min": 90, "people": ["Tom", "Fatima"]}

valid JSON : 98/100
exact match: 2/100
  title         6/100
  day           78/100
  time          68/100
  duration_min  75/100
  people        87/100

It is a capable model. The JSON parses 98 times out of 100, and most fields are right. But it does not know my rules. It writes titles in Title Case and adds "Meeting". When I give no duration, it guesses 0 or 60 instead of 30.

Every one of those choices is reasonable, but none of them follows my rules. 2 exact answers out of 100 is the number to beat.

02the grader, line by lineevaluate.py, all of it+

Forty lines that do the grading

evaluate.py
"""Score the model on a test file, with or without your LoRA adapter.

usage:
  python evaluate.py none                       # base model, prompt only
  python evaluate.py adapters                   # base model + your adapter
  python evaluate.py adapters data/wild.jsonl   # any test file
"""
import json
import sys

from mlx_lm import generate, load

MODEL = "mlx-community/Qwen2.5-1.5B-Instruct-4bit"
FIELDS = ["title", "day", "time", "duration_min", "people"]
SHOW_MISSES = 5


def main():
    adapter = sys.argv[1] if len(sys.argv) > 1 else "none"
    path = sys.argv[2] if len(sys.argv) > 2 else "data/test.jsonl"
    model, tok = load(MODEL, adapter_path=None if adapter == "none" else adapter)
    rows = [json.loads(line) for line in open(path)]

    valid = exact = shown = 0
    field_hits = dict.fromkeys(FIELDS, 0)
    for row in rows:
        # system + user only: the assistant turn is the answer key
        prompt = tok.apply_chat_template(
            row["messages"][:2], add_generation_prompt=True, tokenize=False)
        out = generate(model, tok, prompt=prompt, max_tokens=120).strip()
        gold = json.loads(row["messages"][2]["content"])
        try:
            pred = json.loads(out)
            valid += 1
        except json.JSONDecodeError:
            pred = {}
        if not isinstance(pred, dict):
            pred = {}

        hits = {f: pred.get(f) == gold[f] for f in FIELDS}
        for f in FIELDS:
            field_hits[f] += hits[f]
        if all(hits.values()):
            exact += 1
        elif shown < SHOW_MISSES:
            shown += 1
            print(f"\nmsg : {row['messages'][1]['content']}")
            print(f"got : {out}")
            print(f"want: {json.dumps(gold)}")

    n = len(rows)
    print(f"\nvalid JSON : {valid}/{n}")
    print(f"exact match: {exact}/{n}")
    for f in FIELDS:
        print(f"  {f:<13} {field_hits[f]}/{n}")


if __name__ == "__main__":
    main()
row["messages"][:2]
System and user only. The assistant turn is the answer key, so the model must never see it.
add_generation_prompt=True
Ends the string with <|im_start|>assistant, so the model's next token begins its answer. That is the same boundary the loss mask uses in training.
generate(...)
With no sampler passed, mlx-lm decodes greedily: it always picks the most likely token. The same input always gives the same output, which is why your numbers should match mine exactly.
exact match
All five fields must match. That is the strict metric. The per-field counts show where it fails, and that is what tells you what data to add later.
05 · step 4 · train

Train 0.34% of the model for two minutes.

The fine-tune itself is one command.

run this
mlx_lm.lora \
  --model mlx-community/Qwen2.5-1.5B-Instruct-4bit \
  --train --data data \
  --mask-prompt \
  --batch-size 4 --iters 300 --learning-rate 1e-4 \
  --num-layers 16 \
  --steps-per-report 10 --steps-per-eval 50 \
  --adapter-path adapters
you should see · trimmed
Loading pretrained model
Loading datasets
Training
Trainable parameters: 0.342% (5.276M/1543.714M)
Starting training..., iters: 300
Iter 1: Val loss 0.758, Val took 2.504s
Iter 10: Train loss 0.127, … Trained Tokens 1780, Peak mem 3.498 GB
Iter 20: Train loss 0.105, … Trained Tokens 3580, Peak mem 3.498 GB
Iter 30: Train loss 0.027, … Trained Tokens 5368, Peak mem 3.498 GB
…
Iter 50: Val loss 0.008, Val took 1.795s
Iter 50: Train loss 0.004, … Trained Tokens 8975, Peak mem 3.498 GB
Iter 100: Val loss 0.006, Val took 1.829s
Iter 100: Train loss 0.001, … Trained Tokens 17928, Peak mem 3.498 GB
Iter 110: Train loss 0.000, … Trained Tokens 19716, Peak mem 3.519 GB
…
Iter 200: Val loss 0.000, Val took 1.822s
Iter 200: Train loss 0.000, … Trained Tokens 35922, Peak mem 3.519 GB
Iter 300: Val loss 0.000, Val took 1.789s
Iter 300: Train loss 0.000, … Trained Tokens 53884, Peak mem 3.520 GB
Saved final weights to adapters/adapters.safetensors.

Each flag, in plain words:

--model …-4bit
The frozen base. Because it is 4-bit, this LoRA run is technically QLoRA.
--data data
A folder holding train.jsonl and valid.jsonl.
--mask-prompt
Grade only the answer tokens (step 2). If you leave it off, training still runs but learns worse, and nothing warns you.
--batch-size 4
Four rows per step. Gradients are averaged across them, so one weird row cannot yank the weights around.
--iters 300
300 optimizer steps × 4 = 1,200 rows = two epochs of 600.
--learning-rate 1e-4
How far each step moves the adapter. LoRA runs usually use a value between 1e-4 and 2e-4.
--num-layers 16
Qwen2.5-1.5B has 28 transformer layers. Adapters go on the last 16 (layers 12 to 27). The first 12 stay completely untouched.
--adapter-path adapters
Where the 21 MB result is saved, along with a checkpoint every 100 steps.
01what LoRA doestwo thin matrices beside a frozen one+

A detour around every frozen matrix

Take one weight matrix in the model, say the attention query projection. It maps a 1,536-wide vector to another 1,536-wide vector: 2.36 million frozen numbers. LoRA leaves it alone and adds a side path. The input goes down through A to just 8 numbers, back up through B, gets multiplied by a scale, and is added to the frozen output.

ONE LINEAR LAYER · q_proj IN LAYER 12 x 1536 W · frozen, 4-bit 1536 × 1536 = 2,359,296 A · train 1536 × 8 8 wide B · train 8 × 1536 starts at all zeros × 20 + y 12,288 + 12,288 = 24,576 trainable about 1% of W's size

LoRA bets that the change a fine-tune needs is simple, even when the matrix is not. You do not need 2.36 million new numbers to teach "lowercase the title". A rank-8 correction, 24,576 numbers, is enough.

Step zero is the base model, exactly

A starts as small random numbers. B starts as all zeros. Zero times anything is zero, so before the first update the side path adds nothing, and the model is exactly the base model. Training starts from the model you measured in step 3, not from noise. This is mlx-lm's code, trimmed:

mlx_lm/tuner/lora.py · LoRALinear
# __init__
scale = 1 / math.sqrt(input_dims)
self.lora_a = mx.random.uniform(low=-scale, high=scale, shape=(input_dims, r))
self.lora_b = mx.zeros(shape=(r, output_dims))

# __call__
y = self.linear(x)                                  # the frozen 4-bit matmul
z = (self.dropout(x) @ self.lora_a) @ self.lora_b    # down to 8, back up
return y + (self.scale * z).astype(x.dtype)          # scale defaults to 20.0
02counting the 5,275,648an interactive parameter calculator+

Where every trainable number lives

The training log said Trainable parameters: 0.342% (5.276M/1543.714M). Here is the whole count. Each of the 16 adapted layers has seven linear matrices, and each gets an A and a B. Move the sliders to see what rank and depth cost.

8
16
5,275,648
trainable
0.342%
of the model
21.1 MB
adapter file · fp32

At the defaults you get 16 × 329,728 = 5,275,648, which matches the log. At 4 bytes each that is 21.1 MB, and the file on disk is 21,126,646 bytes. Nothing on this page is estimated.

The shapes also give away the architecture. k_proj and v_proj output only 256, not 1,536, because Qwen2.5 uses grouped-query attention. And the three MLP matrices hold 76% of the adapter, because the MLP is 8,960 wide.

03the loss functioncross-entropy, masked+

Eleven lines

This is the whole training objective, from mlx-lm. Shift the tokens by one so every position predicts the next token. Take the cross-entropy at every position. Zero out everything outside the answer. Average over the tokens that are left.

mlx_lm/tuner/trainer.py · default_loss
def default_loss(model, batch, lengths):
    inputs = batch[:, :-1]
    targets = batch[:, 1:]                       # predict the NEXT token
    logits = model(inputs)

    steps = mx.arange(1, targets.shape[1] + 1)
    mask = mx.logical_and(steps >= lengths[:, 0:1], steps <= lengths[:, 1:])
                                                  # lengths[:,0] = where the answer starts
    ce = nn.losses.cross_entropy(logits, targets) * mask
    ntoks = mask.sum()
    ce = ce.astype(mx.float32).sum() / ntoks
    return ce, ntoks

Cross-entropy is -log(p), where p is the probability the model gave the correct token. A loss of 0.758 means an average correct-token probability of about e-0.758 ≈ 0.47. A loss of 0.008 means 0.992. Keep that conversion handy for the next step.

The log's "Trained Tokens" counter proves the mask works: 53,884 tokens over 1,200 rows is about 45 per row. That matches the 44 answer tokens per row from step 2, not the 126 in the full row.

06 · step 5 · read the loss

When the loss hits zero, worry.

These are the real numbers from both of my runs. v2 is the retrain you will do in step 7. The y-axis is logarithmic. Otherwise everything after step 30 would be squashed flat against the bottom.

train and validation loss · log scale

Validation loss falls from 0.758 to 0.008 by step 50. Training loss prints 0.000 at step 110 and stays there. Zero means the model assigns near-certain probability to every answer token.

On real data, that does not happen. People write ambiguous things, and a model that is certain about all of it has memorized something. Here the reason is simple. My generator has only 7 sentence templates and 15 titles, so the model did not learn how people describe meetings. It learned my script.

And validation could not catch it, because the validation rows came from the same script. A validation set only tells you about the distribution it was drawn from. Mine was drawn from the same seven templates.

01reading any loss curvefour shapes, four diagnoses+

Four shapes

both fall, then flatten
Healthy. Stop near where validation flattens. Rahul's rule: do not add epochs just because training loss is still falling.
train falls, val rises
Classic overfitting. The model is memorizing the training rows. Use an earlier checkpoint (adapters/0000100_adapters.safetensors), or add data.
both hit ~0
My case. The task, or your data, is too easy or too repetitive. The loss curve has nothing more to tell you, and only a harder test can.
spikes, or NaN
Learning rate too high. Halve it.
07 · step 6 · the honest test

Then write ten messages by hand.

First, the frozen test set.

run this
python evaluate.py adapters
you should see
valid JSON : 100/100
exact match: 100/100
  title         100/100
  day           100/100
  time          100/100
  duration_min  100/100
  people        100/100

Exact matches went from 2 to 100. It would be easy to stop here, post the chart, and call it a success.

But that test came from the same script as the training data. So I wrote ten messages the way people actually write them: new names, new meeting types, phrasings my templates never produce. Write your own ten before you look at mine. Then run both.

run this
curl -Lo data/wild.jsonl https://www.heqinghuang.com/finetune-lab/wild.jsonl
python evaluate.py none data/wild.jsonl
python evaluate.py adapters data/wild.jsonl
you should see · the two scores
# base, prompt only
valid JSON : 10/10
exact match: 0/10

# with your adapter
valid JSON : 10/10
exact match: 5/10

The adapter scores 100 out of 100 on the easy test, but only 5 out of 10 on messages written by a person. The second number is much closer to the truth. Step 7 will show that even this number is less solid than it looks.

replay · the recorded outputs from my run
Decoding is greedy, so these are exactly the outputs you will get. Green matches my label. Red does not, and the expected value is shown underneath. Messages marked wild2 belong to step 7. The v2 button shows the fixed v2 (seed 0).

Click through the v1 misses and every one traces back to the generator.

"Two hours."
v1 says 60. The generator's longest meeting is 90 minutes, so 120 was never an answer it saw.
"quarter past ten"
v1 says 10:30. The generator says "half past" but never "quarter past".
"Me, Ana, Viktor, Mei"
v1 puts "Me" in the list of people. In training, "me" only ever appeared in one fixed template.
"tacos + brainstorm fri"
v1 invents the title "brainstorm fri". The generator never puts the title right next to the day.

The adapter learned my rules, but only for the kinds of input the data covered.

08 · step 7 · iterate

Fix the data, not the model.

Every miss pointed to a gap in the generator, so the fix goes into the generator. Here is the whole diff: two-hour meetings, "quarter past" and "quarter to", and five new templates.

make_data.py → make_data_v2.py
--- make_data.py
+++ make_data_v2.py
@@ -33,2 +33,6 @@
         opts.append(f"half past {h12}")
+    if m == 15:
+        opts.append(f"quarter past {h12}")
+    if m == 45:
+        opts.append(f"quarter to {(h + 1) % 12 or 12}")
     if h == 12 and m == 0:
@@ -43,3 +47,4 @@
             45: ["45 minutes", "45m"], 60: ["an hour", "1h", "60 min"],
-            90: ["90 minutes", "an hour and a half"]}[mins]
+            90: ["90 minutes", "an hour and a half"],
+            120: ["2 hours", "two hours", "120 min"]}[mins]
     return random.choice(opts)
@@ -61,2 +66,7 @@
     "ok so {people} want a {title}. thinking {day} around {time}{dur_sfx}",
+    "{Title}. {Day}. {time}. Me, {people}.{dur_sfx_sent}",
+    "Could you set up a {title} for {day} at {time}? Bring in {people}.{dur_sfx_sent}",
+    "Rescheduling our {title} to {day} {time}. Attendees: {people}.{dur_sfx_sent}",
+    "{title} {day} {time} ({dur_paren}) - {people}",
+    "Can {people} and I do a {title} {day} at {time}{dur_sfx}?",
 ]
@@ -69,7 +79,11 @@
     h, m = random.choice(range(8, 18)), random.choice([0, 0, 15, 30, 45])
-    mins = random.choice([15, 30, 45, 60, 90, None])
+    mins = random.choice([15, 30, 45, 60, 90, 120, None])
     dur = say_duration(mins)
+    say_day = random.choice(DAY_SAY[day])
     text = random.choice(TEMPLATES).format(
-        title=title, people=say_people(ppl), day=random.choice(DAY_SAY[day]),
-        time=say_time(h, m), dur_sfx=f", {dur}" if dur else "")
+        title=title, Title=title.capitalize(), people=say_people(ppl),
+        day=say_day, Day=say_day.capitalize(), time=say_time(h, m),
+        dur_sfx=f", {dur}" if dur else "",
+        dur_sfx_sent=f" {dur.capitalize()}." if dur else "",
+        dur_paren=dur or "30 min")
     label = {"title": title, "day": day, "time": f"{h:02d}:{m:02d}",
the trap, sprungLook at those new templates: "Could you set up a … Bring in …" and "Rescheduling our … Attendees: …". I copied the phrasings of the hand-written messages that failed. So from now on wild.jsonl is not a test anymore, because the training data now contains its phrasings. Any score on it will make the model look better than it is.

So before training v2, I wrote a second set of ten, wild2.jsonl. It has new people, new meeting types, and phrasings that neither generator produces: "1.5 hrs", "2:15p", "pencil in". Only wild2 is a fair test of v2.

run this
curl -LO https://www.heqinghuang.com/finetune-lab/make_data_v2.py
curl -Lo data/wild2.jsonl https://www.heqinghuang.com/finetune-lab/wild2.jsonl
python make_data_v2.py          # overwrites data/train, valid, test
mlx_lm.lora --model mlx-community/Qwen2.5-1.5B-Instruct-4bit \
  --train --data data --mask-prompt \
  --batch-size 4 --iters 300 --learning-rate 1e-4 --num-layers 16 \
  --steps-per-report 10 --steps-per-eval 50 \
  --adapter-path adapters_v2
python evaluate.py adapters    data/wild2.jsonl
python evaluate.py adapters_v2 data/wild2.jsonl
you should see · the two scores
# v1 adapter
valid JSON : 10/10
exact match: 5/10

# v2 adapter
valid JSON : 10/10
exact match: 5/10

Five and five. So the new data did nothing?

That is not what I saw the first time. My first v2 scored 7 out of 10, and I had already typed "v2 wins" into this page. Then I read v2's misses on its own test split. Three of the four looked like this:

msg : Can Lena and Wei and I do a roadmap review next thursday at 4:15 PM?
got : {..., "duration_min": 30, ...}
want: {..., "duration_min": 120, ...}

The message gives no duration, so by my own rule the answer is 30. The model was right, and my label was wrong. My new "Can … and I do …" template left the duration out of the message, but the label still used a random one. Every row from that template taught the model to make up a meeting length. The make_data_v2.py you downloaded has the fix, so the v2 you just trained is the fixed one.

With the bug fixed, wild2 went from 7 down to 5. So did fixing a bug make the model worse?

08 · step 7, continued · the noise floor

Ten examples can't tell 5 from 7.

Ten messages could not answer that, so I did two things.

First, I wrote twenty more fresh messages (wild3.jsonl) before training anything else. Together with wild2, that makes 30 messages neither model has seen in any form.

Second, I trained each version three times. The data and flags stayed the same, and only --seed changed. The seed sets two things and nothing else: the random starting values of every A matrix, and the order the rows get shuffled in.

run this · optional, about 12 minutes
curl -Lo data/wild3.jsonl https://www.heqinghuang.com/finetune-lab/wild3.jsonl
cat data/wild2.jsonl data/wild3.jsonl > data/fresh30.jsonl
for s in 1 2; do
  python make_data.py
  mlx_lm.lora --model mlx-community/Qwen2.5-1.5B-Instruct-4bit --train --data data \
    --mask-prompt --batch-size 4 --iters 300 --learning-rate 1e-4 --num-layers 16 \
    --seed $s --adapter-path v1_s$s
  python make_data_v2.py
  mlx_lm.lora --model mlx-community/Qwen2.5-1.5B-Instruct-4bit --train --data data \
    --mask-prompt --batch-size 4 --iters 300 --learning-rate 1e-4 --num-layers 16 \
    --seed $s --adapter-path v2_s$s
done
for a in none adapters v1_s1 v1_s2 adapters_v2 v2_s1 v2_s2; do
  echo $a; python evaluate.py $a data/fresh30.jsonl | grep exact
done
exact out of 30 freshseed 0seed 1seed 2mean
base · prompt only1··1
v1 adapter20121816.7
v2 adapter16222119.7
Seed 0 is the adapter you trained above. The base model decodes greedily and has no training, so it has only one score.

Read the v1 row first. With the same data and the same code, the score goes from 12 to 20 only because the seed changed. That is 8 points of noise, bigger than the gap between v1 and v2.

On average v2 is ahead, 19.7 to 16.7, and it wins two seeds out of three. But if I had only run seed 0, which is what almost everyone does, I would have concluded the opposite and thrown away the better data.

So the rule I took away: never compare two single runs on a small test. Use more test examples than feels necessary, train with a few seeds, and report the range, not just the best number.

What v2 still gets wrong is the to-do list for v3. Pick the messages in the replay above with the v2 button:

"Two hours."
Still 60 on the board-prep message, even though I copied that exact phrasing into the templates. Contaminating your test does not even guarantee a fix.
"1.5 hrs"
v2 says 150. It saw "2 hours" become 120 and generalized the wrong way. Neither generator ever writes hours as a decimal.
"quarter to three"
13:45 instead of 14:45. "Quarter to" means subtracting from the next hour, and that is harder than it looks.
titles
"pair programming" vs. my label "pair programming session". The model's answer is fine here. Sometimes the label is the bug. A rule like "the words the sender used" needs examples at the edges, or the labeler and the model will draw the line in different places.
the actual jobRead the misses. Fix the data, including your own labels. Write a fresh test. Train more than once. The training command never changed. Everything that made the model better, and everything that fooled me, was in the data and the measurement.
09 · step 8 · ship it

From a 21 MB file to an API.

The quickest check: load the adapter next to the base model and ask it something new.

run this
mlx_lm.generate \
  --model mlx-community/Qwen2.5-1.5B-Instruct-4bit \
  --adapter-path adapters_v2 \
  --system-prompt "$(python -c 'from make_data import SYSTEM; print(SYSTEM)')" \
  --prompt "lunch w/ Hugo, Friday 1pm, 45 min"
you should see
==========
{"title": "lunch", "day": "Friday", "time": "13:00", "duration_min": 45, "people": ["Hugo"]}
==========
Prompt: 70 tokens, 1074.725 tokens-per-sec
Generation: 39 tokens, 141.889 tokens-per-sec
Peak memory: 1.112 GB

To ship it, fuse it: fold scale · A · B into each frozen matrix, so the result is one ordinary model folder that needs no adapter. One gotcha I hit: the model loader skips files like README.md when it downloads, but fuse wants the complete snapshot and fails without network access. Download everything first.

run this
hf download mlx-community/Qwen2.5-1.5B-Instruct-4bit
mlx_lm.fuse --model mlx-community/Qwen2.5-1.5B-Instruct-4bit \
  --adapter-path adapters_v2 --save-path fused
du -sh fused adapters_v2/adapters.safetensors
you should see
839M	fused
 20M	adapters_v2/adapters.safetensors

Then serve it. mlx_lm.server implements the OpenAI chat API, so any OpenAI SDK, or curl, can talk to your model.

run this · terminal 1
mlx_lm.server --model ./fused --port 8080
run this · terminal 2
curl -s localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"temperature": 0, "max_tokens": 80, "messages": [
    {"role": "system", "content": "Extract the meeting from the message. Reply with JSON only: {\"title\": str, \"day\": str, \"time\": \"HH:MM\", \"duration_min\": int, \"people\": [str]}"},
    {"role": "user", "content": "retro thursday 4:30pm with Jun and Ama"}]}' \
  | python -c "import json,sys; print(json.load(sys.stdin)['choices'][0]['message']['content'])"
you should see
{"title": "retro", "day": "Thursday", "time": "16:30", "duration_min": 30, "people": ["Jun", "Ama"]}

You now have a model you trained, answering on localhost, with the house rules built in. The message gave no duration, so it answered 30 without being told.

Putting it on the public web is a different job. Vercel functions have no GPU and cannot hold a 0.9 GB model in memory, so this site cannot run it. The replay above is recorded output, not a live model. The real options, cheapest first:

share the adapter
hf upload your-name/meeting-json-lora adapters_v2. The adapter is 21 MB, and anyone with the base model can load it.
a GPU box
vLLM with --enable-lora on Modal, RunPod or similar, behind auth, a rate limit and a queue. Rahul's guide is firm on this: never expose the raw model server.
the browser
Run a smaller model (0.5B or less) on the visitor's own GPU with WebGPU. There is no server bill, but the first download is big, and you need a conversion step that mlx-lm does not do.
10 · the same recipe, elsewhere

No Mac? The same recipe on a GPU.

Rahul's guide runs this recipe with Hugging Face's stack on a free Colab T4: transformers + bitsandbytes for the 4-bit base, peft for LoRA, trl's SFTTrainer for the loop. Every concept above carries over. Only the names change.

mlx-lmHugging Face stack
--model …-4bitBitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4")
rank 8, 7 matricesLoraConfig(r=8, target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"])
--num-layers 16LoraConfig(layers_to_transform=list(range(12, 28)))
--mask-promptSFTConfig(assistant_only_loss=True)
--batch-size 4 --iters 300per_device_train_batch_size=4, max_steps=300
--learning-rate 1e-4learning_rate=1e-4
adapters.safetensorsadapter_model.safetensors
mlx_lm.fusemodel.merge_and_unload()
mlx_lm.servervllm serve … --enable-lora
untestedI did not run the Hugging Face column for this page. It is a translation, not a tested recipe. Check the TRL SFTTrainer docs before you rely on it.

One translation is a real trap. PEFT scales the LoRA path by lora_alpha / r. mlx-lm uses a flat scale, 20 by default. A common PEFT setting, r=16 with alpha=32, comes out to a scale of 2. MLX's default is ten times larger. Both work, but they need different learning rates. If you copy hyperparameters between the two stacks, convert the scale first.

11 · closing

Training takes two minutes. The real work is the data.

What surprised me was how little of this was about the model. The training command never changed. It ran in two minutes, on a laptop, and taught the model every rule I had. All the real work was deciding what "right" means, writing examples that cover it, and building a test the model could not have memorized.

Your turn. Some ways to break it on purpose:

smaller model
Swap in mlx-community/Qwen2.5-0.5B-Instruct-4bit in all three places. How does a model a third the size score on the 30 fresh messages?
rank and depth
Rank is not a flag. Put lora_parameters: {rank: 32, scale: 20.0, dropout: 0.0} in a YAML file and pass -c config.yaml. Also try --num-layers 4. Use the calculator first to predict the adapter size.
no mask
Drop --mask-prompt and compare the loss curve and the wild2 score.
your own task
Replace make_data.py with 300 real examples of something you do by hand every week. That is the version that matters.

The model learns what your data teaches it, and nothing more.

the seed and the tools
the papers, and this page's files