The generic model misses your language.
It writes plausible answers in your domain and gets the details wrong: your schema, your terminology, your edge cases.
L1 Models / fine-tuning measured proof
A smaller model that does your job.
A frontier API is the right answer until it is too slow, too expensive, or not allowed to see the data. Then the model has to change. I fine-tune open-weight models on your cases, distil a large model into a small one, and prove the result against a baseline you already trust.
4.5B
compass model, in training
1×
RTX 4090 training box
3
methods: LoRA, distillation, RL
§01 the symptom n=3
How this usually shows up.
The API bill is the product margin.
Every request carries frontier-model pricing. A smaller model that passes your cases can run at a fixed cost on your own hardware.
Prompting has stopped improving.
You have tuned the prompt for weeks. The pass rate moved two points. The next gain has to come from the weights.
§02 what I change n=5
From a job description to weights.
- 01
Define the job
One model, one job, one eval. I write down what a correct answer looks like before any training starts.
- 02
Build the data
Your tickets, documents and corrections become a training set with held-out cases. Cleaning and deduplication usually matter more than the algorithm.
- 03
Fine-tune
LoRA or full fine-tune, depending on how far the model has to move and how much data you have. Every run is scored against the baseline.
- 04
Distil
A larger model generates the target behaviour; a smaller one learns it. The student is kept only if it passes the eval at the size you can serve.
- 05
Prove it
A model card with pass rate, failure modes, latency and memory. If it is not better than the baseline on your cases, it does not ship.
§03 measured proof
Numbers from my own runs.
4.5Bparams
target size for the compass model
1×4090
where the training and evals run
LoRA · distill · RL
three routes, chosen by the data
evalfirst
the gate every run has to pass
Source: the compass model in training and the open-source training tooling below.
§04 deliverables n=4
What you get.
- A training set built from your data, with held-out cases
- A fine-tuned or distilled model, with the exact training recipe
- A model card: pass rate, failure modes, latency and memory
- The eval suite and the pipeline, so your team can retrain it
The math behind this.
- 2026-10-27 LoRA and QLoRA math: rank, alpha, memory, and what actually trains
- 2026-10-27 Distillation: logit KL, temperature, and on-policy vs off-policy
- 2026-10-27 What training an LLM costs: 6ND FLOPs, 16 bytes per parameter, Chinchilla
- 2026-10-27 Tokenizers: BPE, byte-level models, and what a token costs
§05 answers n=5
Questions I get asked.
How much data do we need?
For LoRA on a narrow task, a few thousand clean examples can be enough. For a new behaviour or a new language, more. The first step is an inventory of what you already have; most teams have more usable data than they think, and less than they need.
LoRA or full fine-tune?
LoRA when the task is narrow and the base model is close. Full fine-tune when the behaviour has to change deeply, and you have the data and the compute to justify it.
Will fine-tuning stop hallucinations?
Not by itself. Fine-tuning teaches format, tone and task behaviour; it does not make a model truthful about facts it never learned. Retrieval and evals still do that job.
Can the result run on our hardware?
That is a design constraint, not an afterthought. I pick the target size from the GPU you can deploy on, then train or distil to fit it.
How long does it take?
Two to four weeks for a first fine-tune with an eval suite. Distillation and RL take longer, because each round has to be scored.
Prompting stopped moving the number?
Describe your problem30-minute technical call with the engineer who builds it. No sales team, no slides.