wall_01 quality
Not good enough.
You need an AI system that handles your tasks, whether you are starting a new build or improving one. I develop the model, harness and evaluations the job calls for.
AI Systems Engineer AI from the tokenizer up Marbella, ES · Local time
When the API isn’t enough.
I build custom AI: models, inference, agent harnesses and integrations. You work directly with the engineer who writes the code, measures the results and hands over what I build.
30-minute technical call with the engineer who builds it. No sales team, no slides.
- years writing software
- 28
- years writing software
- open-source repos
- 7
- open-source repos
- where I train and benchmark
- 1× 4090
- where I train and benchmark
$ git clone github.com/jtrefon/llama.cpp-turboq-mtp# fork of llama.cpp-turboq · TBQ4 KV · MTP# SWA stability in Gated DeltaNet · hot model loadingcontext 262K tokensKV cache 4.125 bpv, TBQ4hardware 1× RTX 4090✓ peak decode 82 tok/s
Open-source proof
§00github.com/jtrefon
§01 Three walls n=3
From an idea to a system that works for your needs.
wall_02 privacy
Not private enough.
The data can’t leave your building, your country, or your customer’s network. I build inference that runs where the data lives: on-prem, in your cloud account, or air-gapped.
wall_03 cost
Not cheap or fast enough.
The token bill grows faster than revenue, or latency kills the experience. I measure the workload and explore serving, quantisation or smaller models where they make sense.
If the API works and the bill is fine, keep the API. I’ll tell you that on the call.
§02 What I build L0 → L3
Models. Inference. Harnesses.
Every layer, top to bottom. I work across the layers your problem requires.
- L3
Harnesses
Custom agents with tools, integrations and evaluations for your workflow.
Custom agents with tools, integrations and evaluations for your workflow.
- tools
- memory
- provider routing
- guardrails
- human handoff
- evals
- L2
Inference
Inference built around your privacy, latency and cost requirements.
Inference built around your privacy, latency and cost requirements.
- serving
- quantisation
- KV cache
- MTP
- OpenAI-compatible API
- L1
Models
Fine-tuning, distillation, reinforcement learning, training from scratch.
Fine-tuning, distillation, reinforcement learning, training from scratch.
- tokenizer
- pre-training
- fine-tuning
- distillation
- RL
- L0
Hardware
Your GPUs, your cloud account, or an air-gapped rack.
Your GPUs, your cloud account, or an air-gapped rack.
- CUDA
- Metal
- on-prem
- air-gapped
§03 Built in your stack
Not everything is Python.
I’ve shipped agents in PHP inside a live app, a multi-provider agent platform in .NET with Angular and React, a harness in C++, an IDE in Swift, and tooling in Rust. The AI fits your architecture, not the other way round.
- C++ llama.cpp fork · amber
- Rust blt tokenizer · distiller
- Swift compass IDE
- Python training · llm-mini · lora
- C# .NET off-plan agent platform
- PHP inbound sales agents
- TypeScript React chat agent · Angular back office
§04 Selected work n=4
Selected builds and research.
- Harness handed over
Off-plan real estate, Dubai
Agentic sales platform: .NET core and API, Angular back office, React chat agent, multiple LLM providers routed through OpenRouter.
- .NET
- Angular
- React
- OpenRouter
- Harness handed over
Inbound sales agents, several markets
Agents written in PHP and deployed inside the client’s own application, wired to local scripts, SOPs and domain knowledge, running on the Gemini API. Built and handed over.
- PHP
- Gemini API
- Models ongoing · NDA
Document extraction, regulated accounting client
Vision-model extraction pipeline. Ongoing, under NDA.
- vision model
- Python
- Models training
compass model (in progress, open source)
A 4.5B agentic coding model: Qwen fine-tuned for terminal and agent work, then distilled and RL-tuned per language and stack, trained on one RTX 4090, to ship inside compass.
- Qwen
- RL
- distillation
- RTX 4090
§05 How I work pipeline
Teardown. Build. Hand over.
- 01 1–2 weeks
Teardown
A paid, scoped investigation after access and data are ready. I measure the problem and deliver findings, options and a proposed next scope.
out
- profile
- evals on your data
- written plan
- 02 scoped
Build
I deliver the harness, inference stack or model, with evals, benchmarks, runbook and handover. We agree what success looks like in writing before I start.
out
- code
- evals
- benchmarks
- runbook
- 03 on request
Come back when needed
Fixes and extensions after handover, on request.
out
- fixes
- extensions
I build it, prove it, and hand it over. You own it, not a dependency on me.
The first call establishes fit. If the scope is clear, I can quote a build directly; otherwise, we agree a teardown first.
§06 Who builds it
I write the code.
I’m Jacek Trefon. 28 years writing software, years as CTO and Head of Engineering, and the last few building models, inference engines and harnesses. I work alone, on purpose: you talk to the person who writes the code.
- years writing software
- 28
- years writing software
- engineers led at CAFU
- 100+
- engineers led at CAFU
- regions, one platform at Logiswift
- 4
- regions, one platform at Logiswift
§07 articles ~/articles
Written from first principles.
- Harnesses10 minAgent context management: budgets, truncation, compaction, and knowing when to stopA 262K context isn't a licence to dump everything in. Each token of context costs speed on every step. How to budget context for an agent, truncate tool output without losing the signal, compact history without breaking the prefix cache, isolate work in sub-agents, and detect loops before they burn the budget.
- Harnesses9 minAgent eval statistics: is 816 vs 808 a real difference?Leaderboards rank models by a few points; the noise is usually bigger. Confidence intervals on pass rates, worked on my own 49-scenario benchmark, how many scenarios you'd need to separate two models, why paired tests are worth the effort, and why pass^k beats pass@k for agents.
- Inference10 minAttention from the kernel's side: FlashAttention, online softmax and fused dequantAttention is three lines of math and one big memory problem. How the online softmax makes tiling exact, what FlashAttention never writes to memory, why decode attention is a stream over the KV cache, and why a quantised cache only pays off when dequant is fused into the kernel.
- Architectures12 minAttention Is All You Need: the paper, the idea, and what it changedThe 2017 Google paper that introduced the transformer: the problem it set out to solve, what it actually proposed (scaled dot-product attention, multi-head attention, sinusoidal positions, an encoder-decoder for translation), the numbers it reported, what later models changed, and how a translation paper became the basis of modern AI.
Bring the problem the API couldn’t solve.
Describe your problem30-minute technical call with the engineer who builds it. No sales team, no slides.