Applied model development

Custom Model Training

Practical guidance for teams that need a model tuned for their reality, not generic benchmark theater. This section focuses on decisions, data, eval gates, and safe rollout.

Playbooks Field notes Operating principles

Core Playbooks

Start with decision quality, then move into data quality, then enforce ruthless evaluation before launch.

Start here Strategy

When Custom Training Is Actually Worth It

How to decide between prompting, RAG, fine-tuning, or full custom training without burning six weeks for a 4% gain.

Read playbook →
Data foundation Dataset

Dataset Design and Curation

Build a dataset that reflects real user behavior, edge cases, and failure modes instead of happy-path vanity examples.

Read playbook →
Quality control Evals

Evaluation and Release Gates

A practical eval stack: offline evals, scenario tests, red-team checks, and hard release thresholds that block bad launches.

Read playbook →
Architecture choice Decision matrix

Fine-Tune vs RAG vs Prompting

A plain-language decision matrix for selecting the lightest approach that achieves your target behavior and reliability.

Read playbook →

Field Notes

Experiments and practical observations from live model-training runs.

Lab experiment Pico models

Experiments in Training Pico LLMs

A 64 MB model, an RPG-style state template, and the failure modes that actually matter when you train tiny specialists.

Read field note →

Operating Principles

  • Use the smallest sufficient intervention: if prompt design solves it, do not train.
  • Ground decisions in production pain: train against real failures, not vibes.
  • Version everything: data, prompts, eval sets, and model artifacts need traceability.
  • Gate every release: no pass on evals means no launch, even when deadlines scream.
  • Measure drift continuously: a model can degrade quietly while dashboards still look pretty.