A framework for building small language models from scratch on a single consumer GPU — and a guided way to learn how language models actually work.
Try it in your browser → A folk-tune model trained with slmkit (864K parameters, under a minute of GPU time) writes a tune on request, draws it as sheet music and plays it. The model runs in your browser; there is no server (how).
ingest → prepare → tokenizer → pack → pretrain → SFT → eval → export → serve
The first reference model learning, trained from random weights in about 3 GPU-minutes: loss on
the left, what it writes on the right. The end-to-end picture is in
docs/LIFECYCLE.md.
Most tutorials either fine-tune somebody else's model or stop at a toy training loop. slmkit covers the whole lifecycle: you train your own tokenizer, pretrain a model from random initialization, instruction-tune it, evaluate it with graders that cannot be fooled, export it to a standard format, and serve it.
Measured on one RTX 5080 (16 GB) under WSL2. Each number links to where it was measured.
| Shakespeare (M1) | Folk tunes (M2) | |
|---|---|---|
| Learned from | 1 MB of Shakespeare's plays, one character at a time | 4,248 public-domain folk tunes, written as text in ABC notation |
| Model | 10.6M parameters | 4.8M parameters (the best of five configurations) |
| Training | 81.9M tokens · 3 GPU-minutes · 1.6 GiB of GPU memory | 30M tokens · about 36 GPU-seconds per run · 2.0 GiB |
| Result | validation loss 1.29; nanoGPT's published reference is about 1.47 | 74% of generated tunes play without errors and 81% of bars have the right length (mean of 3 training seeds; real tunes: ~100% and 99%) |
| Then | fine-tuned to take requests in plain English, exported in Hugging Face format, served to a browser that draws and plays the tunes | |
| Sources | the training loop, runbook M1 | experiments, runbook M2 §F |
Asked to continue JULIET: / O Romeo, Romeo, the Shakespeare model writes "that cohangeth me to his
act / Of God and first that company thee to the bawd.": the look of Shakespeare, invented words included,
from 1 MB of text. Asked for a jig in G major, 6/8 time, the folk-tune model writes a jig in G major in
6/8 whose every bar adds up. How to read both, with no music background.
People who are comfortable with software and infrastructure but new to machine learning.
Everything ML-specific is explained; nothing infrastructure-specific is. Every abbreviation is
defined in docs/GLOSSARY.md — start there if a term is unfamiliar.
The model, training loop, sampler and SFT masking are written out in full on purpose, with no
training framework. There is no Trainer class hiding the interesting parts.
The framework is the deliverable. Individual models exist to exercise and harden it.
- A new model on an existing project = one YAML file.
- A new use case = one directory under
projects/, with no changes to the engine.
Planned projects: character-level Shakespeare (a correctness reference), ABC music notation, chess, and cricket ball-outcome prediction. Each was chosen for one reason — it can be graded by a program, with no human judgment in the loop.
| Path | Role | Terraform analogy |
|---|---|---|
src/slmkit/ |
Engine. Never imports a project. | provider |
presets/ |
Shared model and training config fragments | reusable module |
projects/<name>/ |
One model: ingest, graders, experiments | root module |
projects/<name>/experiments/*.yaml |
One run each | tfvars |
$SLM_HOME (default ~/slm) |
All artifacts, content-addressed and immutable | state |
- GPU: one NVIDIA GPU with CUDA 12.8 support; 12–24 GB of VRAM is the design target, and every model so far fit in about 2 GB.
- Tested on: Windows with WSL2 (Ubuntu 24.04) and an RTX 5080. Native Linux with an NVIDIA
driver is expected to work but hasn't been tested; reports are welcome. The WSL-aware parts have
plain-Linux fallbacks (the studio opens your default browser) or are optional (
slm export --to-windows). - Not supported: macOS and AMD GPUs.
- Without a GPU: the CPU test suite (
make test) runs anywhere Python 3.12 and uv do, and CI runs it on every push.
docs/DESIGN.md §1 lists the platform assumptions and how the numbers change on
other hardware. Nothing here needs a cloud account, a cluster, or a machine that stays powered on.
uv sync
make doctor # environment checks; must pass before any training command
make test # CPU unit tests, under 60s
uv run slm run shakespeare_char/ref
uv run slm studio start # "See it in action": everything you built, in your browserdocs/concepts/two-models.md |
No music background? What the two models do, and how to read their output |
docs/concepts/browser-inference.md |
The model in a browser: the same network in JavaScript, with no server (slm site) |
slm studio |
See it in action. A local web app over your runs, artifacts, models and machine (runbook) |
docs/LIFECYCLE.md |
Start here. The model development lifecycle, with diagrams of the pipeline, architecture and roadmap |
docs/GLOSSARY.md |
Every abbreviation and term, in plain language |
docs/MODEL.md |
The model: type, architecture features, hyperparameters, size presets, file formats |
docs/STACK.md |
Every tool and library: what it is, why it's used, what was rejected |
docs/concepts/ |
Explainers written as each stage is built |
docs/runbooks/ |
Per-milestone: verify it yourself, expected output, and why |
docs/DESIGN.md |
Architecture, contracts, and the rules the code enforces |
docs/ROADMAP.md |
Milestones and their exit criteria |
docs/decisions/ |
ADRs — what was decided, and what was rejected |
CONTRIBUTING.md |
The rules every change follows: hard rules, conventions, commits. Want to help? Start with a good first issue, and see "Picking up an issue" there |
CHANGELOG.md |
What changed in each release |
M0 (environment) and M1 (engine, validated by reproducing a published character-level
Shakespeare result: best validation loss 1.29) are complete. So is M2, the first full-lifecycle
project: folk tunes in ABC notation, trained from scratch, fine-tuned to take requests in words,
graded, compared across 3 training seeds, exported in Hugging Face format and served over HTTP.
M3 (chess) is next. See docs/ROADMAP.md.
Apache License 2.0 — see LICENSE and NOTICE.
Apache-2.0 includes an express patent grant from contributors to users, and terminates that grant for anyone who brings a patent claim over the software. Contributions are accepted under the same terms.