How idkGPT was made
A ~117M-parameter chatbot that runs entirely on your device, and genuinely knows nothing. It chats warmly but never states a fact, because it was never trained on one.
What it does
Ask it anything factual and it echoes the subject back and pleads ignorance. It reads emotional tone, stays casual and terse, and never guesses, translates, or does tasks.
The model
A boring-on-purpose nanoGPT-class decoder-only transformer. It sticks to ops that WebGPU and WASM run natively, with no custom kernels.
- Parameters
- 116.8M
- Layers · heads
- 16 · 12
- Width · MLP
- 768 · 3072
- Context
- 512 tokens
- Vocab
- 4,096 byte-BPE
Depth is favoured over width, because depth is what tracks “is this the same question reframed?”. Weights are tied and bias-free with learned-absolute positions, and a built-in KV cache gives O(1)-per-token generation. Two numerically-identical forward paths exist: fused attention for training, plain MatMul/Softmax for a portable export.
Why it knows nothing
“Knows nothing” is a data property, not a guardrail. The model is only ever trained on LLM-written conversations that contain no facts, plus fact-free TinyStories for fluency. No facts go in, so there are none to leak, regardless of size or how hard it is pushed.
The tokenizer
A byte-level BPE with a 4,096 vocab, so it is out-of-vocabulary free. Everything is lowercase end to end, which shrinks what the tiny model must learn. Six special tokens mark conversation structure, and a completion-only loss mask means it learns to produce replies, never to model the user.
The data
5,750 persona conversations, all LLM-written (programmatic augmentation was tried and dropped for producing incoherent non-sequiturs). Volume and diversity come from coordinated self-play:
- Mine ~3,500 subjects across ~90 narrow domains
- Slice them so every agent gets a unique, non-overlapping set
- Each agent writes long, coherent, all-lowercase chats weaving its subjects
- Parse the delimited text into JSONL (plain text survived where JSON broke on long chats)
Training (on cloud GPUs, never locally)
- Fluency pretrain on TinyStories
- Train the byte-level BPE tokenizer
- SFT with the completion-only loss
- DPO to push mass away from answering (ignore vs answer, varied vs repeated, engage vs over-deflect)
- int4 quantization-aware training
- Evaluate, export, and re-validate
SFT uses early stopping and a style-diversity probe to dodge the repetition-collapse attractor that small models fall into.
Shrinking it to ~84 MB
Weight-only, group-wise int4 quantization (MatMulNBits), with attention raised to int8 where precision matters most. Because it is quantization-aware (a straight-through estimator fake-quantizes weights in the forward pass), the persona trains around the precision loss, so the eval numbers are the real int4 numbers. Embeddings stay int8.
Running in your browser
onnxruntime-web, using WebGPU when available and WASM otherwise. The decoding recipe is part of the model contract: temperature 0.45, top-k 12, repetition penalty 1.08, no-repeat 3-grams, up to 40 new tokens.
A KV-cache graph prefills the prompt once then forwards a single token per step, about 19× faster on WASM, which lets it keep the full context window. Each turn draws best-of-4 and runs three backstops so a fact never slips through: a leakage check (task content), a guess check (asserting a value with no ignorance cue), and a garble check (too many words unseen in training).