Inside a Model

Three things ship together. Two of them are so small you can barely draw them.

Part	What it is	Size	Share
Config	The blueprint: how many layers, how wide, how the pieces are arranged. Chosen by people before training, and frozen.	a few KB	~0%
Tokenizer	The fixed vocabulary of 100,000–250,000 text chunks, plus the rules for splitting text into them.	a few MB	~0%
Weights	Every number learned during training. This is the model.	a few GB to over a TB	>99.9%

Text enters on the left and probabilities come out on the right. In between, the same structure repeated dozens of times.

Open any one of those layers and you find two blocks, always in the same order. This pair is the entire repeating unit of the model.

The whole model is this rhythm, eighty times over. Look around, think it over. Look around, think it over — the same two moves, with different numbers each round.

This is the block that holds most of the model, and the one with the least intuitive name. Mechanically it does something simple: it makes the token's vector much wider, throws most of it away, then squeezes it back to the original width.

Why the wide middle matters

Think of those ~28,000 slots as a bench of very narrow specialists. Each one has learned to react to one particular pattern — and for any given token, almost all of them stay silent. A handful fire.

The expand step asks every specialist whether this token is their business. The filter step discards everyone who said no. The collapse step gathers what the few responders had to contribute and writes it back into the token's vector.

Ask a model “What time is it?” and follow it through every part above. Six steps, and not one of them involves a clock.

1. 01

   Split into tokens

   The tokenizer chops the text into chunks from its fixed vocabulary. Five tokens, not four words — the leading spaces belong to the chunks.

   What time is it?
2. 02

   Look up each vector

   Each token is swapped for its row in the embedding table — about 8,000 numbers apiece. This is a pure lookup: “ it” gets exactly the same vector here that it would get in any other sentence. Nothing about this particular question has touched it yet.

3. 03

   Each layer rewrites the vectors

   Five vectors go in, five come out — same count, same size. What changes is what is written in them.

   Follow position 4, “ it”. The attention half lets it look across the sentence: “ time” scores high, “ is” some, “What” barely — and a weighted blend of their information is mixed in. It now means it, which refers to time. The feed-forward half then takes that updated vector on its own: ~28,000 specialists check whether the pattern is theirs, a few fire, and what they know is written back — this is a request for a current value.

   That is one layer. Layer 2 does the same to the result, and so on. Each round can only build on the last — you cannot notice “a question about a live fact” before you have noticed “‘it’ means time.” Eighty rounds, and the page never gets longer, only denser.

   position 4, before A generic pronoun. Identical in every sentence it appears in.

   →

   position 4, after A compressed summary of this whole question, seen from that word’s position.

4. 04

   Score the vocabulary from the last token

   Only the final token’s vector is used — it is the one standing where the next word goes, and after eighty rounds it carries the whole question. The output matrix compares it against every entry in the vocabulary and returns a score for each: 100,000 to 250,000 numbers, depending on the model.

   in One vector. ~8,000 numbers.

   →

   out One score per token in the vocabulary. 100,000–250,000 numbers.
5. 05

   Read the distribution

   Those scores become probabilities. Here is that distribution — beside the same pipeline run on a question the model can answer.

6. 06

   Pick one, append, run it all again

   Sampling chooses a token from the distribution — at the lowest setting, always the top one. It is appended to the input, which is now six tokens long, and all six steps run again from the top. Once per token of the answer.

Which is the uncomfortable part of the comparison above. Both answers came out of identical machinery, with no step anywhere that distinguishes them. One happens to be a fact the weights encode well; one happens to be a memorized disclaimer. The model cannot tell which is which — and the numbers will not tell you either, because a high score means the text was likely, not that it was true.

Illustrative numbers throughout; real distributions vary by model and by phrasing.

So why does it tell me the time now?

Because something outside the model started writing the answer down for it. The weights did not change. Nothing in any of the four zoom levels above is different.

Two mechanisms do this, both outside the model. Most commonly the product team writes the current date and time into the standing instruction (system prompt) — a block of text put in front of every request, which you never see. Less commonly the model is given a tool it can call, and the clock answers.

Either way the model still does not know the time. It reads the time — out of the same request that carries your conversation history. It is the same trick that makes a model appear to remember your conversation: put the thing into the text of the request, and it appears to possess it.

There is a tell, if you want to catch it happening. A model fed the date is exact about today but has no way to know this morning’s news unless a tool fetched it — and without one, it may offer a plausible guess rather than admit the gap. Ask how it knows: anything written into the request, like the date or a search result, it can point to. Anything that came from training, it can only explain after the fact.

Everything above describes a finished model being used. The first and largest stage of training, pretraining, is those same steps run on ordinary text where the answer is already known, plus a correction at the end: check the guess, work out which numbers to blame, nudge them.

Stage 1: pretraining

1. 01

   Cut a real sentence short

   Take a passage from the training data and stop partway. Everything after the cut is held back.

2. 02

   Run the forward pass

   Exactly the steps in the trace above — tokenize, embed, eighty layers, score the vocabulary. Out comes a probability for every possible next token. Steps 2 to 5 are the training loop itself; every stage of training uses it.

3. 03

   Check it against what was really there

   The text itself is the answer key. If the model gave the true next token a probability of 0.02, that gap is the error — a single number saying how wrong this guess was.

4. 04

   Work backwards through every parameter

   For each of the billions of numbers in the model, work out whether nudging it up or down would have made the right token more likely. This is the expensive half — roughly twice the work of the forward pass. Counted per number: about two pieces of arithmetic going forward and four coming back, six for every token of training data.

5. 05

   Nudge them all, very slightly

   Not enough to get this one sentence right — a tiny fraction of a percent each. The model is being pushed toward being right on average, not on this example.

6. 06

   Do it again, trillions of times

   Every token in the training data is one more cut, one more guess, one more round of nudges.

Stage 2: post-training

Pretraining produces a model that continues text. Post-training makes it answer. It runs the same loop, steps 2 to 5, with three things changed: the text in step 1 is a conversation written or chosen by people; the answer key in step 3 is their reply, or their rating of the model’s own; and step 6 runs thousands of times, not trillions. Training on written replies is called fine-tuning; training on ratings is reinforcement learning from human feedback, or RLHF.

Everything named above, collected. These are the words that will come up when someone technical describes what they are building.

Term	What it actually is
Weights / parameters	The learned numbers. Individually meaningless multipliers; collectively, everything the model knows how to do.
Embedding table	One learned vector per vocabulary entry. The lookup that turns a token into coordinates in meaning-space.
Layer	One repetition of the attention + feed-forward pair. A large model stacks dozens.
Attention block	The only place tokens influence each other. Computes how much every earlier token matters to this one.
Feed-forward block	Expand, filter, collapse — applied to each token separately. Two-thirds to four-fifths of the model's numbers.
Output matrix	Turns the final vector into a score for every entry in the vocabulary. Those scores become the probabilities.
Pretraining	The first and largest stage of training: predict the next word of ordinary text, trillions of times. No labels; the text is its own answer key.
Post-training	The later, much smaller stage that turns a text-continuer into something that answers. Same loop, on material people wrote or rated.
Fine-tuning	Post-training on example conversations written by people. The model is nudged toward their replies.
RLHF	Reinforcement learning from human feedback. Post-training on people’s ratings of the model’s own answers.
Checkpoint	A saved snapshot of all these numbers at a moment in training.
Open weights	The numbers above, published. Not the training data, not the code that made them.

1. Transformer Feed-Forward Layers Are Key-Value Memories — Geva et al., 2020. arxiv.org/abs/2012.14913 Feed-forward blocks behave as associative memories: each key matches a textual pattern, each value induces a distribution over the output vocabulary. Also the source for feed-forward layers being roughly two-thirds of a transformer's parameters in the original design. ↩back

Figures are representative rather than definitive — widths and layer counts vary between models, and architectures differ in the details. The early / middle / late account of what layers do is a working intuition, not a measured result.