Attention mechanism: why a single word cannot predict the future
Looking only at the last word, how can a model predict the next? Why did Q, K, V and Multi-Head Attention evolve?
Many learners get lost in Q, K, V jargon. From first principles, Attention solves one simple flaw: you cannot predict the word after “The weather today is really __” from the word “really” alone. You must mix earlier context into the current token’s vector.
How do we mix context? 1) Uniform average dilutes key clues; 2) Raw dot-product lacks flexibility; 3) Project each token into Query (Q), Key (K), and Value (V) via three trainable matrices; 4) Multi-Head Attention (MHA) runs multiple Q/K/V heads in parallel for semantics and syntax, then concatenates back to 768D.
A Query compares against Keys to weight and pool relevant Values
Attention lets the model selectively focus on relevant context positions.
Score Relevance Against Context Keys
Aligned vectors produce higher dot-product scores.
Normalize Scores into Weights Summing to 100%
Relevant tokens receive high weight; unrelated tokens drop near zero.
Pool Values by Weight (Statistical Association, Not Proof)
Attention shows what influenced the output, not whether the claim is true.
First-principles breakdown: three steps
Upgrade Uniform Sum to Weighted Sum
Context words contribute unequally; “weather” matters more than “today” when predicting “sunny”, requiring relevance weights.
Split Q, K, V for Role Flexibility
Project each token into Query, Key, and Value via three matrices; score with Q·K, normalize via Softmax, and pool V.
Multi-Head Parallel Subspaces
Split vectors into multiple heads to capture syntax, coreference, and semantics simultaneously, then concatenate back to the original width.
Switch across the 4 evolutionary stages of attention in the top panel, then drag the Query direction slider below to inspect live Q·K dot products and V pooling.
See the calculation
score = Q · K / √2
weight = softmax(score)
output = Σ(weight × V)Queries, keys, and values are 2D teaching vectors. This is not a full Transformer with multiple heads, positional information, and training.
Thicker lines show higher weights. These weights come from hand-set vectors, not a real model interpreting this sentence.
Query, Key, Value Vectors
Three projections of a token vector via trainable matrices W_q, W_k, W_v: Q asks, K answers the match, V supplies content.
They are simply three single-layer linear projections enabling asymmetric matching and pooling.Scaled Dot-Product (÷ √d_k)
Dividing the dot product by √d_k prevents large values from pushing Softmax into vanishing-gradient extremes.
A numerical stability trick so Softmax gradients remain healthy during training.Multi-Head Attention (MHA)
Instead of one 768D attention pass, run twelve 64D heads in parallel and concatenate the results back to 768D.
Same compute budget, but lets the model attend to syntax and semantics at once.At its root, Attention is just a weighted sum of context; Q, K, V and multi-head simply give that weighted sum trainable flexibility.
After passing through Multi-Head Attention (MHA), how does the output vector dimension compare to the input dimension?
Explanations, cases, and exercises are authored by Reallier. Links provide further learning. The workplace activity and people are fictional.