Self-attention already helps a transformer understand relationships between words using Query, Key, and Value. But there’s a problem.
One attention mechanism usually ends up focusing on a limited kind of relationship at a time.
Language doesn’t work like that. A sentence can have structure, meaning, and long-range links all at once.
That’s why transformers use multi-head attention .
What happens in multi-head attention
Instead of doing attention once, the model does it multiple times in parallel.
Each run is called a head, and each head has its own learned weights for Query, Key, and Value.
So every head looks at the same sentence, but in its own way.
How it flows
The input embeddings are first prepared
They are split into multiple heads using linear projections
Each head runs its own self-attention
Each head produces its own output
All outputs are joined back together
A final layer mixes them into one result
Why this works better compared to previous approach
Different heads naturally pick…