04-02 Combine Context Vectors

Why this matters

A language head needs one context representation per example before scoring next-token logits.

Intuition first (no jargon)

Pooling and concatenation trade off compactness versus retained position detail.

Code walkthrough

js
export function combineContext(vectors, mode = "concat") {}

Your task

Implement context combination.

• Support mode = "concat" and mode = "mean".
• Validate consistent vector lengths.
• Return deterministic numeric output.

Hints

• Concatenation preserves position detail.
• Mean pooling is smaller but less specific.
• Write tiny shape assertions while learning.

Check your thinking

1. Why can mean pooling lose information?
2. When is concat more expensive?
3. Which mode is easier for a first baseline?

Stretch (optional)

Implement weighted mean pooling with learnable scalar weights.

Likely test focus

• Correct output for both modes.
• Shape correctness.
• Error on inconsistent vector dimensions.

What should improve

Context combination is now explicit, configurable, and testable.

Bridge to next lesson

Next lesson: feed context vector into a prediction head.