03-04 Gradient Step That Reduces Loss

Why this matters

Gradient updates are the mechanism that turns prediction errors into parameter improvements.

Intuition first (no jargon)

Each step should move parameters in a direction that lowers average loss.

Code walkthrough

js
export function sgdStep(params, grads, lr) {}
export function trainEpoch(examples, params, lr) {}

Your task

Implement parameter updates and one training epoch loop.

• Update each parameter as p = p - lr * grad.
• Iterate through examples and accumulate average loss.
• Return updated params and loss summary.

Hints

• Start with tiny synthetic data to debug.
• Keep learning rate small and configurable.
• Log first and last loss for sanity checks.

Check your thinking

1. What if learning rate is too large?
2. Why can loss occasionally spike even if trend is down?
3. Why use average loss across examples?

Stretch (optional)

Track moving-average loss for smoother progress charts.

Likely test focus

• Parameters actually change after update.
• Loss decreases on a simple deterministic dataset.
• Handles empty example list safely.

What should improve

Your model now transitions from static scoring to iterative data-driven optimization.

Bridge to next lesson

Next lesson: replace one-hot vectors with embeddings.