# Understanding backpropagation

A temporary, reading-first MNIST lesson: **one real network, MSE and cross entropy, and the general chain rule**. It assumes programming and matrix-multiplication familiarity, not experience with ML frameworks.

## Open it

From the DigitRecognition project directory:

```bash
npm --prefix learning_assets/backprop_interactive start
```

Open **http://127.0.0.1:8785/**. The entry file is `index.html`. Node.js 22+ is required; there is no dependency installation or runtime CDN. The start command regenerates the HTML from the reviewed Markdown before serving it. If the server is already running on that port, do not start another.

Useful entry points:

- [Page 4: derive the MSE and CE starting gradients](http://127.0.0.1:8785/#signal)
- [Page 5: follow either loss through the complete backward pass](http://127.0.0.1:8785/#backward)
- [Page 8: generic networks, losses, branches, and batches](http://127.0.0.1:8785/#matrix)

Read from page 1 if the notation is unfamiliar. The objective can be changed on the loss, output-gradient, weight-gradient, update, and generalization pages. Those controls stay synchronized. Switching the objective recalculates gradients, not predictions or weights.

## Markdown is the source of truth

**Edit the manuscripts, not the generated HTML.** Each chapter has one complete source, including its default worked example, activity instructions, labels, questions, answers, transitions, citations, and exact dynamic-message templates.

| Markdown manuscript | Generated page | Route |
| --- | --- | --- |
| `content/01-input.md` | `content/input.html` | `#input` |
| `content/02-forward.md` | `content/forward.html` | `#forward` |
| `content/03-loss.md` | `content/loss.html` | `#loss` |
| `content/04-signal.md` | `content/signal.html` | `#signal` |
| `content/05-backward.md` | `content/backward.html` | `#backward` |
| `content/06-compare.md` | `content/compare.html` | `#compare` |
| `content/07-update.md` | `content/update.html` | `#update` |
| `content/08-matrix.md` | `content/matrix.html` | `#matrix` |

The first manuscript’s clearly marked interface block owns shared shell wording. Navigation labels come from manuscript titles. Appendices after `<!-- implementation -->` own interpolation templates and alternative messages; they are not extra paragraphs in the web reading flow.

The small `:::` container notation identifies goals, recaps, activities, readouts, questions, optional details, and transitions. Standard Markdown supplies prose, lists, tables, and code; `$...$` and `$$...$$` supply mathematical notation. Raw HTML is limited to the specified interactive controls and graphics. Derivatives use stacked fractions in prose, tables, answers, and live readouts.

```bash
npm --prefix learning_assets/backprop_interactive run build
npm --prefix learning_assets/backprop_interactive run check:content
```

The build generates the chapter HTML, `index.html`, and `content/lesson.json`. Runtime renderers supply computed values to authored templates rather than writing independent explanations. Missing templates or interpolation values cause an error.

The initial worked numbers document the provided `config.json` and MNIST subset. If changing the initial model, seed, or example, review the numerical manuscript examples too. The browser’s initial-state parity check catches drift; do not treat automatically exporting a changed website as a substitute for reviewing its text.

The content-first review is recorded in `verification/CONTENT_REVIEW.md`.

## What is real, and what is generalized?

The live network remains **784 → 16 ReLU units → 10 softmax outputs**, with 12,730 parameters, manual Float64 matrix arithmetic, random seeded weights, and initially zero biases. It is trainable, not pretrained.

- MSE averages squared probability errors over classes; CE uses the observed label’s negative log probability. Both average over images in a batch, exactly once.
- The two backward comparisons use identical current parameters and cached forward values.
- The slope check and wrong-class bias experiment use copies and do not train.
- Page 7 applies actual individual SGD steps to the live network. Earlier pages disclose and reflect those updated parameters.
- Page 8’s batch demonstration is separate from the one-image walkthrough.
- The generic derivation covers Jacobian products, arbitrary-depth dense stacks, coupled activations, branches, shared parameters, and direct parameter penalties. It is not an arbitrary-network editor or a new autograd framework.
- Separate numerical fixtures verify the generic rules on three affine layers with tanh, both losses with optional direct weight penalties, and a shared parameter under another scalar objective.

Readouts are rounded; calculations use full precision. Reloading restores the configured defaults. No trained checkpoint is saved. This small teaching subset is not a full-MNIST benchmark.

## Verification

```bash
npm --prefix learning_assets/backprop_interactive test
npm --prefix learning_assets/backprop_interactive run check:content
npm --prefix learning_assets/backprop_interactive run test:browser
```

The numerical/content suite checks derivatives, updates, normalization, generalization fixtures, source freshness, and valid stacked mathematical notation. The real-browser suite checks initial and interactive Markdown parity, loss switching, unchanged weights during inspection, update/reset behavior, source-derived navigation, answer reveals, keyboard interaction, invalid input, reduced motion, and desktop/narrow layouts.

Browser checks require a Chromium-family browser opened to this app with remote debugging enabled at the host/port in `config.json` (currently 9333). They use Node’s native WebSocket support. No browser framework installation is required.

See `verification/VALIDATION.md` and `verification/browser-results.json` for the actual checks and limits.

## Data, references, and local libraries

The bundle retains 200 training and 100 held-out MNIST images with original split indices. The walkthrough uses training examples. Archive: <https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz>.

Archive SHA-256: `731c5ac602752760c8e48fbffcf8c3b850d9dc2a2aedcf2cc48468fc17b673d1`. MNIST is by Yann LeCun, Corinna Cortes, and Christopher J. C. Burges. `tools/prepare-data.py` can regenerate the subset from the archive using `--archive` and `--config` (requires NumPy).

Research sources are linked in the manuscripts and generated pages. Mathematics uses local **KaTeX 0.16.22** ([MIT license](vendor/katex/LICENSE.txt)); Markdown uses local **Marked 15.0.12** ([MIT license](vendor/marked/LICENSE.md)). Neither requires internet access while reading the lesson.