REFACTOR_PLAN.md

Shared backward rules and renderer-independent explanations

Branch: refactor.

Objective

Use the walkthrough's symbolic-rule / numerical-application structure in every view. Store the mathematical content independently of HTML, Mermaid, typography, and level of detail. A future compact lecture graph or TikZ renderer must be able to select structured equations without parsing another renderer's strings.

Contracts

• Scalar local rule: differentiate the producing operation's formula with respect to one independent operand slot. Show its symbolic and evaluated derivative.
• Scalar contribution: c_source→target = source_bar * local_rule = values = result.
• Repeated operands: retain distinct slots and contributions. Explain a = x, b = x, then diff(a * b, a) = b = x = 3. Include the operand number in the contribution subscript only when the target occurs more than once.
• Vector/matrix local rule: show the symbolic VJP contribution directly, followed by its numerical substitution and result. Preserve multiplication, outer product, transpose, elementwise product, and broadcast reduction as distinct structured operations. Never label a VJP as diff(out, target).
• Adjoint introduction: reserve target_bar = diff(output, target) for the established total derivative, after all contributions arrive.
• Rule selection depends on local operand/result shapes, not which parser or engine built the surrounding graph.
• The walkthrough is the reference for teaching structure; repair existing mathematical and Code-mode inconsistencies during migration.

Architecture

1. A small immutable mathematical expression tree stores literals, quantity and adjoint references, operand slots, arithmetic, and supported tensor primitives. It supports evaluation and substitution independently of any renderer.
2. Explicit scalar and tensor rule records store the operation formula, selected operand occurrence, contribution expression, and explanatory notes. Scalar records additionally retain the derivative and optional slot mapping.
3. Numerical engine records carry the rule and the values computed from it. Tensor computation and explanation use the same expression. Scalar SymPy differentiation remains the source of scalar local derivatives.
4. The shared graph model stores quantity snapshots, operations, backward steps, contributions, and accumulations. Backward explanations store equations and bindings as structured objects, never HTML fragments.
5. A shared math formatter renders expressions/equations in Mathbook or Code notation and HTML or plain text. View renderers control containers, colors, layout, and detail, without reconstructing differentiation or VJP rules.

Implementation sequence

1. Establish the baseline and structured representation

• Run the existing suite before changes.
• Add expression nodes, evaluation/substitution, and typed rule/explanation data.
• Retain quantity identity independently of its display name; represent operand occurrences explicitly, including repeated references to the same quantity.
• Snapshot numerical values so reevaluating a graph cannot mutate an old model.
• Add one canonical operation-formula builder and math formatter.

2. Migrate scalar explanations

• Preserve scalar engine operand slots and symbolic derivatives in records.
• Construct structured scalar explanations from those records.
• Handle repeated operands with slot mappings, not ambiguous diff(x*x, x)=x.
• Switch diagram, HTML walkthrough, and text walkthrough to shared equations.
• Use the same contribution identity in local steps and accumulation listings.

3. Consolidate tensor backward computation

• Implement one rule factory for negation, arithmetic, powers, supported matrix products/dot, reductions, selection, softmax, and elementwise functions.
• Evaluate those expression trees in the tensor engine and retain them in records.
• Encode scalar broadcasting reduction explicitly in the contribution expression.
• Preserve zero powers, extrema tie sharing, ReLU at zero, negative selection indices, and stop-gradient boundaries.
• Treat scalar operations within tensor graphs through the scalar contract.
• Remove renderers' numerical recomputation and duplicated operation switches.

4. Consolidate presentation and remove legacy paths

• Generate symbolic/substituted operation formulas from the shared representation.
• Share adjoint/contribution symbols and equation formatting across renderers.
• Keep tensor value previews and their truncation metadata consistent.
• Generate explanatory examples through the same rules where practical.
• Remove unused legacy label APIs and engine display strings after migrating their callers/tests. Do not keep competing mathematical implementations.
• Document the contracts and extension point for compact/TikZ renderers.

5. Verification

• Test explicit expected equations as well as cross-view agreement in both notations. Do not weaken old assertions to merely check for diff(.
• Check scalar repeated operands, powers, negative factors, and shared quantities.
• Check tensor values and shapes for all supported rank combinations, broadcast reductions, reductions, selection, softmax, and stopped paths.
• Use finite differences on smooth representative tensor operations affected by numerical refactoring; use explicit expected subgradients at nonsmooth points.
• Verify model snapshots, renderer independence, and forward-only behavior.
• Run the complete suite and inspect real rendered examples for scalar, vector, and matrix layouts, including long equations and contribution subscripts.

Scope and completion

Retain existing parsing, supported operation set, graph topology, and UI controls. Building TikZ or a compact renderer is future work; this refactor supplies their structured input. Completion means one definition of each tensor backward rule, one shared explanation per backward step, no renderer-side numerical rule reconstruction, and passing numerical/semantic/layout verification.

Progress

• Create requested branch and record this plan.
• Establish baseline and expression/rule representation.
• Migrate scalar explanations.
• Consolidate tensor rules and numerical evaluation.
• Migrate views and remove duplicate paths.
• Update documentation and complete verification.

Completed verification

• Baseline: 190 passed, 8 skipped.
• Refactored suite: 224 passed, 8 skipped. The skipped checks require optional PyTorch, which is not installed in the test interpreter.
• All 29 application examples generated diagram and walkthrough output in both notations and both computation modes using the application virtual environment.
• Browser inspection covered scalar repeated operands, vector contributions and scalar broadcast reduction, and matrix products. Labels wrapped without clipping in the inspected examples. Mathbook numerical outer products use explicit outer(...) to avoid row/column ambiguity.
• Model snapshots survive reevaluation; rendering tests forbid expression evaluation. Forward-only legends do not construct backward examples.
• A zero-exponent regression verifies that an infinite incoming adjoint does not turn a constant-power tensor contribution into NaN.
• git diff --check passes.

The implementation and extension API are documented in SHARED_MODEL.md. The changes are on refactor and remain uncommitted for review.