| <!doctype html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <meta name="description" content="An interactive PyTorch shape-arithmetic lesson: follow a small CNN, find a Linear mismatch, and see why a reshape can change your batch."> |
| <meta name="theme-color" content="#ffffff"> |
| <title>PyTorch shape lab · tensorViz</title> |
| <link rel="icon" href="logo.png"> |
| <link rel="stylesheet" href="styles.css"> |
| </head> |
| <body> |
| <a class="skip-link" href="#lab">Skip to the lesson</a> |
| <main> |
| <header class="brand"> |
| <img src="logo.png" width="34" height="34" alt="tensorViz logo"> |
| <span>tensorViz <span class="brand-separator">/</span> <span class="brand-label">shape lab</span></span> |
| <a href="https://github.com/tensorviz-org/tensorviz-examples/blob/1baecebf25ff1067a9112bde382c0a398fb99747/examples/shape-mismatch/README.md">Runnable example</a> |
| </header> |
|
|
| <section class="intro" aria-labelledby="page-title"> |
| <h1 id="page-title">Your batch is fine.<br>Check the features.</h1> |
| <p>A small image model reaches <code>Linear</code> with the wrong width. |
| Change the input, follow the shapes, and find the contract that broke.</p> |
| <p class="boundary">Shape arithmetic only. This page does not run Python, a model or inference.</p> |
| </section> |
|
|
| <section id="lab" class="lab" aria-labelledby="lab-title"> |
| <div class="controls"> |
| <h2 id="lab-title">Try an input</h2> |
| <p class="section-note">NCHW · 3 input channels</p> |
| <form id="shape-form" novalidate> |
| <div class="dimensions"> |
| <label for="batch">Batch <input id="batch" name="batch" type="number" min="1" max="64" step="1" value="2" inputmode="numeric" aria-describedby="input-bounds"></label> |
| <label for="height">Height <input id="height" name="height" type="number" min="2" max="256" step="1" value="16" inputmode="numeric" aria-describedby="input-bounds"></label> |
| <label for="width">Width <input id="width" name="width" type="number" min="2" max="256" step="1" value="16" inputmode="numeric" aria-describedby="input-bounds"></label> |
| </div> |
| <p id="input-bounds" class="field-help">Batch 1–64. Height and width 2–256.</p> |
| <div class="presets" aria-label="Example image sizes"> |
| <button type="button" data-height="16" data-width="16">16 × 16</button> |
| <button type="button" data-height="16" data-width="20">16 × 20</button> |
| <button type="button" data-height="17" data-width="19">17 × 19</button> |
| </div> |
| <fieldset> |
| <legend>Choose the classifier</legend> |
| <label class="mode"><input type="radio" name="mode" value="broken" checked><span>Original head<strong>Linear(128, 5)</strong></span></label> |
| <label class="mode"><input type="radio" name="mode" value="fixed"><span>Match the 16 × 16 input<strong>Linear(512, 5)</strong></span></label> |
| <label class="mode"><input type="radio" name="mode" value="pooled"><span>Add adaptive average pooling<strong>Pool to 1 × 1 → Linear(8, 5)</strong></span></label> |
| </fieldset> |
| </form> |
| <button id="reset" type="button" class="reset">Reset lesson</button> |
| </div> |
|
|
| <div class="results" aria-label="Calculated tensor shapes"> |
| <div id="invalid" class="invalid" role="status" hidden></div> |
| <div id="calculation"> |
| <ol class="flow" aria-label="Shape at each layer"> |
| <li><div><span class="step">01</span><span>Input</span></div><output id="shape-input">[2, 3, 16, 16]</output></li> |
| <li><div><span class="step">02</span><span>Conv2d + ReLU<small>3 → 8 channels · kernel 3 · padding 1</small></span></div><output id="shape-conv">[2, 8, 16, 16]</output></li> |
| <li><div><span class="step">03</span><span>MaxPool2d(2)<small>Spatial sizes round down</small></span></div><output id="shape-pool">[2, 8, 8, 8]</output></li> |
| <li id="adaptive-row" hidden><div><span class="step">+</span><span>AdaptiveAvgPool2d((1, 1))<small>New architecture: average each channel</small></span></div><output id="shape-adaptive"></output></li> |
| <li><div><span class="step">04</span><span>Flatten(start_dim=1)<small>Keep the first dimension: batch</small></span></div><output id="shape-flat">[2, 512]</output></li> |
| <li id="linear-row" class="mismatch"><div><span class="step">05</span><span id="linear-name">Linear(128, 5)<small>Expects 128 features per image</small></span></div><output id="shape-output">512 ≠ 128</output></li> |
| </ol> |
|
|
| <section id="verdict" class="verdict mismatch" aria-live="polite" aria-atomic="true"> |
| <h3 id="verdict-title">512 features arrived. The head expects 128.</h3> |
| <p id="feature-math">8 channels × 8 height × 8 width = 512 features per image.</p> |
| <p class="matrix"><span id="matrix-left">[2, 512]</span><span aria-label="multiplied by">×</span><span id="matrix-right">[128, 5]</span><span id="matrix-answer">incompatible</span></p> |
| <p class="matrix-note">Linear stores its weight as <code id="weight-shape">[5, 128]</code> and multiplies by its transpose.</p> |
| <code id="error-message" class="error">mat1 and mat2 shapes cannot be multiplied (2x512 and 128x5)</code> |
| </section> |
| <p id="mode-note" class="mode-note">Change the head to 512 to match this 16 × 16 example. Then try a wider image: the fixed width is a contract, not a universal fix.</p> |
| </div> |
| </div> |
| </section> |
|
|
| <section class="reshape-section" aria-labelledby="reshape-title"> |
| <div> |
| <h2 id="reshape-title">A reshape can hide the error.</h2> |
| <p>On the original feature map, <code>reshape(-1, 128)</code> asks PyTorch to infer a new first dimension. That dimension may no longer count your images.</p> |
| <p class="field-help">This comparison uses the feature map before any optional adaptive pooling.</p> |
| </div> |
| <div id="reshape-result" class="reshape-result" aria-live="polite"> |
| <p class="reshape-flow"><code id="reshape-before">[2, 512]</code><span>→ reshape →</span><code id="reshape-after">[8, 128]</code></p> |
| <p id="reshape-warning"><strong>2 images became 8 rows.</strong> Linear can now return [8, 5], but that is not one prediction per image.</p> |
| </div> |
| </section> |
|
|
| <section class="takeaway" aria-labelledby="takeaway-title"> |
| <h2 id="takeaway-title">Make the contract explicit.</h2> |
| <p>Use <code>Flatten(start_dim=1)</code> to preserve the batch. Match the classifier width to the feature extractor and intended input size. Adaptive pooling is another design choice: it changes the computation and needs a different classifier.</p> |
| <p>Matching shapes only makes these operations compatible. It does not establish prediction quality or numerical equivalence.</p> |
| <div class="resource-links"> |
| <a href="https://github.com/tensorviz-org/tensorviz-examples/blob/1baecebf25ff1067a9112bde382c0a398fb99747/examples/shape-mismatch/README.md">Read and run the PyTorch example</a> |
| <a href="https://tensorviz.dev/explore/?utm_source=huggingface&utm_medium=referral&utm_campaign=shape_lab_202610&utm_content=shape_lab_recorded_models">Explore recorded models in tensorViz</a> |
| </div> |
| </section> |
| <footer>Made by tensorViz with AI assistance. Original lesson code is MIT licensed. Calculations stay in your browser.</footer> |
| <noscript><p class="invalid">Enable JavaScript to change the inputs. The displayed default example shows a batch of two 16 × 16 images failing at Linear(128, 5).</p></noscript> |
| </main> |
| <script type="module" src="app.mjs"></script> |
| </body> |
| </html> |
|
|
