| <!doctype html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width,initial-scale=1"> |
| <title>Factorial Code Forge Lab</title> |
| <style> |
| :root { color-scheme: dark; font-family: Inter, ui-sans-serif, system-ui; } |
| * { box-sizing: border-box; } |
| body { margin: 0; min-height: 100vh; background: #060817; color: #edf4ff; } |
| canvas { position: fixed; inset: 0; width: 100%; height: 100%; opacity: .55; } |
| main { position: relative; z-index: 1; width: min(1080px, 92vw); margin: auto; |
| padding: 72px 0 96px; } |
| .eyebrow { color: #73e6ff; letter-spacing: .18em; text-transform: uppercase; |
| font-size: .75rem; font-weight: 800; } |
| h1 { font-size: clamp(3rem, 8vw, 7rem); line-height: .9; margin: 14px 0 24px; |
| background: linear-gradient(120deg,#fff,#74e7ff 55%,#b48cff); |
| -webkit-background-clip: text; color: transparent; } |
| .lead { max-width: 760px; color: #b8c7e6; font-size: 1.2rem; line-height: 1.65; } |
| .actions { display: flex; flex-wrap: wrap; gap: 12px; margin: 30px 0 48px; } |
| a { color: inherit; } |
| .button { padding: 12px 18px; border-radius: 999px; text-decoration: none; |
| background: #eaf8ff; color: #07101c; font-weight: 800; } |
| .button.alt { background: #171d38cc; color: #dce8ff; border: 1px solid #415078; } |
| .grid { display: grid; grid-template-columns: 1.1fr .9fr; gap: 20px; } |
| .card { border: 1px solid #344269; background: #0c1128dd; border-radius: 24px; |
| padding: 24px; backdrop-filter: blur(18px); box-shadow: 0 24px 80px #0008; } |
| h2 { margin-top: 0; } |
| pre { white-space: pre-wrap; word-break: break-word; color: #a9bddf; |
| max-height: 520px; overflow: auto; } |
| ul { max-height: 520px; overflow: auto; padding-left: 1.2rem; color: #a9bddf; } |
| li { margin: 8px 0; } |
| input { width: 100%; padding: 12px; border-radius: 12px; border: 1px solid #344269; |
| background: #070b1a; color: white; margin-bottom: 12px; } |
| @media (max-width: 780px) { .grid { grid-template-columns: 1fr; } } |
| </style> |
| </head> |
| <body> |
| <canvas id="field"></canvas> |
| <main> |
| <div class="eyebrow">Jacob Garcia · Hugging Face Model Foundry</div> |
| <h1>Factorial Code Forge Lab</h1> |
| <p class="lead">Interactive factorial latent-code explorer. This showcase backs up the |
| trained artifacts, measured evaluation, and complete runnable source.</p> |
| <div class="actions"> |
| <a class="button" href="https://huggingface.co/spaces/ARotting/factorial-code-forge-lab/tree/main">Explore every file</a> |
| <a class="button alt" href="https://huggingface.co/ARotting">View the full foundry</a> |
| </div> |
| <div class="grid"> |
| <section class="card"> |
| <h2>Verified project card</h2> |
| <pre># Factorial Code Forge |
|
|
| Factorial Code Forge mixes two statistically independent non-Gaussian sources and |
| asks a tiny autoencoder to recover a useful two-dimensional code. In the experimental |
| variant, two adversarial predictors try to infer each latent coordinate from the |
| other while the encoder tries to make those predictions fail. Reconstruction and |
| variance constraints prevent a constant-code shortcut. |
|
|
| The historical anchor is Schmidhuber's 1992 |
| [Learning Factorial Codes by Predictability |
| Minimization](https://doi.org/10.1162/neco.1992.4.6.863). The modern experiment |
| compares predictability minimization with an architecture-matched reconstructive |
| encoder, PCA whitening, and FastICA. It is a new controlled benchmark, not a |
| reproduction of the original paper's implementation or datasets. |
|
|
| ## Verified results |
|
|
| Models trained on 20,000 mixed samples and were measured once on 5,000 new samples: |
|
|
| | Code | Absolute correlation | Symmetric MI estimate | |
| | --- | ---: | ---: | |
| | Autoencoder control | 0.02200 | 0.22345 | |
| | Online neural-adversarial ablation | 0.01886 | 0.22678 | |
| | Validation-selected predictability minimum | 0.01158 | 0.00280 | |
| | PCA whitening | 0.02039 | 0.22275 | |
| | FastICA | 0.00590 | 0.00326 | |
|
|
| The selected 12-parameter linear codec reconstructed the mixtures at |
| `1.08e-14` MSE. Its rotation was selected using polynomial cross-predictors on a |
| held-out portion of the training set, never the test set. The online neural |
| minimax game did not improve nonlinear dependence and is retained as a negative |
| stability result rather than omitted. |
|
|
| ## Reproduce |
|
|
| ```powershell |
| uv run python projects/factorial-code-forge/train.py |
| ``` |
| </pre> |
| <h2>Evaluation snapshot</h2> |
| <pre>{ |
| "benchmark": "Two-source factorial code recovery", |
| "codec_parameters": 12, |
| "adversary_parameters": 1346, |
| "test_reconstruction_mse": 1.0849382441166684e-14, |
| "selected_rotation_radians": 0.6335255903647925, |
| "validation_polynomial_predictor_error": 2.0412044050069174, |
| "results": { |
| "autoencoder_control": { |
| "absolute_correlation": 0.02199843078760486, |
| "symmetric_mutual_information_estimate": 0.22345146065238453, |
| "linear_source_recovery_r2": 1.0 |
| }, |
| "neural_adversarial_ablation": { |
| "absolute_correlation": 0.018864639802400177, |
| "symmetric_mutual_information_estimate": 0.22677681116066317, |
| "linear_source_recovery_r2": 1.0 |
| }, |
| "predictability_minimization": { |
| "absolute_correlation": 0.011581345795229356, |
| "symmetric_mutual_information_estimate": 0.002795710068105528, |
| "linear_source_recovery_r2": 1.0 |
| }, |
| "pca_whitened": { |
| "absolute_correlation": 0.02038519159561663, |
| "symmetric_mutual_information_estimate": 0.22274881647833844, |
| "linear_source_recovery_r2": 1.0 |
| }, |
| "fastica": { |
| "absolute_correlation": 0.005903947226495519, |
| "symmetric_mutual_information_estimate": 0.003260830744820531, |
| "linear_source_recovery_r2": 1.0 |
| } |
| }, |
| "training_history": [ |
| { |
| "step": 100, |
| "reconstruction_loss": 0.5751938223838806, |
| "whitening_loss": 0.4999920129776001, |
| "predictor_loss": 0.03917834907770157 |
| }, |
| { |
| "step": 200, |
| "reconstruction_loss": 0.10436813533306122, |
| "whitening_loss": 0.4999969005584717, |
| "predictor_loss": 0.012760449200868607 |
| }, |
| { |
| "step": 300, |
| "reconstruction_loss": 0.09469099342823029, |
| "whitening_loss": 0.49983352422714233, |
| "predictor_loss": 0.0026066915597766638 |
| }, |
| { |
| "step": 400, |
| "reconstruction_loss": 0.0830593854188919, |
| "whitening_loss": 0.49922531843185425, |
| "predictor_loss": 0.004627525806427002 |
| }, |
| { |
| "step": 500, |
| "reconstruction_loss": 0.08060496300458908, |
| "whitening_loss": 0.4975007474422455, |
| "predictor_loss": 0.01004670187830925 |
| }, |
| { |
| "step": 600, |
| "reconstruction_loss": 0.07426909357309341, |
| "whitening_loss": 0.4930780827999115, |
| "predictor_loss": 0.02585626393556595 |
| }, |
| { |
| "step": 700, |
| "reconstruction_loss": 0.0617077499628067, |
| "whitening_loss": 0.48229295015335083, |
| "predictor_loss": 0.06387390196323395 |
| }, |
| { |
| "step": 800, |
| "reconstruction_loss": 0.040828146040439606, |
| "whitening_loss": 0.4678165912628174, |
| "predictor_loss": 0.1077573299407959 |
| }, |
| { |
| "step": 900, |
| "reconstruction_loss": 0.022691549733281136, |
| "whitening_loss": 0.3986736834049225, |
| "predictor_loss": 0.36198264360427856 |
| }, |
| { |
| "step": 1000, |
| "reconstruction_loss": 0.021030452102422714, |
| "whitening_loss": 0.23594792187213898, |
| "predictor_loss": 0.9913151860237122 |
| }, |
| { |
| "step": 1100, |
| "reconstruction_loss": 0.04022720083594322, |
| "whitening_loss": 0.009166543371975422, |
| "predictor_loss": 1.8695604801177979 |
| }, |
| { |
| "step": 1200, |
| "reconstruction_loss": 0.003848953638225794, |
| "whitening_loss": 0.00017961427511181682, |
| "predictor_loss": 1.7930779457092285 |
| }, |
| { |
| "step": 1300, |
| "reconstruction_loss": 0.0014683017507195473, |
| "whitening_loss": 0.004436715971678495, |
| "predictor_loss": 1.7725040912628174 |
| }, |
| { |
| "step": 1400, |
| "reconstruction_loss": 0.0005974461091682315, |
| "whitening_loss": 0.0012278903741389513, |
| "predictor_loss": 1.8158133029937744 |
| }, |
| { |
| "step": 1500, |
| "reconstruction_loss": 0.0003378715191502124, |
| "whitening_loss": 0.003598805284127593, |
| "predictor_loss": 1.7867335081100464 |
| }, |
| { |
| "step": 1600, |
| "reconstruction_loss": 6.85915001668036e-05, |
| "whitening_loss": 0.0005436374340206385, |
| "predictor_loss": 1.8233861923217773 |
| }, |
| { |
| "step": 1700, |
| "reconstruction_loss": 0.0001249544438906014, |
| "whitening_loss": 0.00044188593165017664, |
| "predictor_loss": 1.8262327909469604 |
| }, |
| { |
| "step": 1800, |
| "reconstruction_loss": 1.3117482012603432e-05, |
| "whitening_loss": 0.00011215780978091061, |
| "predictor_loss": 1.8308453559875488 |
| }, |
| { |
| "step": 1900, |
| "reconstruction_loss": 1.1593769158935174e-05, |
| "whitening_loss": 0.0016796954441815615, |
| "predictor_loss": 1.7771589756011963 |
| }, |
| { |
| "step": 2000, |
| "reconstruction_loss": 0.0004471217398531735, |
| "whitening_loss": 0.004560444969683886, |
| "predictor_loss": 1.7743375301361084 |
| }, |
| { |
| "step": 2100, |
| "reconstruction_loss": 2.6598576368996873e-05, |
| "whitening_loss": 0.0024056474212557077, |
| "predictor_loss": 1.7827980518341064 |
| }, |
| { |
| "step": 2200, |
| "reconstruction_loss": 4.374821946839802e-05, |
| "whitening_loss": 7.564837142126635e-05, |
| "predictor_loss": 1.8309648036956787 |
| }, |
| { |
| "step": 2300, |
| "reconstruction_loss": 0.00020850264991167933, |
| "whitening_loss": 0.005683479364961386, |
| "predictor_loss": 1.822058916091919 |
| }, |
| { |
| "step": 2400, |
| "reconstruction_loss": 2.0745503206853755e-05, |
| "whitening_loss": 8.849342520989012e-07, |
| "predictor_loss": 1.8361434936523438 |
| }, |
| { |
| "step": 2500, |
| "reconstruction_loss": 1.0664540241123177e-05, |
| "whitening_loss": 0.0009145042859017849, |
| "predictor_loss": 1.7433021068572998 |
| } |
| ] |
| }</pre> |
| </section> |
| <section class="card"> |
| <h2>Backed-up artifact tree</h2> |
| <input id="filter" placeholder="Filter files…" autocomplete="off"> |
| <ul id="files"><li><code>README.md</code></li> |
| <li><code>__pycache__/app.cpython-311.pyc</code></li> |
| <li><code>__pycache__/data.cpython-311.pyc</code></li> |
| <li><code>__pycache__/model.cpython-311.pyc</code></li> |
| <li><code>app.py</code></li> |
| <li><code>artifacts/factorial-code-forge/codec.safetensors</code></li> |
| <li><code>artifacts/factorial-code-forge/evaluation.json</code></li> |
| <li><code>artifacts/factorial-code-forge/latent_comparison.npz</code></li> |
| <li><code>artifacts/factorial-code-forge/predictor_01.safetensors</code></li> |
| <li><code>artifacts/factorial-code-forge/predictor_10.safetensors</code></li> |
| <li><code>data.py</code></li> |
| <li><code>data/factorial_sources.parquet</code></li> |
| <li><code>model.py</code></li> |
| <li><code>requirements.txt</code></li> |
| <li><code>train.py</code></li></ul> |
| </section> |
| </div> |
| </main> |
| <script> |
| const canvas=document.querySelector('#field'),ctx=canvas.getContext('2d'); |
| let dots=[]; |
| function resize(){canvas.width=innerWidth;canvas.height=innerHeight; |
| dots=Array.from({length:90},()=>({x:Math.random()*innerWidth, |
| y:Math.random()*innerHeight,vx:(Math.random()-.5)*.35,vy:(Math.random()-.5)*.35}));} |
| function draw(){ctx.clearRect(0,0,canvas.width,canvas.height); |
| for(const d of dots){d.x=(d.x+d.vx+innerWidth)%innerWidth; |
| d.y=(d.y+d.vy+innerHeight)%innerHeight;ctx.fillStyle='#65dcff99'; |
| ctx.beginPath();ctx.arc(d.x,d.y,1.4,0,7);ctx.fill();}requestAnimationFrame(draw);} |
| addEventListener('resize',resize);resize();draw(); |
| document.querySelector('#filter').addEventListener('input',e=>{ |
| const q=e.target.value.toLowerCase();for(const li of document.querySelectorAll('li')) |
| li.hidden=!li.textContent.toLowerCase().includes(q);}); |
| </script> |
| </body> |
| </html> |
