Spaces:
Running
Running
| <html lang="en" data-theme="dark"> <head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"><title>About · Inside Beam</title><meta name="description" content="How Inside Beam was made, how Beam was used, and what the numbers mean."><link rel="canonical" href="https://rycerzes-inside-beam.hf.space/about.html"><meta property="og:type" content="article"><meta property="og:title" content="About · Inside Beam"><meta property="og:description" content="How Inside Beam was made, how Beam was used, and what the numbers mean."><meta property="og:url" content="https://rycerzes-inside-beam.hf.space/about.html"><meta property="og:locale" content="en_US"><meta property="og:site_name" content="About · Inside Beam"><meta property="og:image" content="https://rycerzes-inside-beam.static.hf.space/og.png"><meta property="og:image:alt" content="About · Inside Beam"><meta property="article:author" content="Swapnil Dutta"><meta name="twitter:card" content="summary_large_image"><meta name="twitter:title" content="About · Inside Beam"><meta name="twitter:description" content="How Inside Beam was made, how Beam was used, and what the numbers mean."><meta name="twitter:image" content="https://rycerzes-inside-beam.static.hf.space/og.png"><meta name="twitter:image:alt" content="About · Inside Beam"><script type="application/ld+json">{"@context":"https://schema.org","@type":"Article","headline":"About · Inside Beam","description":"How Inside Beam was made, how Beam was used, and what the numbers mean.","author":[{"@type":"Person","name":"Swapnil Dutta"}],"publisher":{"@type":"Organization","name":"Hugging Face","url":"https://huggingface.co"},"keywords":"interpretability","mainEntityOfPage":"https://rycerzes-inside-beam.hf.space/about.html","image":["https://rycerzes-inside-beam.static.hf.space/og.png"]}</script><link rel="stylesheet" href="/_astro/about.DLgs01lq.css"><script type="module" src="/_astro/page.BgPfbzVk.js"></script></head> <body class="app-body"> <header class="site-head"> <div class="sh-brand"> <a class="sh-name" href="/index.html">Inside Beam</a> <span class="sh-sub">12,544 experts of Beam-1.0-501B</span> </div> <nav class="sh-pills" aria-label="Pages"> <a href="/index.html">Studio</a><a href="/recipes.html">Recipes</a><a href="/explorer.html">Explorer</a><a href="/about.html" class="on" aria-current="page">About</a> </nav> <nav class="sh-links" aria-label="Project"> <a href="https://rycerz.es/blog/inside-beam">Write-up</a> <a href="https://github.com/rycerzes/beam-inside/blob/main/research_log.md">Research log</a> <a href="https://huggingface.co/datasets/rycerzes/beam-expert-atlas">Dataset</a> <a href="https://github.com/rycerzes/beam-inside">Code</a> </nav> </header> <main class="app-main"> <h1>About</h1> <p class="app-lede">By <b>Swapnil Dutta</b> (swappy · <a href="https://huggingface.co/rycerzes">rycerzes on Hugging Face</a> · <a href="https://github.com/rycerzes">GitHub</a>).</p> <p class="app-lede">Inside Beam is a map of the 12,544 routed experts of Reflection's | |
| <a href="https://huggingface.co/reflection/Beam-1.0-501B-A23B">Beam-1.0-501B-A23B</a>: 12,066 of them were pushed | |
| inside the running model and checked against random controls (478 lost their data to a time limit). The write-up is on | |
| <a href="https://rycerz.es/blog/inside-beam">rycerz.es</a>, and the <a href="https://github.com/rycerzes/beam-inside/blob/main/research_log.md">research log</a> has every step.</p> <h2>How Beam was used</h2> <ul> <li><b>The subject.</b> Every push, re-route and ban ran inside <a href="https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4">Beam-1.0-501B-A23B-NVFP4</a>, Reflection's NVFP4 release, in vLLM on 4× | |
| RTX PRO 6000 (HF Jobs), through our <a href="https://github.com/rycerzes/beam-inside/tree/main/steer">beam-steer</a> plugin.</li> <li><b>The labeller.</b> Beam named 10,926 of its own expert directions (name, description, subject, coherent or not) | |
| through HF Inference Providers; those labels chose the experts for the bans.</li> <li><b>The measuring stick.</b> Its identity line, its effort setting and its benchmark answers are the outcomes.</li> </ul> <h2>What you are looking at</h2> <ul> <li><b>Push.</b> Each expert's main write direction was added to the residual stream at three doses (gentle, full, and | |
| full the other way), on four prompts, greedy, up to 160 tokens (512 for the effort prompt). The studio shows those | |
| real outputs; nothing runs live. Long answers are cut by the token budget by design, and each answer says how it | |
| ended: finished, cut by the budget, stuck in a loop, or a broken reply format.</li> <li><b>Significant</b> means the outputs drift toward one theme more than 95% of random directions that move Beam by a | |
| similar amount. <b>Verified</b> means a blind judge also picked the expert's output from three given its top tokens, | |
| with high confidence (its top confidence score).</li> <li><b>Judges.</b> The blind test was scored by DeepSeek-V4-Flash (<code>deepseek-ai/DeepSeek-V4-Flash-0731</code>, via HF | |
| Inference Providers on DeepInfra, no thinking); it sees a label and three steered outputs, never which one is the | |
| expert's. Pilots on 200 items used Claude Sonnet 5.5, Qwen3.8-Flash-Next and Beam itself as judges, to choose the judge.</li> <li><b>Effort dial.</b> Router choices recomputed from captured router logits for 200 GSM8K questions at each effort | |
| setting, with Beam's own grouped top-8 rule.</li> <li><b>Ban.</b> MMLU-Pro (12,032 questions, no thinking) with the experts Beam labelled with a subject removed exactly | |
| via the routing bias, compared question by question with a ban of the same number of random experts matched on | |
| routing load (per-expert routing counts from the REAP calibration file, <code>reap_saliency.json</code> in | |
| <a href="https://huggingface.co/mrfakename/Beam-1.0-REAP-255B-A23B">mrfakename/Beam-1.0-REAP-255B-A23B</a>, Apache-2.0).</li> </ul> <h2>Limits</h2> <ul> <li>One greedy sample per prompt; 100 competition problems for the effort arms.</li> <li>Early MoE layers (3–23) are underdosed and their few hits are at or near chance with the judge (27–42% vs 33%); layers 48+ collapse into | |
| repetition at the doses needed and often have no fair control.</li> <li>478 experts have no sweep data (3 of 79 waves hit a time limit).</li> <li>Labels describe what an expert writes, read from its top tokens, not when the router picks it.</li> </ul> <h2>Credits</h2> <p><b>Where this started.</b> <a href="https://x.com/Simon_Vt/status/2114444548500005044">Simon V's post</a> quoting <a href="https://x.com/_lyraaaa_">@_lyraaaa_</a>'s two-neuron "evil Qwen" steering recipe, and Goodfire's essay <a href="https://already-clean.goodfire.com/">Neural Networks Are Already Clean</a>, which argues most units have one readable job and that you check a label by steering it.</p> <p><b>Method.</b></p><ul><li>Lyra's <a href="https://github.com/lyramakesmusic/neuron-explorer">neuron-explorer</a>: steering-first labelling with random controls, doses in units of the median residual norm, the who / snake / orange prompts, and the research-log format.</li><li><a href="https://github.com/RhizoNymph/vllm">RhizoNymph/vllm</a>, the steering fork of vLLM that neuron-explorer runs on: our beam-steer plugin's per-request steering table follows its design (the fork has no Beam support, so we wrote our own).</li><li>Expert steering and bans: SteerMoE (<a href="https://arxiv.org/abs/2509.09660">arXiv 2509.09660</a>) and MoTE (<a href="https://arxiv.org/abs/2502.11096">arXiv 2502.11096</a>).</li><li>The blind three-way label test: Gur-Arieh et al. (<a href="https://arxiv.org/abs/2501.08319">arXiv 2501.08319</a>). A model describing its own units: Li et al. (<a href="https://arxiv.org/abs/2511.08579">arXiv 2511.08579</a>). Reasoning strength read from activations: Sheng et al. (<a href="https://arxiv.org/abs/2506.08390">arXiv 2506.08390</a>). Expert-level MoE interpretability: Herbst et al. (<a href="https://arxiv.org/abs/2604.02178">arXiv 2604.02178</a>). The logit lens: <a href="https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens">nostalgebraist (2020)</a>.</li><li>Static weight analysis of Beam: <a href="https://github.com/Barneyjm/beam-teardown">Barneyjm/beam-teardown</a>.</li></ul> <p><b>Model, serving and compute.</b> Reflection's <a href="https://huggingface.co/reflection/Beam-1.0-501B-A23B">Beam-1.0-501B-A23B</a> and its <a href="https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4">NVFP4 release</a> (Apache-2.0), served with <a href="https://github.com/vllm-project/vllm">vLLM</a> following its <a href="https://recipes.vllm.ai/reflection/Beam-1.0-501B-A23B">Beam recipe</a>; transformers' Beam implementation for the reference checks; HF Jobs for the GPU runs; HF Inference Providers for Beam's labels (Baseten) and the judge, DeepSeek-V4-Flash (DeepInfra). Pilot judges: Claude Sonnet 5.5, Qwen3.8-Flash-Next and Beam itself.</p> <p><b>Data.</b> <a href="https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro">MMLU-Pro</a>, <a href="https://huggingface.co/datasets/openai/gsm8k">GSM8K</a>, <a href="https://huggingface.co/datasets/HuggingFaceH4/aime_2024">AIME 2024</a>, <a href="https://huggingface.co/datasets/MathArena/aime_2025">AIME 2025</a>, <a href="https://huggingface.co/datasets/math-ai/amc23">AMC 2023</a>, <a href="https://huggingface.co/datasets/Salesforce/wikitext">WikiText-103</a> (statistics only). The ban controls are matched on per-expert routing counts from the REAP calibration file (<code>reap_saliency.json</code> in <a href="https://huggingface.co/mrfakename/Beam-1.0-REAP-255B-A23B">mrfakename/Beam-1.0-REAP-255B-A23B</a>, Apache-2.0).</p> <p><b>The site.</b> Built on Thibaud Frere's <a href="https://huggingface.co/spaces/tfrere/research-article-template">research article template</a> (CC BY 4.0). The push panel's strength control follows <a href="https://huggingface.co/spaces/webml-community/neuron-studio">Neuron Studio</a>, the recipes follow <a href="https://neuronary.org/">Neuronary</a>, the map follows <a href="https://www.neuronpedia.org/">Neuronpedia</a>'s feature maps, and the route view follows the data-flow overview of Polo Club's <a href="https://github.com/poloclub/transformer-explainer">Transformer Explainer</a>. Charts with <a href="https://d3js.org/">D3</a>; the map layout with <a href="https://github.com/lmcinnes/umap">UMAP</a>.</p> <p><b>Tooling.</b> Built with the help of <a href="https://www.anthropic.com/claude-code">Claude Code</a> (Claude Opus 5.5).</p> <p><b>The contest.</b> Reflection and Hugging Face's <a href="https://huggingface.co/reflection/Beam-1.0-501B-A23B/discussions/5">Build with Beam</a> (<a href="https://huggingface.co/blog/reflection-beam">announcement</a>). Not affiliated with Reflection.</p> <h2>References</h2> <ul class="refs"><li>Cho, A., Kim, G. C., Karpekov, A., et al. (2024). Transformer Explainer: Learning LLM Transformers with Interactive Visual Explanation and Experimentation. <a href="https://arxiv.org/abs/2408.04619">arXiv:2408.04619</a>.</li><li>Cobbe, K., Kosaraju, V., Bavarian, M., et al. (2021). Training Verifiers to Solve Math Word Problems (GSM8K). <a href="https://arxiv.org/abs/2110.14168">arXiv:2110.14168</a>.</li><li>Dahlke, R., Klagges, H., Zecha, D., Merkel, B., et al. (2025). Mixture of Tunable Experts: Behavior Modification of DeepSeek-R1 at Inference Time. <a href="https://arxiv.org/abs/2502.11096">arXiv:2502.11096</a>.</li><li>Fayyaz, M., Modarressi, A., Deilamsalehy, H., Dernoncourt, F., et al. (2025). Steering MoE LLMs via Expert (De)Activation. <a href="https://arxiv.org/abs/2509.09660">arXiv:2509.09660</a>.</li><li>Frere, T. (2025). Research Article Template. <a href="https://huggingface.co/spaces/tfrere/research-article-template">Hugging Face Space</a>.</li><li>Goodfire (2026). Neural Networks Are Already Clean. <a href="https://already-clean.goodfire.com/">already-clean.goodfire.com</a>.</li><li>Gur-Arieh, Y., Mayan, R., Agassy, C., et al. (2025). Enhancing Automated Interpretability with Output-Centric Feature Descriptions. <a href="https://arxiv.org/abs/2501.08319">arXiv:2501.08319</a>.</li><li>Herbst, J., Wermter, S., Lee, J. H. (2026). The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level. <a href="https://arxiv.org/abs/2604.02178">arXiv:2604.02178</a>.</li><li>Kwon, W., Li, Z., Zhuang, S., et al. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM). <a href="https://arxiv.org/abs/2309.06180">arXiv:2309.06180</a>.</li><li>Li, B. Z., Guo, Z. C., Huang, V., et al. (2025). Training Language Models to Explain Their Own Computations. <a href="https://arxiv.org/abs/2511.08579">arXiv:2511.08579</a>.</li><li>Lyra (2026). neuron-explorer: steering-first neuron labelling. <a href="https://github.com/lyramakesmusic/neuron-explorer">github.com/lyramakesmusic/neuron-explorer</a>.</li><li>McInnes, L., Healy, J., Melville, J. (2018). UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. <a href="https://arxiv.org/abs/1802.03426">arXiv:1802.03426</a>.</li><li>Merity, S., Xiong, C., Bradbury, J., et al. (2016). Pointer Sentinel Mixture Models (WikiText). <a href="https://arxiv.org/abs/1609.07843">arXiv:1609.07843</a>.</li><li>nostalgebraist (2020). Interpreting GPT: the logit lens. <a href="https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens">LessWrong</a>.</li><li>Reflection (2026). Beam-1.0-501B-A23B. <a href="https://huggingface.co/reflection/Beam-1.0-501B-A23B">Hugging Face model card</a>.</li><li>Reflection (2026). Beam-1.0-501B-A23B-NVFP4. <a href="https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4">Hugging Face model card</a>.</li><li>Sheng, L., Zhang, A., Wu, Z., et al. (2025). On Reasoning Strength Planning in Large Reasoning Models. <a href="https://arxiv.org/abs/2506.08390">arXiv:2506.08390</a>.</li><li>Wang, Y., Ma, X., Zhang, G., et al. (2024). MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark. <a href="https://arxiv.org/abs/2406.01574">arXiv:2406.01574</a>.</li></ul> <h2>Cite</h2> <pre class="bib"><code>@misc{dutta2026insidebeam, | |
| title = {Inside Beam: steering, labelling and banning the experts of Beam-1.0-501B}, | |
| author = {Dutta, Swapnil}, | |
| year = {2026}, | |
| howpublished = {\url{https://huggingface.co/spaces/rycerzes/inside-beam}}, | |
| note = {Code: https://github.com/rycerzes/beam-inside; data: https://huggingface.co/datasets/rycerzes/beam-expert-atlas} | |
| }</code></pre> </main> <footer class="app-footer"> | |
| By Swapnil Dutta (<a href="https://huggingface.co/rycerzes">rycerzes</a>). Built for Reflection's <a href="https://huggingface.co/reflection/Beam-1.0-501B-A23B/discussions/5">Build with Beam</a> | |
| contest; not affiliated with Reflection. Data and code Apache-2.0 | |
| (<a href="https://huggingface.co/datasets/rycerzes/beam-expert-atlas">dataset</a>, | |
| <a href="https://github.com/rycerzes/beam-inside">code</a>). Site built on the | |
| <a href="https://huggingface.co/spaces/tfrere/research-article-template">research article template</a> by Thibaud Frere (CC BY 4.0). | |
| </footer> </body></html> |