Undocumented config params + two questions on loop depth and length-control RL
Hi!
I've been running inference-time experiments on 4.2-3B, mostly around what happens when num_loops is pushed past the trained value of 2 with various interventions (and eventually some lightweight training). Two things, one small contribution and two questions.
Documentation gap in configuration_nanbeige.py
The class docstring documents the n-gram embedding arguments thoroughly, but the Args: block ends before these five, which appear in init with no entries:
num_loops
loop_loss_weights
enable_double_loop_split
enable_mhc
enable_depth_attention
The README and the technical report explain what the last three are for (LoopSplit and mHC with depth attention, landing in 4.5), so this looks like an oversight rather than a deliberate omission, but num_loops in particular is the parameter that defines the architecture, and it's the one people will look up first.
Q1. Loop depth ablation. Technical report section 2.1 says increasing the number of passes gave "only marginal additional improvement" while slowing training and destabilizing optimization. Were those separate models trained from scratch at each fixed loop count, and is there any sense of how marginal the improvement was - a benchmark point, a few? I'm trying to understand whether 2 is a soft economic optimum or something closer to a stability limit, which changes what's reasonable to expect when extending depth after the fact.
Q2. Length-control RL (section 3.2.3). Two implementation details that aren't in the paper: is the offline budget b_q computed per individual problem, or pooled across a difficulty tier? And does the alternation between constrained and free-expansion phases follow a fixed schedule, or is it triggered by a metric?
Thanks for releasing the report and the modeling code with the 4.5 previews in it — it's a lot more than most releases at this size ship with. I was very impressed with Nanbeige 4.1, and so far 4.2 is an improvement on every axis (particularly token efficiency). Can't wait to try the 4.5 release! Side note, 4.2 feels like a major enough architectural change that could be a full version increase, or is nanbeige 4 the name of the model rather than the 4th major version?
edit: oops, closed and reopened the discussion by mistake.
Hi, thanks for your thoughtful questions!
Configuration documentation: Thank you for pointing this out. The missing parameter documentation has now been added to the docstring in the configuration code.
Loop depth: We ran small-scale, from-scratch experiments with 1, 2, and 3 loops. Three loops performed better than two, but the gain was much smaller than the improvement from one to two, so the additional FLOPs were not cost-effective. Three loops also showed more frequent loss spikes than two. Therefore, two loops in Nanbeige4.2 is primarily an efficiency trade-off rather than a hard stability limit.
In Nanbeige 4.5, we introduce LoopSplit, where some layers are not looped while others are looped two or more times. We have also addressed the loss-spike issue associated with multiple loops. The LoopSplit architecture delivers further performance improvements while using comparable FLOPs to Nanbeige 4.2.Length-control RL: The offline budget (b_q) is computed separately for each problem. The length-constrained and free-expansion phases alternate on a fixed schedule during training.
Regarding versioning: Based on our experiments, with very little increase in FLOPs over Nanbeige 4.2, the next version will achieve a performance improvement many times larger than the improvement from Nanbeige 4.1 to 4.2. Therefore, we named the current version Nanbeige 4.2, and the next version will be Nanbeige 4.5. Nanbeige 5 will introduce even more features. Please stay tuned!