About the scaling-law checkpoints releases
Hi! Congrats on a great release! I've found the section on the parameter scaling law exciting. Is there any chance that you could share the checkpoints from that figure somehow? I think that would greatly help researchers in the field (akin to OLMO or Pythia in LLMs, I see this scaling set of results could be used in many forms for all sorts of interesting experiments)
Thank you for this thoughtful suggestion. We recognize that access to checkpoints from different points along the scaling trajectory could support follow-up research.
Currently, however, we are not able to release the full set of checkpoints used in the scaling-law experiments, as some of these checkpoints are closely tied to our ongoing productization and commercial deployment plans.
We nevertheless recognize the potential research value of making representative checkpoints at different scales available. One possibility we are considering is to release selected models corresponding to important points along the scaling curve, in a similar spirit to LimiX-2M, rather than releasing the complete internal checkpoint set.
We do not yet have a concrete release schedule or scope to share, but this is something we are actively considering as part of our future open-source releases. Thank you again for raising this point. It is valuable feedback for us.