Commit History
-
Let callers set the critic's request parameters
VLMQualityJudge built its chat-completions request with fixed parameters, so the reasoning and sampling settings used for the critic could not be configured: callers reproducing a specific evaluation setup had no way to pin e.g. reasoning_effort. Add an extra_body passthrough (default None, so the request is unchanged unless a caller sets it) and raise the critic's max_tokens default to 16384. With reasoning models the thinking tokens come out of the same budget (measured up to ~6.7k tokens for gemini-3.1-pro at reasoning_effort=high), which leaves little headroom under 8192 for the analysis JSON itself.9ae57c4 -
Draw one seed base per item when seed_base is unset
With RunnerConfig.seed_base=None the generator was called with seed=None, so every iteration re-rolled the noise. The best-of-N loop then compared images that differed by both the rewritten prompt and the noise, so a score change could not be attributed to the rewrite, and a run could not be reproduced. Draw a single base per item and reuse it across iterations (sample slot j always gets base + j), which is how an explicit seed_base already behaves. The drawn value is recorded on the result as "seed_base" so a run can be reproduced afterwards.9d3c345
-
Super-squash branch 'main' using huggingface_hub
mingyuliutw +8: CYChenv, tangyue0820, fferroni, harrim-nv, liang1225, shilinzhu-nvidia, mli0603, mbalaNV Co-authored-by: CYChenv <CYChenv@users.noreply.huggingface.co> Co-authored-by: tangyue0820 <tangyue0820@users.noreply.huggingface.co> Co-authored-by: fferroni <fferroni@users.noreply.huggingface.co> Co-authored-by: harrim-nv <harrim-nv@users.noreply.huggingface.co> Co-authored-by: liang1225 <liang1225@users.noreply.huggingface.co> Co-authored-by: shilinzhu-nvidia <shilinzhu-nvidia@users.noreply.huggingface.co> Co-authored-by: mli0603 <mli0603@users.noreply.huggingface.co> Co-authored-by: mbalaNV <mbalaNV@users.noreply.huggingface.co>754f0cd