distinct / tests /test_tool_budget_agrees.py
User1342's picture
Offload what the card can hold, survive a dropped connection, and stop sending prompts the model cannot read
95200a7
Raw History Blame Contribute Delete
1.34 kB
"""Four budgets bound a run, and the run gets the smallest of them.
Raising one of them is indistinguishable, from the outside, from raising none.
That is not a hypothetical: the harness limit was raised from three to ten,
every test passed, and the next benchmark behaved exactly as before, because
the worker's hard cap and the server's per-run config were both still three.
So the numbers are asserted to agree here rather than trusted to stay in step.
If a future change means one of them genuinely should differ, this test is the
place to say so, in writing, on purpose.
"""
from __future__ import annotations
from distinct_agent.harness import StructuredToolHarness
from distinct_agent.tools import HARD_TOOL_LIMITS
from distinct_server.ui import PER_RUN_TOOL_LIMITS
from distinct_tools.core import JobPolicy
EXPECTED = 10
def test_every_budget_that_bounds_a_run_agrees_on_the_number() -> None:
assert StructuredToolHarness.hard_max_tool_calls == EXPECTED
assert HARD_TOOL_LIMITS["max_calls"] == EXPECTED
assert PER_RUN_TOOL_LIMITS["max_calls"] == EXPECTED
assert JobPolicy(allowed_tools=frozenset()).max_calls == EXPECTED
def test_the_number_is_high_enough_for_the_request_that_exposed_it() -> None:
"""Plan, convert, scale, write the document, build the spreadsheet, answer."""
assert EXPECTED >= 6