Edwin Sam Claude Opus 4.8 (1M context) commited on
Commit
4223fe3
·
1 Parent(s): 9b3cb78

Keep one container of each model warm during the demo/judging window

Browse files

The vLLM coach cold start is ~90s, which made the live demo feel broken on the
first question after idle. min_containers=1 holds one warm container per service
so judges never pay that. Revert to 0 after judging to stop idle GPU burn.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. modal_app.py +6 -0
modal_app.py CHANGED
@@ -104,6 +104,9 @@ hf_cache = modal.Volume.from_name("coach-cockpit-cache", create_if_missing=True)
104
  volumes={"/cache": hf_cache},
105
  timeout=20 * MINUTES,
106
  scaledown_window=300,
 
 
 
107
  )
108
  class Extractor:
109
  @modal.enter()
@@ -261,6 +264,9 @@ class Extractor:
261
  volumes={"/cache": hf_cache},
262
  timeout=20 * MINUTES,
263
  scaledown_window=300,
 
 
 
264
  )
265
  class Coach:
266
  @modal.enter()
 
104
  volumes={"/cache": hf_cache},
105
  timeout=20 * MINUTES,
106
  scaledown_window=300,
107
+ # Keep one container warm so judges never hit a cold start during the demo
108
+ # window. Set back to 0 after judging to stop the idle GPU burn.
109
+ min_containers=1,
110
  )
111
  class Extractor:
112
  @modal.enter()
 
264
  volumes={"/cache": hf_cache},
265
  timeout=20 * MINUTES,
266
  scaledown_window=300,
267
+ # vLLM cold start is ~90s — keep one container warm so the demo never eats it.
268
+ # Set back to 0 after judging to stop the idle GPU burn.
269
+ min_containers=1,
270
  )
271
  class Coach:
272
  @modal.enter()