adelelsayed1991 commited on
Commit
7bbc0fc
·
verified ·
1 Parent(s): 7b67c79

Raise ZeroGPU duration 180->300s: 180 was killing longer completions mid-generation

Browse files
Files changed (1) hide show
  1. app.py +8 -3
app.py CHANGED
@@ -105,9 +105,14 @@ print(
105
  )
106
 
107
 
108
- # Raised alongside MAX_NEW_TOKENS above -- 2048 tokens of greedy decoding needs
109
- # more wall-clock headroom than the SQL-only completion this was tuned for.
110
- @spaces.GPU(duration=180)
 
 
 
 
 
111
  def generate_sql(question: str) -> tuple[str, str]:
112
  """Generate a plan+SQL completion for a natural-language question.
113
 
 
105
  )
106
 
107
 
108
+ # This is the GPU time the request is allowed, not a hint: ZeroGPU kills the
109
+ # task the moment it runs over, and the caller sees only "GPU task aborted".
110
+ # At 180s that happened intermittently -- short completions finished, longer
111
+ # ones were killed -- which reads as a flaky Space rather than a budget that
112
+ # is simply too small. Generation has to cover a 14B model in 4-bit emitting
113
+ # a JSON plan and then the SQL, so the budget is set to the maximum rather
114
+ # than trimmed to the average.
115
+ @spaces.GPU(duration=300)
116
  def generate_sql(question: str) -> tuple[str, str]:
117
  """Generate a plan+SQL completion for a natural-language question.
118