harp-dev Claude Opus 5.5 commited on
Commit
459123a
·
verified ·
1 Parent(s): 4bf5da8

Build the model when the worker starts

Browse files

load_model is cached, but it was first called inside the GPU call, and ZeroGPU runs each request in a process of its own, so every request built the model inside its reservation. pyharp's worker now builds it at startup, where ZeroGPU's emulation records the placement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Files changed (1) hide show
  1. app.py +9 -1
app.py CHANGED
@@ -9,7 +9,7 @@ import gradio as gr
9
  from pyharp import ModelCard, build_endpoint
10
  from pyharp.tags import Subcategory
11
 
12
- from runtime import EXAMPLE, render
13
 
14
  model_card = ModelCard(
15
  name="DExter",
@@ -23,6 +23,14 @@ model_card = ModelCard(
23
  )
24
 
25
 
 
 
 
 
 
 
 
 
26
  def process_fn(xml_path: str | None, use_example: bool, seed: int) -> str:
27
  # Checked before the GPU is reserved, so a request without a score spends none of the
28
  # caller's ZeroGPU allowance
 
9
  from pyharp import ModelCard, build_endpoint
10
  from pyharp.tags import Subcategory
11
 
12
+ from runtime import EXAMPLE, load_model, render
13
 
14
  model_card = ModelCard(
15
  name="DExter",
 
23
  )
24
 
25
 
26
+ # pyharp runs process_fn in a worker, which imports this file as __mp_main__ and starts with the
27
+ # server. The model is built and placed there alone, where ZeroGPU's emulation records the CUDA
28
+ # placement. ZeroGPU runs each request in a process of its own, so a model first built inside the
29
+ # GPU call was built again, inside the reservation, every time.
30
+ if __name__ != "__main__":
31
+ load_model()
32
+
33
+
34
  def process_fn(xml_path: str | None, use_example: bool, seed: int) -> str:
35
  # Checked before the GPU is reserved, so a request without a score spends none of the
36
  # caller's ZeroGPU allowance