Kagzy commited on
Commit
3072ff2
Β·
verified Β·
1 Parent(s): 4c396f2

launch-then-attach: ZeroGPU owns the port until demo.launch()

Browse files
Files changed (1) hide show
  1. app.py +41 -27
app.py CHANGED
@@ -6,18 +6,23 @@ there.
6
 
7
  Why this shape
8
  --------------
9
- The obvious reading of "Gradio SDK" is `demo.launch()`. That does not work here,
10
- because ResearchOS is a FastAPI application first: the pipeline is built in the
11
- app's *lifespan*, and CORS for the Vercel front end is *middleware*. Copying the
12
- routes onto Gradio's own app loses both β€” Starlette refuses to add middleware
13
- once an app has started, and the lifespan never runs, so `state` stays empty and
14
- every request raises KeyError.
15
-
16
- So the nesting goes the other way. Our FastAPI app stays the root, keeping its
17
- lifespan and its middleware, and Gradio is mounted onto it at `/`. One process
18
- serves the human interface at `/` and the machine interface at `/papers`,
19
- `/explain`, `/upload` and the rest, which is what the browser extension and the
20
- Next.js front end call.
 
 
 
 
 
21
 
22
  Why there is no GPU code
23
  ------------------------
@@ -50,11 +55,11 @@ sys.path.insert(0, str(_root))
50
  os.chdir(_root) # every path in the config is relative to the project root
51
 
52
  import gradio as gr # noqa: E402
53
- import uvicorn # noqa: E402
54
 
55
  from src.api.llm import is_configured # noqa: E402
56
- from src.api.ml_app import app as api # noqa: E402 (root app: lifespan + CORS)
57
- from src.api.ml_app import state # noqa: E402
58
  from src.api.security import _limiter # noqa: E402
59
  from src.api.uploads import UploadError # noqa: E402
60
 
@@ -331,17 +336,26 @@ with gr.Blocks(title="ResearchOS", analytics_enabled=False) as demo:
331
  demo.load(corpus_line, None, status)
332
 
333
 
334
- # Gradio mounts onto our FastAPI, not the reverse β€” see the module docstring.
335
- # Gradio 6 moved `theme` off the Blocks constructor onto whatever starts the
336
- # server, which here is mount_gradio_app rather than launch().
337
- app = gr.mount_gradio_app(
338
- api, demo, path="/", theme=gr.themes.Soft(), max_file_size="25mb"
339
- )
340
-
341
-
342
  if __name__ == "__main__":
343
- uvicorn.run(
344
- app,
345
- host=os.environ.get("GRADIO_SERVER_NAME", "0.0.0.0"),
346
- port=int(os.environ.get("GRADIO_SERVER_PORT", "7860")),
 
 
 
 
 
 
347
  )
 
 
 
 
 
 
 
 
 
 
 
 
6
 
7
  Why this shape
8
  --------------
9
+ Gradio must own the server here, and the API is attached to it after launch.
10
+
11
+ That is forced, not chosen. ZeroGPU's runtime holds port 7860 with a placeholder
12
+ server and only releases it inside `demo.launch()` β€” running our own uvicorn
13
+ dies with "address already in use" after the whole pipeline has loaded, which is
14
+ exactly what happened on the first deployment. And the attachment must come
15
+ *after* launch: `launch()` builds a fresh FastAPI app, so anything added to
16
+ `demo.app` beforehand is silently discarded (verified: routes added pre-launch
17
+ 404).
18
+
19
+ The FastAPI app keeps its identity by being mounted whole onto Gradio's server.
20
+ A mounted ASGI app runs its own middleware stack β€” Starlette builds it lazily on
21
+ first request β€” so CORS for the Vercel front end survives. Its lifespan does not
22
+ run when mounted, so the state it would have built is built directly via
23
+ `build_state()` first. The mount is appended after Gradio's routes, and route
24
+ matching is first-match, so the UI wins at `/` and everything else β€” `/papers`,
25
+ `/explain`, `/upload`, `/health` β€” falls through to the API.
26
 
27
  Why there is no GPU code
28
  ------------------------
 
55
  os.chdir(_root) # every path in the config is relative to the project root
56
 
57
  import gradio as gr # noqa: E402
58
+ from starlette.routing import Mount # noqa: E402
59
 
60
  from src.api.llm import is_configured # noqa: E402
61
+ from src.api.ml_app import app as api # noqa: E402 (carries routes + CORS)
62
+ from src.api.ml_app import build_state, state # noqa: E402
63
  from src.api.security import _limiter # noqa: E402
64
  from src.api.uploads import UploadError # noqa: E402
65
 
 
336
  demo.load(corpus_line, None, status)
337
 
338
 
 
 
 
 
 
 
 
 
339
  if __name__ == "__main__":
340
+ # Launch first: on ZeroGPU this is the handshake that takes over port 7860
341
+ # from the platform's placeholder server. The UI is up immediately; its
342
+ # handlers answer "still loading" until build_state() finishes below.
343
+ # launch() reads GRADIO_SERVER_NAME/GRADIO_SERVER_PORT from the environment,
344
+ # which Spaces sets. Gradio 6 takes `theme` here, not on the constructor.
345
+ demo.launch(
346
+ prevent_thread_lock=True,
347
+ theme=gr.themes.Soft(),
348
+ max_file_size="25mb",
349
+ quiet=False,
350
  )
351
+
352
+ build_state()
353
+
354
+ # Attach the whole FastAPI app to the live server. Appended last, so
355
+ # Gradio's own routes keep winning where they exist and the rest falls
356
+ # through to the API β€” and a mounted app builds its own middleware stack
357
+ # (CORS included) lazily on first request.
358
+ demo.app.router.routes.append(Mount("/", app=api))
359
+ print(f"[space] API attached | UI and REST on one server", flush=True)
360
+
361
+ demo.block_thread()