cynstl commited on
Commit
762eed0
Β·
1 Parent(s): f8b39f5

fix: add @spaces.GPU decorator to load_chain_components for ZeroGPU

Browse files

The HF Spaces runtime emits 'No @spaces.GPU function detected during
startup' when no function is decorated with @spaces.GPU, and may tear
down the SSR/Node proxy as a result.

Fix:
- Decorate load_chain_components (the function that instantiates the
torch-based HuggingFaceEmbeddings model) with @spaces.GPU
- Move import spaces to the very top of rag_chain.py, before the
langchain_huggingface β†’ sentence-transformers β†’ torch import chain
(preserves the CUDA init order fix from the previous commit)
- Keep import spaces in app.py as belt-and-suspenders safety

Files changed (2) hide show
  1. app.py +4 -8
  2. rag_chain.py +19 -0
app.py CHANGED
@@ -1,12 +1,8 @@
1
- # Import the `spaces` package BEFORE any torch/CUDA-initializing import.
2
- # In HF Spaces, gradio's background watcher thread calls import spaces
3
- # lazily β€” if torch (loaded via langchain_huggingface β†’ sentence-transformers)
4
- # has already initialized CUDA, that lazy import raises:
5
- # RuntimeError: CUDA has been initialized before importing the `spaces` package.
6
- # Importing it here (top of app.py, before rag_chain pulls in torch) pre-empts
7
- # the error. The try/except keeps local dev working (spaces isn't installed outside HF Spaces).
8
  try:
9
- import spaces # noqa: F401
10
  except ImportError:
11
  pass
12
 
 
1
+ # Import `spaces` first β€” must happen before any torch/CUDA initialization
2
+ # (torch is pulled in transitively via rag_chain β†’ langchain_huggingface).
3
+ # The try/except keeps local dev working (spaces is HF-Spaces-only).
 
 
 
 
4
  try:
5
+ import spaces # noqa: F401 β€” imported for GPU-context registration, not called directly
6
  except ImportError:
7
  pass
8
 
rag_chain.py CHANGED
@@ -1,4 +1,22 @@
1
  import os
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  from dotenv import load_dotenv
3
  from langchain_community.vectorstores import FAISS
4
  from langchain_huggingface import HuggingFaceEmbeddings
@@ -30,6 +48,7 @@ Question: {question}
30
  Answer (with inline [n] citations):"""
31
 
32
 
 
33
  def load_chain_components():
34
  embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
35
  vectorstore = FAISS.load_local(INDEX_DIR, embeddings, allow_dangerous_deserialization=True)
 
1
  import os
2
+
3
+ # HF Spaces ZeroGPU: the `spaces` package scans for @spaces.GPU-decorated
4
+ # functions so the runtime knows which code path needs GPU access.
5
+ # When no GPU function is registered, Spaces emits:
6
+ # "No @spaces.GPU function detected during startup"
7
+ # and may tear down the SSR proxy.
8
+ #
9
+ # CRITICAL: this import MUST come before any torch/CUDA-initializing import
10
+ # (e.g. langchain_huggingface β†’ sentence-transformers β†’ torch). If torch
11
+ # is imported first and initializes CUDA, `import spaces` raises:
12
+ # RuntimeError: CUDA has been initialized before importing the `spaces` package.
13
+ try:
14
+ import spaces
15
+ GPU_DECORATOR = spaces.GPU
16
+ except ImportError:
17
+ def GPU_DECORATOR(func):
18
+ return func
19
+
20
  from dotenv import load_dotenv
21
  from langchain_community.vectorstores import FAISS
22
  from langchain_huggingface import HuggingFaceEmbeddings
 
48
  Answer (with inline [n] citations):"""
49
 
50
 
51
+ @GPU_DECORATOR
52
  def load_chain_components():
53
  embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
54
  vectorstore = FAISS.load_local(INDEX_DIR, embeddings, allow_dangerous_deserialization=True)