Spaces:
Running on Zero
Running on Zero
Commit History
-
revert: simple lazy loading inside @spaces.GPU — working baseline Running
- @spaces.GPU on respond() with duration=60 - Pipeline loaded lazily on first query inside GPU context - device_map='auto' lets transformers pick GPU automatically - No CPU/GPU dance, no try/finally — straightforward and reliable7346617 -
fix: initialize self.pipe = None in __init__ — prevent AttributeError
c196b3e -
fix: @spaces.GPU on standalone function, not class method — ZeroGPU compat
- run_generation() is standalone with @spaces.GPU, calls generator.generate() - try/finally: model.to('cuda') → generate → model.to('cpu') + empty_cache - generator.generate() is plain method (no decorator), uses self.pipe - app.py manages GPU lifecycle, generation.py is pure logic9917e61 -
fix: proper ZeroGPU lifecycle — try/finally CPU offload, @spaces.GPU on generate()
- @spaces.GPU on generate() not respond() — GPU quota isolated to forward pass - try/finally block: self.model.to('cpu') + torch.cuda.empty_cache() on exit - torch_dtype (not dtype) for AutoModelForCausalLM - respond() is pure CPU — no GPU lease wasted on FAISS/BGE retrieval54657b7 -
fix: recreate pipeline each request — ZeroGPU deallocates GPU between calls
- Don't cache self.pipe across requests (GPU state invalid after deallocation) - Move model to GPU, create pipeline, generate, model stays on GPU until next dealloc - Each @spaces.GPU call gets a fresh pipeline on the newly allocated GPU4bac7af -
perf: decouple CPU retrieval from GPU generation — 10x GPU quota savings
- Load retriever + model skeleton on CPU at app boot (0 GPU time) - @spaces.GPU wraps only generator.generate() for ~1.5s forward pass - CPU handles FAISS search + BGE embedding in 0.01s - Reduces GPU duration from ~18s to ~1.5s per query746d35c -
fix: torch_dtype→dtype deprecation, remove pad_token_id conflict with generation_config
665b6f5 -
fix: remove device_map conflict, load onto GPU inside @spaces.GPU context
- Remove redundant device_map='cpu' from pipeline() (model already carries assignment) - Use device_map='cuda' since _load_model() runs inside @spaces.GPU with active GPU - Eliminates ValueError from double device_map and enables ~50 tok/s GPU inference6fbfd9a -
fix: full lazy model loading for ZeroGPU — remove preload thread, defer pipeline init to first request
- Remove _preload() background thread from app.py - Refactor AethronGenerator.__init__ to store nothing; _load_model() called lazily inside @spaces.GPU context - Add device_map='cpu' to pipeline constructor - Bump LLM_MAX_NEW_TOKENS to 512 - Strengthen SYSTEM_PROMPT with strict grounding and contact info rules - Deduplicate source titles in generation output0070274
-
refactor: Gradio 5.30 compat + spaces first import + deferred torch in generation.py
0bb8b99 -
refactor: ZeroGPU-compatible — spaces first import, @spaces.GPU on generate(), deferred torch/transformers
c1272c5 -
fix: dummy @spaces.GPU placeholder — no GPU allocation, no quota consumption (same pattern as neurorag-2026)
acbca58 -
fix: keep @spaces.GPU(duration=15) — zero-a10g requires it, but with HF_TOKEN secret for authenticated quota
31071e1 -
fix: remove @spaces.GPU — run on CPU to eliminate ZeroGPU quota errors for website visitors
d7b1aaa -
fix: use GPU properly — @spaces.GPU + device_map=auto for A10G
09d4d23 -
fix: load pipeline inside @spaces.GPU + force SentenceTransformer to CPU
582cafa -
Aethron Portfolio Agent v3: 27+ projects, 8 verticals, Phi-3.5-mini, expanded KB with domain sections
6c3069e
-
Aethron Portfolio Agent v2: Phi-3.5-mini + FAISS + keyword fallback, HF Spaces compatible
a3037d6