bio-nexus-api / implementationplan.md
Samad14's picture
Fix pipeline MSA/AI stall, add SwissADME-parity ADMET panel, PubChem search, docking ligand data
5bb077e
|
Raw History Blame Contribute Delete
8.19 kB
# implementationplan.md β€” BioFlow AI Implementation Plan
> **Current status (v4.0, 2026-08-05):** Track A prototype completed. Track B sprints 1–10 completed. Additional sprints 11–15 (structure suite, docking, MD, ADMET, function, sequencing, design system) shipped. See Track B below.
## Track A β€” Prototype Sprint (18 Days) βœ… Completed
One golden path only: **Sequence In β†’ BLAST β†’ AI-interpreted report.** Everything else is Track B.
### Day 0 β€” Pre-flight (~2 hrs, today)
- [x] Create `bioflow-frontend` and `bioflow-backend` repos
- [x] Create Supabase project, run schema.md migrations
- [x] Create Cloudflare R2 bucket
- [x] Get API keys: Gemini (Resend optional for prototype)
- [x] Confirm NCBI BLAST URL API access β€” **no key required, but max 1 request every 10s**; build this constraint in from day one, not as a fix later
### Days 1–2 β€” Foundations
**Backend**
- FastAPI scaffold + `/health`
- Supabase client wrapper (`core/db.py`), R2 wrapper (`core/storage.py`)
- Pydantic models for `jobs` / `job_steps`
- `POST /jobs` (create, status=`queued`), `GET /jobs/{id}`
**Frontend**
- Next.js + TS + Tailwind scaffold, design tokens from design.md β†’ `tailwind.config.ts` / `globals.css`
- Supabase client (anon key) + anonymous session on first visit
- Layout shell, static landing page
> Job CRUD is the spine everything attaches to β€” get it working with dummy data before touching NCBI.
### Days 3–4 β€” NCBI BLAST Integration (core engine)
- `integrations/ncbi/blast.py`: `submit_blast()`, `check_status()`, `fetch_results()` against the QBLAST URL API
- Background task (start with `asyncio.create_task` β€” sufficient for a single-instance prototype): on job create, submit β†’ poll every 15s β†’ on `READY`, fetch XML, push to R2, update `job_steps`
- XML parser β†’ hit list (accession, description, % identity, E-value, bit score, alignment)
**Risk flag:** NCBI `nr` searches can take 1–5 min and occasionally queue longer. Build a **demo-mode fallback** now β€” pick 1–2 well-characterized sequences (e.g., human insulin), pre-run them, cache the result. This is your insurance for Day 18.
### Day 5 β€” Sequence Input + Validation
- `integrations/ncbi/efetch.py` β€” fetch sequence by accession
- Sequence-type detection (nucleotide vs. protein, composition-based)
- `POST /sequences/validate`
- Frontend: wizard shell, step indicator, Step 1 (operation grid), Step 2 (paste/accession tabs + validation feedback)
### Days 6–7 β€” Wizard β†’ Job Creation β†’ Processing Screen
- Step 3 (confirm/run) β†’ `POST /jobs` β†’ `/jobs/[id]`
- Processing screen: `useJobStatus` polling hook, animated status indicator, friendly per-state copy
- Backend: wire job creation to the Day 3–4 background task, update `job_steps` at each stage
**Checkpoint:** by end of Day 7 you can paste a sequence, hit run, and watch real NCBI status updates. First "it's alive" moment.
### Days 8–9 β€” Results Rendering (raw, no AI yet)
- Frontend: hits table, score-bar visualization (confidence bands), alignment view for top hit
- Backend: `GET /jobs/{id}/results` β€” parsed hits + top-hit alignment from stored XML
### Days 10–11 β€” AI Interpretation Layer
- `services/interpreter.py` β€” Gemini call
- Prompt template at `/prompts/blast_interpretation.md`: takes parsed hits, returns a 2–3 sentence summary + per-term explanations (E-value, bit score, % identity) *in context of this specific result*. System instruction bakes in the "faithful execution, not 100% accuracy" framing from the PRD.
- New step `interpreting` β†’ result stored in `job_steps.result_json`
- Frontend: AI Summary component, "What does this mean?" expandables wired to the AI response
### Day 12 β€” Guest β†’ Account Flow
- Banner component (appflow Β§3.5/3.6)
- Sign-up modal β†’ Supabase `linkIdentity` upgrade
- Post-conversion redirect handling
### Day 13 β€” Dashboard
- `GET /jobs?user_id=`
- Dashboard page β€” job cards, empty state, click-through
### Day 14 β€” Landing Page Polish
- Full design system applied
- Sequence typewriter hero animation
- CTA wiring
### Day 15 β€” Error States & Edge Cases
- Invalid sequence input messaging
- NCBI timeout/failure β†’ `failed` status + retry
- Zero significant hits β†’ its own AI framing ("no strong matches β€” here's what that can mean")
- Rate-limit queueing if multiple jobs fire close together
### Day 16 β€” Deploy
- Frontend β†’ Vercel, Backend β†’ Railway or Render
- Env vars wired per techspec.md
- Full flow test on deployed URLs
### Day 17 β€” Demo Prep + Buffer
- Finalize 2–3 demo sequences with rich, interesting hits; pre-run and cache them
- Fix whatever broke on Day 16
### Day 18 β€” Demo Day
- Final run-through with cached demo-mode results as backup
- Buffer only β€” no new features
---
## Track B β€” Phase 1 Full Build (post-prototype, ~10–12 weeks)
### Sprint 1–2: Pairwise Alignment + Pipeline Chaining Foundation βœ…
- Add Clustal Omega pairwise alignment as the second live operation βœ…
- Implement `parent_job_id` chaining (schema already supports this) βœ…
- Operation grid: two live cards βœ…
### Sprint 3–4: UniProt + PDB βœ…
- UniProt annotation lookup βœ…
- PDB structure fetch + Mol* 3D viewer βœ… (3Dmol.js viewer)
- Chain: "from this BLAST hit β†’ fetch structure" βœ…
### Sprint 5–6: MSA + Phylogenetic Tree βœ…
- Clustal Omega multi-sequence alignment βœ… (multi-method: ClustalOmega/MUSCLE/Kalign/MAFFT/T-Coffee)
- Basic tree construction + visualization βœ…
- Completes the BLAST β†’ shortlist β†’ MSA β†’ tree workflow from your syllabus mapping βœ…
### Sprint 7: Pathway Integration βœ…
- Gene/protein β†’ pathway lookup via Reactome/WikiPathways βœ…
- Pathway diagram viewer βœ…
### Sprint 8: Onboarding + `/learn` βœ…
- First-run tutorial (TutorialWalkthrough component, 5 steps, localStorage flag) βœ…
- 10+ documentation pages at `/learn` (BLAST, Alignment, Domains, Phylo, Structure, Pathways, Interactions, Primers, Tools, Glossary) βœ…
- LearnPopover component for inline `(?)` help tooltips βœ…
### Sprint 9–10: Hardening βœ…
- PDF report export (`GET /api/export/job/{id}?format=pdf|json`) βœ…
- Cache-hit check with `from_cache` flag, `/api/admin/cache-stats` endpoint βœ…
- `@ttl_cache` coverage: pathway enrichment, NCBI search, BLAST, UniProt, AlphaFold βœ…
- Sentry error monitoring (`@sentry/nextjs` frontend + `sentry-sdk` backend) βœ…
### Sprint 11: Pipeline v2 Engine + Domain/Structure Depth βœ…
- 8-step in-memory pipeline: BLAST β†’ UniProt β†’ MSA β†’ Phylo β†’ Domains β†’ Pathway Enrichment β†’ AlphaFold β†’ AI βœ…
- Pipeline wizard with step checkboxes, progressive reveal results βœ…
- Global/local BLAST modes, DNA query support, poll cap 65 min βœ…
- Pairwise: global/local via "Align pair" from BLAST hits + standalone tool + full-length view βœ…
- Domains: PROSITE raw-sequence scan, reviewed/organism UniProt filters βœ…
- Structure analysis suite: Ramachandran, secondary structure, Foldseek comparison βœ…
### Sprint 12: Drug Discovery Compute βœ…
- AutoDock Vina docking: full 1.2.7 log (RMSD l.b./u.b., version, seed), RMSD table, run-config UI βœ…
- MD simulation: verified force field Γ— solvent matrix, startup probe, 25-min budget βœ…
- ADMET: RDKit descriptor computation + traffic-light output βœ…
- Function prediction + protein interactions βœ…
### Sprint 13: Sequencing MVP βœ…
- FASTQ upload β†’ QC β†’ trimming β†’ assembly/consensus β†’ variant calling β†’ annotation βœ…
- SARS-CoV-2 reference, job persistence, progress polling βœ…
### Sprint 14: Reliability βœ…
- AI model fallback chain (Groq β†’ Gemini β†’ Ollama), honest failure banner βœ…
- Share links fixed for all job types + enriched share message βœ…
- Wizard jobs persisted to dashboard history βœ…
- BLAST params honored (program/db/max_hits), long MD jobs not abandoned βœ…
### Sprint 15: Design System (v4.0) βœ…
- Dark-only OLED theme, semantic color tokens, AA text tiers, 4-band confidence bands βœ…
- Geist fonts, Phosphor icons, HUD glass components, tiered motion βœ…
- Landing rebuild: DNA-helix hero, bento features, route-style pipeline βœ…
- Dashboard/tools/results converted to native dark tokens βœ