Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
Commit History
-
JSON import: convert to the form layout, load for editing; accept slash/URL DOIs Running
Import: uploaded rubrics are rearranged into the exact layout the form shows (every dimension block for the type, in order, padded to the default criterion rows), so a partially-specified derivation_required question gets all three blocks instead of one. The button is always enabled and now 'Load into the form'; it collapses the import panel, clears the uploader and shows the result banner at the top of Questions (load/copy/import results now appear there; Save draft/Submit results stay by their buttons). A draft is still saved automatically when DOI + username are filled. DOI: normalize_doi() maps 10.1056/NEJMoa2511478 and doi.org links to the stored 10.1056_nejmoa2511478; applied to the DOI field on edit, every read site, the trials.csv join, and both table searches. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>f39cb78 -
Add a global search box (matches all columns) to the trial and reference tables
Reactable-style search: one input at the top of each table; every whitespace-separated term must appear somewhere in the row (any column, case-insensitive), so "oncology phase 3 nejm" works. The old per-column boxes are kept as optional filters inside a "Filter by column" expander and combine with the global query (AND). Shared logic in lib.ui.row_matches. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>878d20c
-
Reference browser: searchable table showing which trial each benchmark is for
The 'See how others filled the form' panel was a dropdown of 'user · DOI · #Q · date' — a bare DOI doesn't tell you what the benchmark is about. It is now a searchable table with the same UX as the trial browser: - Rows are joined with assets/trials.csv (case-insensitive DOI match) so each shows Paper Title, Journal, Year, Therapeutic Area and Phase alongside Username, DOI, #Questions, #Versions and Submitted. Unknown DOIs keep their row with blank trial fields. - Six per-column search boxes filter the table (AND, case-insensitive), capped at 100 rows like the trial browser. - Click a row to preview it (native st.dataframe single-row selection, with a selectbox fallback if the running Streamlit lacks on_select); the preview is headed by the paper title, then the existing read-only rendering and the 'Copy these questions into my form' button. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>857049a -
Import externally written questions from a JSON file as the first draft
People sometimes write their questions outside the intake system. The form now has a toggle-gated 'Import questions from a JSON file' panel: - lib/schema.py: validate_and_normalize_prompts() checks an uploaded payload (bare list, {prompts: [...]}, or a full saved record) and reports precise per-field errors; it leniently normalizes enum casing, assigns missing ids, maps an unknown design_element to Others, generates empty dimension blocks when rubrics are omitted, fills importance/scoring defaults, and converts the legacy single-criterion rubric shape. example_import_json() provides a downloadable template that round-trips through the validator. - app.py: template download + field reference + uploader; errors block the import, warnings are shown, valid input is previewed read-only, and 'Import as first draft' populates the form and saves a draft under the DOI + username (app-scope rerun so the questions fragment refreshes). Verified: template validates cleanly and n…2a9e0f5
-
Actually make the Question box grow: native content height + CSS that wins
Root cause of the no-op: Streamlit sets the textarea height as an INLINE style, so the stylesheet's field-sizing rule never applied. - lib/ui.py feature-detects height='content' (Streamlit >=1.44) from the docstring and uses it when available, falling back to the pixel estimate on older versions. - The CSS now overrides the inline height (height: auto !important), gated behind @supports (field-sizing: content) so unsupported browsers keep Streamlit's height instead of collapsing. - Bump the Space's sdk_version 1.39.0 -> 1.58.0 to match what we run locally, so the native content height is available there too.1fd676d -
Grow the Question/criterion boxes with their content
height='content' only exists in Streamlit >=1.44 and the Space runs 1.39, so lib/ui.py adds autoheight(): a pixel height estimated from the text (clamped 80-420px), used by the Question box in the form and the Admin read-only view and by the criterion boxes. A one-line CSS rule (field-sizing: content) additionally grows the box live while typing on browsers that support it; where it isn't supported the computed height still fits the saved text.f2ce67c -
Show long questions in full (Question field -> text_area)
The Question field was a single-line text_input, so a long question was clipped to one scrolling line. It is now a wrapping text_area in both the intake form and the Admin read-only view, so the whole question is visible.a65a1eb
-
Add a viewable system prompt (link + in-app expander)
Links to lib/agent.py on GitHub next to the demo video, and adds an expander that renders the live SYSTEM_PROMPT with a download button so it can never drift from the code. Importing SYSTEM_PROMPT is safe without the LLM SDKs installed — lib/agent.py only imports stdlib at module level (anthropic/openai are imported inside the call functions).0d5e141
-
Add a demo video link under the form title
Links out to the Google Drive video in a new tab (Drive /view pages can't be iframed, and inline embedding was what made the PDF panel laggy/blocked).a0cccae
-
Revert "Password-gate the intake form (separate from the reviewer password)"
This reverts commit 60008e1356b4c36258cbfd291e6a3bdcd44fbaba.7765139 -
Password-gate the intake form (separate from the reviewer password)
- storage: FORM_PASSWORD env var + check_form_password(), mirroring the admin gate; unset means open (dev mode). - app.py: password screen before the form renders, same pattern as the Admin page. The two passwords are independent — neither opens the other's page. - README: document FORM_PASSWORD in the Space secrets table.60008e1
-
Disable the Run agent page for deployment
Moved pages/2_Run_Agent.py -> disabled_pages/ so Streamlit no longer loads it (the work is preserved and re-enabling is one git mv). Also dropped apt.txt and commented out anthropic/openai in requirements.txt, since only that page needed them — this avoids installing r-base-core and the LLM SDKs on the Space, making the build much faster. app.py and the Admin page never imported the agent code, so nothing else changes; llm/run_llm.py still uses lib/agent.py locally. README documents how to re-enable.e764714 -
Intake form: browse other people's submissions as a reference
- storage.list_reference_submissions(): latest version of every submission, one file read each and no review loading (much lighter than list_submissions); drafts and agent runs excluded, real DOI preserved. - app.py: toggle-gated 'See how others filled the form' fragment with a dropdown (user · DOI · #Q · date), a read-only markdown view of the questions and rubrics (incl. importance/scoring, with legacy fallback), a Refresh button that clears the 5-min cache, and a 'Copy these questions into my form' button that populates the editor and triggers an app-scope rerun so the (separate) questions fragment picks it up.1b1eb38 -
Turn Run agent into a real agentic loop with a workspace and tools
The page was a single LLM call with the whole SAP in the prompt. Now: - lib/tools.py: a Workspace sandboxed to one root, exposing list_files, read_file (line-paged), grep, write_file and run_r (Rscript with a timeout). Paths are confined chroot-style; traversal is rejected. - lib/agent_loop.py: the tool-use loop for both providers. The agent gets the document as a file (sap.txt/protocol.txt with page markers) instead of ~120K tokens of prompt, greps/reads it on demand, writes output.json + output.R into the workspace, and can run the R script and fix it before finishing. Tracks iterations, token usage and a full transcript, with an iteration cap. - Page: builds a temp workspace per run, streams each tool call to a live status log, collects the files the agent produced, and saves the transcript/usage alongside the outputs. - apt.txt installs r-base-core so run_r works on the Space; without R the tool says so and the agent continues. Verified with stubbed SDKs (no API …74239a8 -
Run agent: cascading provider -> model -> effort -> key; current models
Verified against the providers' docs (Aug 2026): - Add a MODELS registry in lib/agent.py — Anthropic: claude-opus-5 (default), claude-fable-5, claude-sonnet-5, claude-haiku-4-5-20251001, plus 4.8/4.6 legacy; OpenAI: gpt-5.6-sol/terra/luna. Replaces the stale gpt-4o/opus-4-8 suggestions. - Effort is passed correctly per provider: Anthropic needs output_config={'effort': ...} (NOT a top-level arg), OpenAI uses reasoning_effort. Per-model level lists (Haiku 4.5 supports none; Sonnet 4.6 has no xhigh) drive the picker, and high-effort Anthropic runs get a larger max_tokens since thinking counts against it. - Page now selects provider, then model (or a custom id), then reasoning effort, then the matching API key — one model per run. - save_agent_run records provider + effort; past runs show the effort used. - run_llm.py gains --effort and updated DEFAULT_MODELS.7fccc75 -
Add 'Run agent' page: run LLMs on a submitted version and view results
- New pages/2_Run_Agent.py: pick a submission version, choose models (claude-* -> Anthropic, gpt-*/o*-* -> OpenAI, or type a custom id), paste your OWN API key (never stored), and run the trial-statistician prompt against the trial's SAP/protocol text. Answers render per question with output.R and download buttons; past runs are browsable. - New lib/agent.py: single source of truth for the prompt, SAP loading, prompt block, model calls (explicit api_key) and response parsing. - llm/run_llm.py now imports from lib/agent.py instead of duplicating the prompt (verified: --help and a full offline run still work). - storage: save_agent_run/list_agent_runs write to submissions/<DOI>__<user>/agent_runs/<stamp>__<model>.json; agent runs are excluded from list_versions and the admin list_submissions. - requirements.txt: add anthropic + openai for the Space.9ceb217 -
PDF panel: only show links for DOIs that have a source document
Validate the entered DOI against the known set (from trials.csv); show a warning instead of dead links for unknown DOIs. Falls back to showing links if the trials list can't be loaded.22d63d4
-
Fix trial-browser lag: gate table behind a toggle + cap to 100 rows
An st.expander renders its contents on every rerun even when collapsed, so the 1641-row dataframe was rebuilt/re-sent constantly (laggy 'Running', occasional 500s). Now the table renders only when a toggle is on, and is capped to 100 rows (refine search to narrow).696e43f -
Drafts are timestamped; Load draft restores the latest
- save_draft writes drafts/<stamp>.json (keeps history) instead of overwriting a single draft.json; get_draft returns the newest draft. - Messages updated; still excluded from version/admin listings.b92c42a -
Persist Save draft to a drafts/ subfolder + add Load draft
- save_draft() writes submissions/<DOI>__<username>/drafts/draft.json (overwrite); get_draft() reads it back. - Form: Save draft now persists (needs DOI + username); new 'Load draft' button next to 'Find versions' restores it. - Drafts are excluded from list_versions and the admin list_submissions (paths containing /drafts/ are filtered out). - Refactored the load path into a shared _populate_form() used by both version load and draft load.2e38fe8
-
Rename Scoring field label to 'Scoring direction' (key unchanged)
f3fbf7b -
Add per-criterion scoring (Add/Deduct points)
Each criterion now has a Scoring selectbox (Add / Deduct) next to Importance, defaulting to Add. Threaded through build/load/seed, content hash, and the admin read-only display.e53de88
-
Trial browser: remove Paper Link column
Drop Paper Link from TRIAL_COLS, the search layout, the dataframe, and assets/trials.csv.b67f6a4 -
Rename trial browser column document_id -> DOI
e9b2a97 -
Add searchable trial browser to the form
- Export tdr.parquet's key columns (Journal, Year, Therapeutic Area, Paper Title, Paper Link + Phase) plus a derived document_id to assets/trials.csv (1641 rows, PDF-backed only). - New 🔎 Browse trials expander: per-column search boxes filter the table; copy a document_id into the DOI field. Runs in a fragment; reads the CSV with stdlib csv (no pandas at runtime).62511e9
-
DOI is case-insensitive: normalize to lowercase
Lowercase the entered DOI wherever it's used as an identifier (PDF links, submission storage path, version lookup) so case doesn't matter and the same DOI always maps to the same document folder / submission.36898d4
-
Remove the tolerance field from criteria
Each criterion is now just criterion text + importance. Dropped tolerance from the Criterion type, content hash, form UI, build/load, and the admin display (old saved tolerances are simply ignored on load).842481b -
Default Importance to Medium
Seed new criteria's importance to 'Medium' (in _on_type_change and _add_criterion) and use it as the build/load fallback, keeping the High/Medium/Low display order.a57f7cf -
Move PDF reference links under the Find versions button (single column)
- render_pdf_panel() now renders inline right after 'Find versions' instead of a left side column; revert to centered single-column layout. (PDF panel is just links now, so no side-by-side needed.)9d454da
-
Consistent casing: importance High/Medium/Low; capitalize field labels
- IMPORTANCE_OPTIONS now title-case (High/Medium/Low); load seeding maps old HIGH/medium/low case-insensitively. - Capitalize visible labels: Username, Design element, Question type, Question, Criterion N, Importance, Tolerance (form + admin read-only).086c738 -
Method dimension: 3 default criteria (extras optional, empty ones not saved)
- dimensions_for_type carries a default_criteria count; Method=3, others=1. - Extra criterion rows are labeled '(optional)'. - _build_prompts drops any criterion with empty text, so unfilled optional rows are not saved.042edfb -
Rubrics: drop Reproducibility, multiple criteria per dimension, importance
- derivation_required now has 3 dimensions on output.json (Inputs used, Calculated value, Method) — Reproducibility/output.R removed. - Each dimension holds multiple criteria; each criterion has its own criterion text, importance (HIGH/medium/low), and tolerance ('points' replaced by an importance selectbox). - Stable criterion ids (cid) keep widget state correct across add/remove. - Admin renders criteria read-only; content hash + load handle the new shape (with backward-compat for old single-criterion records).23b513a -
Label trial_id field as 'DOI' with example 10.1200_jco.22.01989
UI label only; the internal key stays trial_id. Placeholder + help updated, and the PDF panel hint now says DOI.322e8c4 -
PDF panel: link out (new tab) and use trial_id directly as document id
- Drop the inline base64 embed entirely (kills lag) — just open-in-new-tab links to sap.pdf / protocol.pdf. - Use the entered trial_id directly as the document id (e.g. 10.1200_jco.22.01989), removing the ambiguous NCT->document mapping. - Remove now-unused assets/nct_to_docs.json and the base64/iframe/map code.02ff7d5 -
Fix PDF lag: isolate questions editor in an st.fragment
Editing the form previously triggered a full-app rerun, re-sending the ~3MB base64 PDF each time (very laggy). The questions editor + actions now run inside an st.fragment, so edits there rerun only that fragment — the PDF panel (outside the fragment) is not re-rendered. Falls back to a no-op decorator on Streamlit versions without st.fragment.b431be7 -
PDF viewer: embed via base64 data: URI instead of iframing HF
Chrome blocks iframing huggingface.co (X-Frame-Options). Now the PDF bytes are fetched server-side (public dataset, cached) and embedded as a data:application/pdf;base64 iframe, which isn't subject to cross-origin frame blocking. Open-in-new-tab link + download button remain as fallbacks.afc5741 -
Add SAP/protocol PDF viewer beside the form
- Wide two-column layout: left panel shows the trial's SAP (or protocol) PDF, resolved from the entered trial_id via assets/nct_to_docs.json (precomputed NCT->DOI-folder map, served from the public source dataset). Right column is the form (refactored into render_form()). - PDF embedded via iframe + 'open in new tab' link + download button fallback. - No new runtime deps (map is precomputed; uses requests already present).09ecb5e -
LLM runner: add --version to pin an exact submission version
--version accepts a bare filename, full repo path, or URL-encoded %2B and normalizes to the literal '+'. Without it, the latest version is used.435a487 -
LLM runner: uv inline deps + clearer latest-version selection
- Add PEP 723 inline script metadata so 'uv run run_llm.py' auto-installs anthropic/openai/huggingface_hub/pandas/pyarrow (fixes 'No module named anthropic/openai'). - load_submission sorts version files by basename timestamp and logs how many versions were found and which (latest) is used. - README documents the uv workflow.2b1b795 -
LLM runner: support fully-local input (no HF read/write)
- add --submission-file and --sap-file to read inputs from local disk, skipping HF entirely (no HF_TOKEN needed). Outputs were already local-only; nothing is ever uploaded to HF.4c913da -
Add LLM runner to populate submission answers from the SAP
llm/run_llm.py: loads a submission (private intake_form_data dataset) + its parsed SAP (public source dataset, sap.lines.json with page markers), builds the null-placeholder prompt block, and runs each configured model (Anthropic / OpenAI) with the trial-statistician prompt to produce output.json + output.R per model, written under llm/out/. NCT->doc resolved via data/tdr.parquet.2f8293f
-
Fix 'have to type twice' bug: session_state as single source of truth
The form mixed value=/index= with key= and wrote back to the dict each render, which made edits lag by one rerun (needing a second Enter). Now each question is keyed by a stable uid, widgets use key= only (no value=/index=), state is seeded on load/add/type-change, and the payload is read from session_state at submit.dc4c942 -
Admin: fix nested-expander error (raw JSON now a checkbox toggle)
Streamlit forbids an expander inside another expander; the trial row is an expander, so the 'Raw submission JSON' inner expander was replaced with a checkbox toggle.7427811 -
Move 'modified since last review' flag from app to Admin
- App: per-question badge shows the latest review status only (no stale flag). - Admin: each question shows '✏️ Modified since last review' when the latest version's content differs from the hash recorded on its most recent review, so the reviewer knows it needs re-review.8af357f -
Invalidate a question's review status when the question is edited
- add schema.question_content_hash(): stable hash of a question's content (excludes id). - add_review() snapshots the reviewed question's content hash. - Form compares the current question's content hash to the latest review's stored hash; if they differ, shows '✏️ Modified since last review — resubmit for re-review' instead of the stale Pass/needs_fix badge.3475963 -
Form: show full review history across all versions (not just loaded one)
- add storage.pair_reviews(trial_id, username) returning all reviews across every version, tagged with version. - Form fetches it on 'Find versions'; overall feedback box and per-question feedback now show the whole history with version tags, and the per-question Pass/Needs fix badge reflects the latest review across versions.916f12b -
Admin: single batch review submit instead of per-question buttons
- One form per trial: reviewer name (shared), a status selectbox + comment for each question, plus an overall status + comment, and one 'Submit all reviews' button. Questions left as '— no review —' are skipped. - On submit, every chosen review is written (per-question or overall), then the form inputs reset.e5a0f0d -
Form: show a Pass / Needs fix / Pending badge per question
When a loaded version's question has reviews, show its latest review status as a badge — '✅ Pass' for reviewed, '🔴 Needs fix', or '🟡 Pending' — above the per-question feedback.07a6a51 -
Form: show per-question reviewer feedback inside each question block
- Per-question reviews (question_id matching a question) render in a bordered box within that question's container after loading a version. - The top feedback box now shows only overall reviews; per-question feedback appears next to its question.7225056 -
Per-question reviews; render questionnaire in app layout
- add_review() takes question_id; reviews can target a specific question or be overall (empty question_id). Review file name carries the question tag. - Admin renders each question in the form's layout (read-only) with its own review thread + 'Add review for this question' form. - Overall status now ignores per-question reviews (driven by overall reviews on the latest version). Overall review history shown separately. - Removed unused _legacy_base_id; README updated.b67309b