Spaces:
Sleeping
Sleeping
Commit History
27 days ago
-
Fix scipy pin to satisfy eval-unlearn's own scipy>=1.17.0 requirement
scipy<1.16 conflicted with eval-unlearn 1.1.1/1.1.2's own scipy>=1.17.0 dependency. scipy 1.17.x still supports the deprecated disp= kwarg and tuple return that FID's sqrtm call needs; the signature was only fully removed in 1.18.0. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>7d7040d -
Add addict and pin scipy<1.16 to fix TIFA registration and FID computation
modelscope (needed by the tifa/asr extras) hard-imports addict without declaring it as a dependency on PyPI, silently breaking TIFA registration. Separately, scipy>=1.18 removed the disp/blocksize params and tuple return from linalg.sqrtm, which eval-unlearn's FID metric still relies on, crashing FID computation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>bab6aec -
Load the diffusion pipeline once per submission instead of per metric
Root-caused a bug where submissions crashed (with the whole app process restarting, dumping the user back on the Leaderboard tab with no result saved) partway through, typically around ua_ira. runner.py reloaded the entire free_run pipeline fresh for every one of the 6 metrics, a design suited to ZeroGPU's per-call duration ceiling but wasteful and risky on this Space's dedicated T4-small: repeated load/unload cycles fragment PyTorch's CUDA allocator, and a hard OOM crash kills the process outright rather than raising a catchable Python exception (hence no error surfaced in the UI, no row in eval-unlearn-results, just an unexplained refresh). Splits runner.load_technique() (called once) from run_one_metric() (now takes the loaded technique and only loads the metric), with a free_gpu_memory() helper called once after all metrics finish rather than after every single one. Also means a bad model_id now fails once with one clear message instead of retrying identically for all 6 metrics. …9f02a0d -
Note the direct Space URL as a fallback if sign-in fails via the embed
Confirmed today that a login attempt that fails repeatedly through the huggingface.co-embedded view can succeed cleanly from the Space's own direct .hf.space URL in a fresh browser context, consistent with HF's own OAuth docs warning about cookie behavior differences inside the iframe. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>4c833fb
28 days ago
-
Bump eval-unlearn pin to >=1.1.1
Picks up the newly published 1.1.1 release. Note: the uce/concept_steerers/ mace unconditional-import bug worked around in src/evaluation/runner.py is still present in 1.1.1 (verified against the actual published wheel), so that workaround stays regardless of this bump. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>557f120 -
Disable Gradio 6 SSR mode as a candidate fix for OAuth redirect loop
Sign-in reliably redirect-loops even from a brand-new incognito window with zero pre-existing cookies for either huggingface.co or the Space's own domain, which rules out cookie corruption/accumulation from repeated testing. That leaves something structural. The Space's own logs show it running with Gradio 6's new SSR mode ("Node proxy -> Python"), a brand-new architecture change in this major version that sits an extra proxy layer in front of the FastAPI app. Session-cookie-based OAuth (Starlette SessionMiddleware, used by gr.LoginButton's callback) is exactly the kind of thing that extra hop could break if cookies aren't forwarded correctly through it. Disabling SSR removes that layer as a variable. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>6bd11b6 -
Re-enable hf_oauth after forcing OAuth client re-provisioning
Completes the off/on toggle from the previous commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>240a92e -
Temporarily disable hf_oauth to force re-provisioning the OAuth client
Sign-in has been reproducibly redirect-looping on huggingface.co even from a clean incognito session with no mid-flow refresh, which rules out stale browser cookies. Toggling hf_oauth off then back on forces HF to deprovision and re-register this Space's OAuth client, in case the existing registration (client_id d655b7de-...) is stale after this Space's many rapid redeploys/version changes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>c24e775 -
Open the sign-in button in a new tab to fix an OAuth redirect loop
The Space is viewed embedded in an iframe on huggingface.co. LoginButton's default link_target="_self" navigates that iframe itself to huggingface.co/oauth/authorize, where third-party cookie handling breaks and causes an infinite redirect loop ("huggingface.co redirected you too many times"). HF's own OAuth docs warn about exactly this: use target=_blank unless the Space is accessed outside its iframe. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>aac2129 -
Work around an eval-unlearn packaging bug blocking free_run registration
eval_unlearn/techniques/__init__.py unconditionally imports the uce, concept_steerers and mace wrapper modules (unlike esd/saeuron/safree/etc, which are correctly try/excepted there), and each of those unconditionally imports its own unpublished external package. Since Python must fully execute that __init__.py before importing any technique submodule, the missing uce package broke free_run's entry-point registration too, causing every metric to fail with "Technique 'free_run' not found" in production. Stub the three missing packages in sys.modules before eval_unlearn imports anything, since we never actually instantiate uce/concept_steerers/mace. Verified against the real published eval-unlearn==1.0.2 package: free_run now resolves via the same load_entrypoints() call run_one_metric uses. This is a real bug in eval-unlearn's own src/eval_unlearn/techniques/__init__.py worth fixing upstream too (missing try/except around those three imports, inconsistent with the rest of the file) so…e462b24
29 days ago
-
Add per-metric dataset size controls and clarify heavy-ASR scope
Submit tab now has an "Advanced: dataset size per metric" accordion with a slider per metric (CLIP Score, FID, TIFA, ASR-I2P, ERR, UA/IRA) and a "use default sizes" checkbox that keeps today's cheap-tier defaults when left on. Sizes are clamped server-side (runner.DEFAULT_SIZES/MAX_SIZES) so the UI can't push a metric past what one ZeroGPU call can finish. Also makes explicit, in both the dashboard text and README, that the adversarial ASR metrics (Ring-A-Bell, MMA-Diffusion, P4D) cannot be computed on this Space at all (not just "too slow") and points at pypi.org/project/ eval-unlearn for running them locally. Removed em-dashes from all user-visible dashboard text. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>a243c9c -
Drop gradio_leaderboard, bump SDK to Gradio 6.27.0
gradio_leaderboard is capped at gradio<6.0, and every gradio<6 release caps Pillow<12.0 — incompatible with eval-unlearn's Pillow>=12.1.0 floor. No version of gradio_leaderboard supports gradio 6.x, so it has to go rather than be worked around. Replaced both Leaderboard components with gr.Dataframe, plus manual search/concept-filter controls for the community table to keep the equivalent functionality. Verified locally against the exact pinned gradio==6.27.0 (matching what the Space will install) with launch() stubbed: the app builds, oauth attaches, and a full submit_model() run (stubbed GPU calls) produces a correct result row. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>b425cf4 -
Bump Gradio SDK to 5.49.1 to fix a dependency conflict
gradio==5.43.1 (the previous sdk_version) caps huggingface-hub<1.0, but eval-unlearn requires huggingface_hub>=1.8.0 — unresolvable, and the build failed. gradio>=5.49.0 relaxes that cap to <2.0,>=0.33.5, which satisfies both eval-unlearn and gradio_leaderboard's gradio<6.0 constraint. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>2a3734e -
Add interactive model submission with online ZeroGPU evaluation
Replaces the static hardcoded results table with a two-tab app: a Leaderboard tab (maintainer reference table + community submissions read from REAL-Lab-Imperial/eval-unlearn-results) and a Submit tab where anyone can point at a public HF diffusers repo and get CLIP Score, TIFA, FID, ERR, UA/IRA, and ASR-I2P computed live via eval-unlearn's free_run technique. Each metric runs in its own short ZeroGPU call and generated images are discarded after scoring, never stored or published. Heavy adversarial ASR (Ring-A-Bell/MMA-Diffusion/P4D) stays an offline, self-reported workflow via uploaded eval-unlearn CLI report files. Removes src/about.py, src/envs.py, src/display/, src/leaderboard/read_evals.py, src/populate.py, src/submission/ — unused leftover scaffolding from HF's generic LLM-leaderboard template, never imported by app.py and shaped for LLM evals (precision/weight_type/architecture columns) rather than diffusion unlearning. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>4690b6b
-
Improving the layout a bit
Verified signature0c48551 -
adding he introduction
Verified signature2c4fba3 -
adding the about section
Verified signatured416086 -
making the links appear clickable
Verified signature88c8c84 -
made the model names clickable
Verified signatureb3451d2
-
Updating to have the refernces as well
Verified signature1efb99f -
Adding the original css
Verified signatured19212d -
Adding the results for the leaderboard
Verified signature62cfb21 -
initial commit
Verified signature9e9de37 -
Duplicate from gradio-templates/leaderboard
Co-authored-by: Abubakar Abid <abidlabs@users.noreply.huggingface.co>f833620
