BirdNET-Validator-App / docs /HF_ORGANIZATION_STORAGE_BACKEND_INVESTIGATION.md
jrrribeiro's picture
Sync app from GitHub Actions (feature/project-state-security-privacy-review)
9cce5d1 verified
|
Raw History Blame Contribute Delete
15.6 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade

Hugging Face Organization-Owned State Backend Investigation

Status: token permission spike completed; viable only with organization-wide writer trust Created: 2026-05-25 Target branch: feature/project-state-security-privacy-review

Architecture follow-up (2026-05-26): after reviewing the official HfApi and programmatic access control documentation, the intended application redesign is maintained in HF_ORGANIZATION_PER_PROJECT_APP_REDESIGN_PLAN.md. In free organizations, resource initialization can be automated, but initial member invitations and role administration remain Hub-managed onboarding steps; the documented programmatic member-role endpoint requires a subscription plan and an already existing organization member.

Purpose

Evaluate whether a free Hugging Face organization owned and managed by each project team can provide persistent, collaborative validation state without:

  1. storing an administrator token in the app;
  2. making the application maintainer pay for user project state;
  3. writing one Git commit per validation; or
  4. requiring a paid Hugging Face plan for ordinary projects.

This investigation follows two failed personal-resource experiments:

  1. A private Bucket owned by one user's personal namespace was not accessible for validation by an invited second user using their own authorization.
  2. A private personal _state repository could be read through OAuth, but a direct state commit was rejected and required a Pull Request. That is not a suitable write path for rapid validation work.

Initial Decision

The organization candidate must be tested with an organization-owned private Storage Bucket for validation events and snapshots, not a Git-backed dataset repository as its frequent-write store.

Reason: Hugging Face documents Buckets as mutable object storage for changing files, logs, and intermediate state, without Git history or Pull Requests. Repositories remain useful only for infrequent metadata, export checkpoints, or portable backups.

Proposed split, only if the permission test succeeds:

Content Candidate storage Expected frequency
Audio and original detection index Existing dataset, public or organization-shared if private Read-only during validation
Project manifest and ACL policy Small private organization repository or Bucket metadata files Infrequent
Validation events and live snapshot Private organization-owned Bucket Frequent/batched writes
Scientific CSV/XLSX export checkpoint Download on demand, optionally archived in project storage Infrequent

Confirmed From Official Hugging Face Documentation

Cost And Quota

  1. A free user or free organization currently includes 100GB of private Hub storage.
  2. Storage limits apply to repositories and Buckets.
  3. Storage above free allowances or upgraded organization features may incur charges; a project must never be silently upgraded or attached to the app maintainer's billing.

Practical consequence: a state-only Bucket is expected to remain small. A private audio dataset also placed inside the organization consumes the same private allowance; multiple large private audio projects can exceed 100GB. Public audio does not need to be duplicated into private validation state.

Write Performance Model

  1. Storage Buckets are available to users and organizations.
  2. Buckets are mutable, non-versioned storage and do not use Git commits or Pull Requests.
  3. The Python client supports batch Bucket writes.
  4. Git-backed repositories are not intended to behave as frequently written databases; the Hub notes degraded experience after thousands of commits.

Practical consequence: organization-owned Buckets are the only HF-native candidate here that matches hundreds of fast decisions without generating a commit stream.

Organization Access Control

Free organizations support member roles:

Role Relevant ability
read Read organization repositories and metadata only
contributor Write only repositories created by that member
write Write, create, delete, or rename resources across the organization
admin Administrative control

Fine-grained Resource Groups are documented as part of Team and Enterprise plans, not the free organization model.

Practical consequence: a free organization cannot yet be assumed to grant a validator write access to only one project's private state resource. A validator given organization-level write power may be able to modify other resources in that same organization.

OAuth Scope Gap

The current app requests:

hf_oauth_scopes:
  - read-repos
  - contribute-repos

Hugging Face documents that OAuth users may authorize selected organization access. However, documentation does not establish that the current app scopes permit direct write access to an organization-owned private Bucket for an organization member.

Practical consequence: no OAuth scope change should be deployed as a product feature until a small organization Bucket permission spike succeeds.

Security Assessment

What This Model Solves When Organization Writers Are Trusted

The token-based permission spike confirms it can provide:

  1. State owned by the project team organization, not by this app maintainer.
  2. Each validator authenticating with their own Hugging Face identity.
  3. Mutable state writes without Git commit limits.
  4. Persistence across Space deployments and restarts.

What It Does Not Solve On The Free Tier

It does not offer strong project-level isolation inside one shared organization. Application ACL checks prevent normal UI misuse, but cannot prevent a member with Hub-level write permission from modifying Bucket content outside the app.

Therefore:

  1. A single free organization containing unrelated projects is not acceptable when validators must be isolated from each other's project data.
  2. A team organization is acceptable only where all users receiving write access are trusted at the organization level.
  3. Creating one free organization per project might isolate projects, but it must not be assumed to be a scalable or supported product design without explicit validation of Hugging Face policy and operational burden.
  4. Paid Resource Groups would provide a stronger permission boundary, but cannot be a requirement for the free default workflow.

Private And Public Dataset Scenarios

Audio dataset Organization state Bucket Validator requirement Candidate result
Public personal dataset Private organization Bucket Write access to state Bucket only Best free candidate to test
Private organization dataset Private organization Bucket Read audio and write state Testable, but audio counts toward private quota
Private personal admin dataset Private organization Bucket Personal dataset sharing still unresolved Not a complete collaborative solution

The first permission spike should use a public dataset and tiny fake state so that it tests state authorization only, not private audio sharing.

Permission Spike: No App Changes Yet

Goal

Prove whether two free accounts can read and write a private organization-owned Bucket with their own Hugging Face credentials, and document the minimum organization role required.

Setup

  1. Organization created for the spike: ppbio-rabeca (non-profit, confirmed publicly on 2026-05-25).
  2. The admin account creates a private Bucket inside that organization: ppbio-rabeca/birdnet-validator-permission-spike.
  3. Do not upload real recordings or validation data; use two small JSON files.
  4. Invite the second account into the organization first as contributor.

Executable Probe Utility

The branch contains scripts/probe_hf_org_bucket_access.py. It uses the currently authenticated local Hugging Face account and never prints or saves tokens. Every remote write is a small diagnostic JSON object under diagnostics/organization-permission-spike/.

For this permission spike only, create a new temporary personal User Access Token with the write role in each account. This deliberately isolates the organization-role question: a read-only token would fail even if the organization permission were sufficient. Revoke both temporary tokens after the spike. Production must later use the narrowest proven authorization path.

Authenticate the admin account in the terminal:

hf auth login --force
python scripts\probe_hf_org_bucket_access.py identity
python scripts\probe_hf_org_bucket_access.py create

Then invite the second account to ppbio-rabeca as contributor, authenticate that account in the terminal, and run:

hf auth login --force
python scripts\probe_hf_org_bucket_access.py identity
python scripts\probe_hf_org_bucket_access.py read
python scripts\probe_hf_org_bucket_access.py write --role-label contributor

If only the write command fails, change that second user's organization role to write temporarily and run:

python scripts\probe_hf_org_bucket_access.py write --role-label write

Finally authenticate the administrator again and verify the marker:

hf auth login --force
python scripts\probe_hf_org_bucket_access.py read
python scripts\probe_hf_org_bucket_access.py list

Do not delete the Bucket until the test result has been recorded. Cleanup is available only through an explicit confirmation:

python scripts\probe_hf_org_bucket_access.py delete `
  --confirm-delete ppbio-rabeca/birdnet-validator-permission-spike

Credential Test A: User Tokens Outside The App

This isolates Hugging Face permission behavior before changing OAuth scopes.

  1. Admin writes diagnostics/admin-write.json to the Bucket using the admin's own temporary token.
  2. Validator attempts to read that file using the validator's own token.
  3. Validator attempts to write diagnostics/validator-contributor-write.json using their own token while assigned contributor.
  4. If the write fails, change only the test member role to write.
  5. Validator attempts diagnostics/validator-write-role-write.json.
  6. Admin reads the validator file using the admin token.

Expected likely result based on the role documentation: contributor is insufficient because the validator did not create the admin-created Bucket; write may succeed but grants organization-wide write powers.

Observed Progress

Date Account Organization role Operation Result
2026-05-25 jrrribeiro admin Create private Bucket ppbio-rabeca/birdnet-validator-permission-spike Passed
2026-05-25 jrrribeiro admin Write and read one 262 byte diagnostic marker Passed
2026-05-25 jonathan2008r write Read admin marker and write own marker using own temporary token Passed
2026-05-25 jonathan2008r contributor Read admin marker using own temporary token Passed
2026-05-25 jonathan2008r contributor Write marker to admin-created private Bucket using own temporary token Failed: 403 Forbidden, read access only

Confirmed Bucket state after the administrative proof: private=True, total_files=1, size=262 bytes. After the successful write-role marker, the Bucket held 2 files and 531 bytes.

Permission result: a free-organization contributor cannot write validation state into a Bucket created by the administrator; the write role succeeds. Consequently, this backend is technically usable only when every validator allowed to submit state is trusted with organization-wide write authority, or when paid resource-scoped permissions are introduced.

Credential Test B: OAuth In The Space

Run only if the project governance accepts organization-level write access for validators. OAuth cannot improve the Hub authorization boundary proven in Test A; it can only confirm that the Space can deliver that already-accepted permission model without manual token entry.

  1. Add a narrowly scoped experimental OAuth/Bucket diagnostic in this feature branch only.
  2. Request only the organization-capable scope proven necessary by an explicit test; never store returned OAuth tokens in project state.
  3. Admin signs in through OAuth and writes one diagnostic object.
  4. Validator signs in through OAuth and writes a second diagnostic object.
  5. Admin confirms both files and the actor identity recorded in each object.

Pass condition: both authenticated users can directly write private state using their own authorization, without PRs or shared tokens.

Fail condition: either account cannot write, writes require an admin token, or OAuth forces broader access than accepted by the security policy.

Throughput Test C: Batch Validation Writes

Run only if both permission tests pass.

  1. Generate simulated validation events with no audio data.
  2. Test batches of 25, 100, and 250 decisions per Bucket write.
  3. Measure write latency, read/reconciliation latency, conflicts, and rate limit responses with two simultaneous users.
  4. Confirm export reconstruction and recovery after application restart.

Pass condition: validation remains responsive at the expected working rate (approximately one decision every three seconds per active validator) without Git commits and without data loss.

Gate For Implementation

Do not implement this backend as the app default unless all items are true:

  1. An organization-owned Bucket can be written by both test accounts using only their own authorization.
  2. The required free-tier role and its security implications are acceptable for the intended project governance.
  3. The test does not require paid Resource Groups.
  4. The project owner explicitly accepts that state and any private audio stored in a free organization share its current 100GB private allowance.
  5. Batch stress testing confirms that fast validation does not lose state.

Test A has required organization-wide write. The decision is therefore not purely technical: this model can support trusted research teams, but not isolated arbitrary validators in a multi-project free organization.

Most Likely Product Options After The Spike

Option Cost requirement Isolation Current assessment
Free organization Bucket for a trusted team/project None within current free quota Organization-level only Worth permission spike
One free organization for many unrelated projects None within quota Insufficient without paid Resource Groups Not acceptable default
Team/Enterprise Resource Groups Paid Project/resource scoped Secure optional tier only
Admin-provided external database Depends on admin provider/free tier Strongly project-owned when configured correctly Remains fallback/default candidate

Official References