jrrribeiro commited on
Commit
77fa5a9
·
verified ·
1 Parent(s): 9cce5d1

Sync app from GitHub Actions (feature/project-state-security-privacy-review)

Browse files
README.md CHANGED
@@ -10,7 +10,6 @@ app_file: app.py
10
  hf_oauth: true
11
  hf_oauth_scopes:
12
  - read-repos
13
- - contribute-repos
14
  pinned: false
15
  ---
16
 
@@ -178,7 +177,7 @@ User access bootstrap file example (`BIRDNET_USER_ACCESS_FILE`):
178
  - SDK: `Gradio`
179
  - Python: `3.11`
180
  - OAuth: enabled (`hf_oauth: true` in this README metadata)
181
- - OAuth scopes: `read-repos` for authorized datasets and `contribute-repos` for project state stores created through this app
182
 
183
  2. Push this repository to the Space.
184
 
@@ -199,11 +198,24 @@ Optional runtime settings:
199
  - `BIRDNET_PAGE_SIZE` (default `25`)
200
  - `BIRDNET_ENABLE_DEMO_BOOTSTRAP` (`false` in production)
201
  - `BIRDNET_HF_PROJECT_STATE_WRITES_ENABLED=true` (experimental `_state` diagnostics and recovery only while a new collaborative backend is selected)
202
- - `BIRDNET_HF_BUCKET_VALIDATIONS_ENABLED=true` (legacy experimental access only for projects already declaring Bucket state; new projects no longer create Buckets)
 
 
203
 
204
- Do not create real collaborative projects with either experimental backend yet. The personal private Bucket experiment failed its two-account validator access test. The private `_state` OAuth experiment also failed as a direct validation store: `contribute-repos` allowed repository contribution only through Pull Requests rather than durable direct writes to `main`.
205
 
206
- Projects previously created with a private `_state` repository can be recovered from the **Admin** tab using **Connect Existing State**. Sign in with the Hugging Face account that is recorded as an administrator in that repository's `acl.json`, then provide the `_state` repo id (for example, `owner/audio_dataset_state`). The app loads the saved manifest, ACL, and pending invites; it does not accept a new local definition over an existing state repository.
 
 
 
 
 
 
 
 
 
 
 
207
 
208
  Legacy Supabase state backend, available while migrating existing projects:
209
 
@@ -211,9 +223,9 @@ Legacy Supabase state backend, available while migrating existing projects:
211
  - `SUPABASE_URL` (Secret, example: `https://xxxx.supabase.co`)
212
  - `SUPABASE_SERVICE_ROLE_KEY` (Secret)
213
 
214
- When Supabase is enabled, projects, ACL, invites, validation events, and current validation snapshots are stored in Supabase instead of `/data`. New community deployments should prefer admin-owned Hugging Face project state once the high-frequency state backend is enabled and verified.
215
 
216
- The Bucket validation backend remains available only to diagnose or migrate existing experimental Bucket projects. The **Private state OAuth diagnostic** action in **Projects** now reports when an experimental `_state` repository is limited to Pull Request contributions and must not be treated as a production validation store.
217
 
218
  Optional invite email settings:
219
 
@@ -532,6 +544,10 @@ python -m src.uploader_cli.main start --repo-id alice/birdnet-2026 --segments C:
532
  - `BIRDNET_STATE_BACKEND`
533
  - `SUPABASE_URL`
534
  - `SUPABASE_SERVICE_ROLE_KEY`
 
 
 
 
535
 
536
  ### Uploader CLI Environment Variables
537
 
 
10
  hf_oauth: true
11
  hf_oauth_scopes:
12
  - read-repos
 
13
  pinned: false
14
  ---
15
 
 
177
  - SDK: `Gradio`
178
  - Python: `3.11`
179
  - OAuth: enabled (`hf_oauth: true` in this README metadata)
180
+ - OAuth scopes: `read-repos` only; trusted-team validation writes use each collaborator's explicitly entered personal token
181
 
182
  2. Push this repository to the Space.
183
 
 
198
  - `BIRDNET_PAGE_SIZE` (default `25`)
199
  - `BIRDNET_ENABLE_DEMO_BOOTSTRAP` (`false` in production)
200
  - `BIRDNET_HF_PROJECT_STATE_WRITES_ENABLED=true` (experimental `_state` diagnostics and recovery only while a new collaborative backend is selected)
201
+ - `BIRDNET_HF_BUCKET_VALIDATIONS_ENABLED=true` (legacy experimental access for existing Bucket projects)
202
+ - `BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED=true` (private HF-only delivery mode for small trusted teams)
203
+ - `BIRDNET_HF_PROJECT_STATE_REPOS=project-org/audio-dataset_state` (comma-separated private state repos restored after Space rebuild)
204
 
205
+ The HF trusted-team mode is intended only for confidential projects whose validators are known collaborators. It creates a private `_state` repository for infrequent configuration/ACL updates and a private Storage Bucket for validation events and snapshots. The source dataset must already be private and owned by a Hugging Face organization; personal-namespace datasets are rejected in this mode because they did not support the required second-user write workflow in testing. The demonstrated free-tier permission path uses a dedicated project organization whose trusted validators have Hugging Face `write` membership; users outside the organization cannot access its private resources. Select `collaborative` when creating the project inside the app: this authorizes approved app members and does not make the Hugging Face resources public. Each member works with their own HF authorization, and the app does not persist a shared project token for projects created in this mode.
206
 
207
+ Recommended trusted-team Space settings:
208
+
209
+ ```text
210
+ BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED=true
211
+ BIRDNET_AUTH_MODE=hf_token
212
+ BIRDNET_HF_PROJECT_STATE_REPOS=project-org/audio-dataset_state
213
+ HF_TOKEN=<administrator token stored as a Space secret for startup restore>
214
+ ```
215
+
216
+ After creating a project, add its generated `_state` repository to `BIRDNET_HF_PROJECT_STATE_REPOS` before relying on rebuild recovery. Each administrator and validator should open **Projects > Private project storage access** and pass the access check with their own token before validating real records; the check writes and removes only a small diagnostic marker. A private personal Bucket did not permit the invited second-user workflow in testing; the confirmed collaborative route is an organization-owned private dataset and Bucket with trusted `write` members. The private `_state` OAuth experiment also failed as a high-frequency direct validation store because `contribute-repos` permitted contribution through Pull Requests rather than direct durable writes to `main`.
217
+
218
+ Projects previously created with a private `_state` repository can be recovered from the **Admin** tab using **Connect Existing State**. Sign in with the Hugging Face account that is recorded as an administrator in that repository's `acl.json`, then provide the `_state` repo id (for example, `project-org/audio_dataset_state`). In trusted-team mode, recovery accepts only a manifest that declares a private validation Bucket and still points to a private, organization-owned collaborative project. The app loads the saved manifest, ACL, and pending invites; it does not accept a new local definition over an existing state repository.
219
 
220
  Legacy Supabase state backend, available while migrating existing projects:
221
 
 
223
  - `SUPABASE_URL` (Secret, example: `https://xxxx.supabase.co`)
224
  - `SUPABASE_SERVICE_ROLE_KEY` (Secret)
225
 
226
+ When Supabase is enabled, projects, ACL, invites, validation events, and current validation snapshots are stored in Supabase instead of `/data`. For eventual public distribution, the planned backend is administrator-owned Supabase provisioned through OAuth with an Edge Function validating Hugging Face identity; see `docs/TEMP_BYO_SUPABASE_EDGE_FUNCTION_HF_IDENTITY_PLAN.md`.
227
 
228
+ The **Private state OAuth diagnostic** action in **Projects** applies to experimental direct `_state` writes. In trusted-team mode, frequent validation state uses the Bucket rather than direct repository commits.
229
 
230
  Optional invite email settings:
231
 
 
544
  - `BIRDNET_STATE_BACKEND`
545
  - `SUPABASE_URL`
546
  - `SUPABASE_SERVICE_ROLE_KEY`
547
+ - `BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED`
548
+ - `BIRDNET_HF_PROJECT_STATE_REPOS`
549
+ - `BIRDNET_HF_PROJECT_STATE_WRITES_ENABLED`
550
+ - `BIRDNET_HF_BUCKET_VALIDATIONS_ENABLED`
551
 
552
  ### Uploader CLI Environment Variables
553
 
docs/TEMP_BYO_SUPABASE_EDGE_FUNCTION_HF_IDENTITY_PLAN.md ADDED
@@ -0,0 +1,381 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # BYO Supabase With Edge Function And Hugging Face Identity Plan
2
+
3
+ Status: deferred target architecture for public distribution
4
+ Created: 2026-05-26
5
+ Implementation target: a future branch after the trusted-team HF delivery is stable
6
+
7
+ ## Decision Summary
8
+
9
+ For a distributed public application, each project administrator should own a
10
+ Supabase backend in their own account. The BirdNET Validator app should:
11
+
12
+ 1. authenticate scientists through Hugging Face;
13
+ 2. let administrators connect a Supabase account through Supabase OAuth;
14
+ 3. provision or connect an administrator-owned Supabase project;
15
+ 4. install the validation schema and a project write API in an Edge Function;
16
+ 5. issue short-lived validation authorization tokens after checking the user's
17
+ Hugging Face identity and project role; and
18
+ 6. never distribute an administrator database secret to validators.
19
+
20
+ This model keeps relational-database performance while shifting storage quota,
21
+ state ownership, exports, and eventual billing responsibility to the project
22
+ administrator.
23
+
24
+ ## Why This Is The Distribution Target
25
+
26
+ The application must support:
27
+
28
+ 1. thousands or millions of source detections;
29
+ 2. multiple simultaneous validators;
30
+ 3. validators capable of submitting one decision every few seconds;
31
+ 4. complete audit history and reconstructable current state;
32
+ 5. fast progress dashboards and scientific exports;
33
+ 6. private pre-publication research data; and
34
+ 7. projects owned by independent administrators, not by the Space maintainer.
35
+
36
+ A Supabase database is structurally appropriate for concurrent events,
37
+ transactions, queries, and reports. Google Drive is useful for backups but
38
+ would require a custom append-only synchronization engine as the primary
39
+ backend. Hugging Face organization storage is appropriate for a small trusted
40
+ team but requires broad organization write access for validators on the
41
+ currently proven free-tier path.
42
+
43
+ ## Verified Constraints And Required Proofs
44
+
45
+ ### Confirmed Through Official Documentation
46
+
47
+ 1. Supabase exposes a Management API intended for platform integrations.
48
+ 2. Supabase integrations can use OAuth rather than requiring administrators to
49
+ paste personal access tokens.
50
+ 3. OAuth scopes include project, database, secret, and Edge Function
51
+ management permissions.
52
+ 4. Supabase Edge Functions can run trusted server code close to the
53
+ administrator-owned database.
54
+ 5. Hugging Face identity is not listed as a built-in Supabase third-party auth
55
+ provider.
56
+
57
+ ### Must Be Proven Before Product Implementation
58
+
59
+ 1. A regular free Supabase administrator can authorize our OAuth integration.
60
+ 2. The granted scopes allow creating or selecting a project without manual
61
+ administrator secrets.
62
+ 3. The integration can install schema migrations in that project, or there is
63
+ an acceptable one-time SQL fallback.
64
+ 4. The integration can deploy/update an Edge Function in a free project.
65
+ 5. A validator authenticated only with Hugging Face can submit through that
66
+ function using a short-lived app authorization token.
67
+ 6. Revocation, replay protection, simultaneous writes, exports, and restore
68
+ paths work under real use.
69
+
70
+ No production redesign begins until this spike has passed with two distinct
71
+ Hugging Face users and a Supabase account owned by the test administrator.
72
+
73
+ ## Ownership Model
74
+
75
+ ### Central Space Owns
76
+
77
+ The shared Space owns only application runtime concerns:
78
+
79
+ 1. UI and validation workflows;
80
+ 2. Hugging Face login/session handling;
81
+ 3. Supabase OAuth integration identity;
82
+ 4. the private signing key used to issue short-lived validation assertions;
83
+ 5. minimal encrypted connection registry, only if required to reconnect an
84
+ administrator's backend; and
85
+ 6. diagnostics and compatibility migrations.
86
+
87
+ ### Administrator-Owned Supabase Project Owns
88
+
89
+ One Supabase project may contain several BirdNET projects belonging to the
90
+ same administrator:
91
+
92
+ ```text
93
+ admin Supabase account
94
+ birdnet-validator-state project
95
+ birdnet_projects
96
+ project_members
97
+ project_invites
98
+ detection_catalog
99
+ validation_events
100
+ current_validations
101
+ export_jobs
102
+ audit_events
103
+ edge function: submit-validation
104
+ ```
105
+
106
+ It stores project metadata, authorization, validation history, derived current
107
+ state, and exports. It does not need to store audio files.
108
+
109
+ ### Hugging Face Dataset Owns
110
+
111
+ The administrator's Hugging Face dataset continues to own:
112
+
113
+ 1. audio segments;
114
+ 2. uploader-generated indexes;
115
+ 3. original BirdNET detection metadata; and
116
+ 4. stable identifiers required to join exports.
117
+
118
+ For private audio, validators still need either individual read permission in
119
+ Hugging Face or a separately designed read proxy. BYO Supabase solves state
120
+ ownership, not private source-audio authorization.
121
+
122
+ ## Authentication And Authorization Model
123
+
124
+ ### Administrator Setup
125
+
126
+ 1. Administrator logs into BirdNET Validator with Hugging Face.
127
+ 2. Administrator chooses **Connect Supabase Storage**.
128
+ 3. App redirects to Supabase OAuth consent.
129
+ 4. Administrator grants only required scopes.
130
+ 5. App creates or selects a Supabase project owned by that administrator.
131
+ 6. App provisions schema and Edge Function.
132
+ 7. App records a backend reference for the BirdNET project.
133
+
134
+ ### Validator Session
135
+
136
+ 1. Validator logs in with Hugging Face only.
137
+ 2. App verifies identity from the HF session/token.
138
+ 3. App reads project membership from the administrator-owned backend through a
139
+ read path or cached signed project mapping.
140
+ 4. When the validator submits a decision, the app issues a signed, short-lived
141
+ authorization assertion containing:
142
+
143
+ ```json
144
+ {
145
+ "iss": "birdnet-validator-space",
146
+ "sub": "hf:validator_username",
147
+ "project_id": "uuid",
148
+ "role": "validator",
149
+ "action": "validation:submit",
150
+ "jti": "unique-id",
151
+ "iat": 0,
152
+ "exp": 0
153
+ }
154
+ ```
155
+
156
+ 5. The Supabase Edge Function validates signature, expiry, action, project
157
+ membership, and replay rules before performing the database transaction.
158
+
159
+ ### Trust Boundary
160
+
161
+ The Space remains the identity broker: it asserts that a Hugging Face user is
162
+ currently signed in and permitted to take an action. It does not own the
163
+ scientific validation database or distribute database administrator keys.
164
+
165
+ ## Supabase OAuth And Secret Policy
166
+
167
+ Required principles:
168
+
169
+ 1. Prefer Supabase OAuth over personal access tokens.
170
+ 2. Request the minimum scopes needed for the provisioning phase.
171
+ 3. Keep OAuth refresh tokens encrypted at rest if persistent reconnection is
172
+ required.
173
+ 4. Separate provisioning authority from runtime validation authority.
174
+ 5. Do not save `service_role` or modern `secret` API keys in project rows.
175
+ 6. Never send privileged Supabase credentials to a browser or validator.
176
+ 7. Provide a **Disconnect Supabase** action that revokes local references and
177
+ explains how to revoke authorization in Supabase.
178
+
179
+ Candidate scopes must be finalized after the spike. Expected needs include:
180
+
181
+ | Capability | Expected purpose |
182
+ |---|---|
183
+ | Project read/write | Select or create admin-owned Supabase project |
184
+ | Database write | Install/update schema if available to the integration |
185
+ | Edge Function write | Deploy validation submission handler |
186
+ | Secret write | Install function verification material without exposing it |
187
+
188
+ ## Database Schema
189
+
190
+ ### `birdnet_projects`
191
+
192
+ | Column | Purpose |
193
+ |---|---|
194
+ | `project_id` | Stable UUID |
195
+ | `project_slug` | User-facing key |
196
+ | `owner_hf_username` | Project administrator identity |
197
+ | `dataset_repo_id` | HF source dataset |
198
+ | `created_at`, `updated_at` | Audit timestamps |
199
+ | `active` | Archive control |
200
+
201
+ ### `project_members`
202
+
203
+ | Column | Purpose |
204
+ |---|---|
205
+ | `project_id` | Project scope |
206
+ | `hf_username` | Verified collaborator identity |
207
+ | `role` | `admin`, `validator`, or `viewer` |
208
+ | `active` | Revocation control |
209
+ | `granted_by`, `granted_at` | Audit |
210
+
211
+ ### `validation_events`
212
+
213
+ Append-only source of truth:
214
+
215
+ | Column | Purpose |
216
+ |---|---|
217
+ | `event_id` | UUID/idempotency key |
218
+ | `project_id` | Scope |
219
+ | `detection_key` | Stable segment identifier |
220
+ | `actor_hf_username` | Validator |
221
+ | `status`, `corrected_species`, `notes` | Scientific review |
222
+ | `expected_version`, `new_version` | Concurrency control |
223
+ | `created_at` | Audit ordering |
224
+ | `auth_jti` | Replay/audit link |
225
+
226
+ ### `current_validations`
227
+
228
+ Fast derived state keyed by `(project_id, detection_key)`. It must be updated
229
+ within the same database transaction as event insertion.
230
+
231
+ ### Detection Catalog Policy
232
+
233
+ The full source detection table may be loaded from HF on demand or cached in
234
+ Supabase when necessary for reports. The original dataset index remains the
235
+ authoritative source for audio, species, confidence, start time, and end time.
236
+
237
+ ## Edge Function Write Contract
238
+
239
+ Suggested endpoint:
240
+
241
+ ```http
242
+ POST /functions/v1/submit-validation
243
+ Authorization: Bearer <short-lived app assertion>
244
+ Content-Type: application/json
245
+ Idempotency-Key: <event uuid>
246
+ ```
247
+
248
+ Request:
249
+
250
+ ```json
251
+ {
252
+ "project_id": "uuid",
253
+ "detection_key": "stable-key",
254
+ "status": "positive",
255
+ "corrected_species": null,
256
+ "notes": "",
257
+ "expected_version": 0
258
+ }
259
+ ```
260
+
261
+ Function requirements:
262
+
263
+ 1. verify signature against installed public key or shared verifier secret;
264
+ 2. reject expired or replayed assertions;
265
+ 3. verify active member role;
266
+ 4. verify request project equals asserted project;
267
+ 5. execute atomic event insert and snapshot update;
268
+ 6. return the new version or a structured conflict;
269
+ 7. append an audit record for authorization failures and accepted writes.
270
+
271
+ ## Conflict And Throughput Strategy
272
+
273
+ 1. Every decision is an immutable event.
274
+ 2. Current state updates use an atomic database transaction or SQL RPC.
275
+ 3. Two decisions for the same expected version return a conflict for review;
276
+ neither is silently discarded.
277
+ 4. The UI may batch network calls only if every event keeps an idempotency key.
278
+ 5. Dashboard queries read `current_validations`, not all historical events.
279
+
280
+ The expected workload of rapid human validation is natural for PostgreSQL and
281
+ must be tested with simulated multi-user bursts.
282
+
283
+ ## Administrator Experience
284
+
285
+ New UI surfaces:
286
+
287
+ 1. **Storage Setup**: connect Supabase, choose existing/create backend, verify
288
+ schema and function health.
289
+ 2. **Project Management**: create BirdNET project in connected backend.
290
+ 3. **Member Management**: app-level invitation and role controls persisted in
291
+ the administrator's Supabase.
292
+ 4. **Health**: backend owner, connection state, migration version, last
293
+ successful write, last export, and reconnect action.
294
+ 5. **Export And Backup**: prepare CSV/XLSX and optionally back them up to
295
+ administrator-owned Google Drive or HF resources.
296
+
297
+ ## Implementation Phases
298
+
299
+ ### Phase 0 - Feasibility Spike
300
+
301
+ 1. Register a private Supabase OAuth integration.
302
+ 2. Complete OAuth from a local test app and from an HF Space.
303
+ 3. List/create a free Supabase project through Management API.
304
+ 4. Attempt automated schema provisioning.
305
+ 5. Attempt Edge Function deployment and secret installation.
306
+ 6. Submit one validation using a separately authenticated HF validator.
307
+ 7. Revoke OAuth and confirm configuration writes stop.
308
+
309
+ Exit gate: all required operations succeed without a manually supplied
310
+ administrator database key, or the remaining one-time manual step is explicitly
311
+ accepted.
312
+
313
+ ### Phase 1 - Backend Contracts
314
+
315
+ 1. Add `supabase_owned` project backend type.
316
+ 2. Add encrypted connection-registry interface.
317
+ 3. Implement provisioning service behind a mockable client.
318
+ 4. Implement Edge Function source and schema migrations.
319
+ 5. Add connection health and migration version checks.
320
+
321
+ ### Phase 2 - Identity Broker
322
+
323
+ 1. Require verified HF login in production.
324
+ 2. Implement short-lived assertion signer and key rotation.
325
+ 3. Deploy verifier configuration to administrator Edge Functions.
326
+ 4. Add membership checks, expiration, idempotency, and replay defense.
327
+
328
+ ### Phase 3 - Product Workflows
329
+
330
+ 1. Implement **Connect Supabase Storage** UI.
331
+ 2. Create projects and membership records in admin-owned storage.
332
+ 3. Route validation, progress, and export operations through that backend.
333
+ 4. Remove shared project token persistence from the default flow.
334
+
335
+ ### Phase 4 - Reliability And Migration
336
+
337
+ 1. Import projects/events from central Supabase or trusted-team HF Buckets.
338
+ 2. Add automated export backups.
339
+ 3. Load-test simultaneous validation.
340
+ 4. Security review before enabling for third-party projects.
341
+
342
+ ## Test Matrix
343
+
344
+ | Area | Required tests |
345
+ |---|---|
346
+ | OAuth setup | consent, revoked token, expired token, denied scopes |
347
+ | Provisioning | new project, existing project, migration retry, failed setup recovery |
348
+ | Identity | valid HF actor, removed member, forged assertion, expired assertion |
349
+ | Writes | idempotent retry, same-audio conflict, parallel different audio submissions |
350
+ | Reports | ongoing project export, completed export, species aggregation |
351
+ | Privacy | no secret in UI/log/export, unauthorized user cannot query or submit |
352
+ | Resilience | Space rebuild, Edge Function redeploy, Supabase outage and recovery |
353
+
354
+ ## Acceptance Criteria
355
+
356
+ The distribution architecture is acceptable only when:
357
+
358
+ 1. project state is durably stored in the administrator's Supabase account;
359
+ 2. validators require only their Hugging Face identity during routine work;
360
+ 3. no validator receives database administrator secrets;
361
+ 4. concurrent validation writes do not silently overwrite each other;
362
+ 5. the administrator can revoke a validator and immediately stop new writes;
363
+ 6. a Space rebuild does not erase projects or validation history;
364
+ 7. scientific CSV/XLSX exports remain complete and reproducible; and
365
+ 8. the app exposes the ownership, quota, and recovery status clearly.
366
+
367
+ ## Official References
368
+
369
+ 1. Supabase Management API: <https://supabase.com/docs/reference/api/introduction>
370
+ 2. Supabase OAuth integrations:
371
+ <https://supabase.com/docs/guides/integrations/build-a-supabase-oauth-integration>
372
+ 3. Supabase for Platforms:
373
+ <https://supabase.com/docs/guides/integrations/supabase-for-platforms>
374
+ 4. Supabase Edge Functions:
375
+ <https://supabase.com/docs/guides/functions>
376
+ 5. Supabase Row Level Security:
377
+ <https://supabase.com/docs/guides/database/postgres/row-level-security>
378
+ 6. Supabase third-party auth:
379
+ <https://supabase.com/docs/guides/auth/third-party/overview>
380
+ 7. Hugging Face OAuth:
381
+ <https://huggingface.co/docs/hub/oauth>
docs/TEMP_HF_TRUSTED_TEAM_PRIVATE_STORAGE_DELIVERY_PLAN.md ADDED
@@ -0,0 +1,362 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # HF Trusted-Team Private Storage Delivery Plan
2
+
3
+ Status: immediate delivery architecture, implementation in progress on
4
+ `feature/project-state-security-privacy-review`
5
+ Created: 2026-05-26
6
+ Purpose: enable real validation by a small trusted research group while
7
+ protecting unpublished material from external access
8
+
9
+ ## Decision Summary
10
+
11
+ For the immediate research workflow, all durable project data will live in
12
+ private Hugging Face resources under a dedicated project namespace:
13
+
14
+ ```text
15
+ project organization namespace/
16
+ audio-dataset private Dataset repository
17
+ audio-dataset_state private Dataset repository for manifest/ACL/invites
18
+ audio-dataset_validation_state private Storage Bucket for validation events/snapshots
19
+ ```
20
+
21
+ The working permission model is intentionally broad:
22
+
23
+ 1. the project administrator owns or administers the namespace;
24
+ 2. each trusted validator is granted Hugging Face `write` membership in the
25
+ project organization;
26
+ 3. each user signs in using their own Hugging Face authorization;
27
+ 4. the app writes validation decisions to the private Bucket using the current
28
+ validator's token, never a shared administrator token; and
29
+ 5. users outside the organization cannot read the private dataset or state.
30
+
31
+ This is not the public distribution architecture. Trusted validators can
32
+ technically modify resources within the project organization outside the app.
33
+ That trade-off is explicitly accepted for a small trained research team.
34
+
35
+ ## Why This Route Is Appropriate Now
36
+
37
+ The project needs to begin real validation before the BYO Supabase integration
38
+ can be researched and implemented. This route has already produced the key
39
+ permission proof:
40
+
41
+ | Actor | Organization role | Read private Bucket | Write private Bucket |
42
+ |---|---|---:|---:|
43
+ | Administrator | `admin` | passed | passed |
44
+ | Trusted validator | `write` | passed | passed |
45
+ | Contributor | `contributor` | passed | failed (`403`) |
46
+
47
+ Therefore, the free and currently demonstrated HF-only workflow requires
48
+ trusted validators to be `write` members of an isolated project organization.
49
+
50
+ ## Current Branch Implementation Status
51
+
52
+ Implemented in `feature/project-state-security-privacy-review`:
53
+
54
+ 1. explicit `BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED` runtime switch, disabled by
55
+ default for existing deployments;
56
+ 2. automatic creation of a private validation Bucket during new project setup
57
+ in trusted-team mode;
58
+ 3. automatic private `_state` repository creation with the Bucket reference in
59
+ the project manifest;
60
+ 4. rejection of public source datasets, personal namespaces, and app-private
61
+ visibility in trusted-team mode;
62
+ 5. no persisted shared project token for newly created Bucket-backed projects;
63
+ 6. validation routing through the signed-in actor's HF token with no
64
+ administrator-token fallback;
65
+ 7. HF-mode health status and an Admin warning explaining organization `write`
66
+ membership; and
67
+ 8. a private Bucket access probe that reads, writes, verifies, and removes a
68
+ diagnostic marker using the signed-in collaborator's own token; and
69
+ 9. unit tests for configuration, private-dataset enforcement, organization
70
+ enforcement, token removal, Bucket access/routing, conflict/reconciliation,
71
+ state recovery, and UI creation.
72
+
73
+ Recovery through **Connect Existing State** also refuses manifests that do not
74
+ declare Bucket storage or no longer identify a compatible private
75
+ organization-owned collaborative project.
76
+
77
+ Remaining before merge with `main`:
78
+
79
+ 1. push/deploy this branch to the Space;
80
+ 2. configure the runtime variables/secrets;
81
+ 3. run the two-account private organization test described in the merge gate;
82
+ 4. confirm rebuild recovery through `BIRDNET_HF_PROJECT_STATE_REPOS`; and
83
+ 5. confirm exports after real Bucket-backed validations.
84
+
85
+ Automated verification completed on 2026-05-26:
86
+
87
+ 1. `python -m pytest -q`: `246 passed`;
88
+ 2. trusted-team UI construction with
89
+ `BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED=true`: passed; and
90
+ 3. default UI construction without the new mode: passed; and
91
+ 4. `git diff --check`: no patch whitespace errors.
92
+
93
+ ## Security Boundary
94
+
95
+ ### Protected Against
96
+
97
+ 1. external users not granted organization membership;
98
+ 2. accidental loss caused by Space rebuilds or lack of persistent disk;
99
+ 3. disclosure of a shared administrator dataset token through the app;
100
+ 4. high-frequency Git commit limits for individual validation actions, because
101
+ validation events are written to a Bucket rather than a dataset repository.
102
+
103
+ ### Not Protected Against
104
+
105
+ 1. intentional or accidental modification by a trusted organization member
106
+ with `write` permission;
107
+ 2. a compromised member Hugging Face token;
108
+ 3. project-to-project privilege spillover if multiple confidential projects
109
+ are placed in the same organization.
110
+
111
+ Mitigation: create a separate HF organization for each confidential project or
112
+ for each group of projects whose validators may safely share broad access.
113
+
114
+ ## Resource Responsibilities
115
+
116
+ | Resource | Purpose | Write pattern | Authority |
117
+ |---|---|---|---|
118
+ | Private audio dataset | Audio and original detection index | Uploader only before/during ingest | Project admin/uploader |
119
+ | Private `_state` dataset | `project.json`, `acl.json`, `invites.json` | Infrequent admin operations | App admin flow |
120
+ | Private validation Bucket | events, snapshots, archives | Frequent validator writes | Current signed-in HF user |
121
+ | Space runtime/local files | Cache only | Ephemeral | Never authoritative |
122
+
123
+ The audio dataset must be private before a real project is created in this
124
+ mode. The source dataset should be treated as immutable once active validation
125
+ begins, other than controlled uploader completion.
126
+
127
+ ## State Layout
128
+
129
+ ### Private `_state` Repository
130
+
131
+ ```text
132
+ project.json
133
+ acl.json
134
+ invites.json
135
+ README.md
136
+ ```
137
+
138
+ The repository contains project configuration and app-level access metadata.
139
+ It must not contain HF tokens.
140
+
141
+ ### Private Validation Bucket
142
+
143
+ ```text
144
+ metadata/project.json
145
+ snapshots/current.json
146
+ events/YYYYMMDD/<event-id>.json
147
+ archives/events/<archive-id>.jsonl
148
+ ```
149
+
150
+ Rules:
151
+
152
+ 1. individual decisions are append-oriented events;
153
+ 2. snapshots accelerate queues and reports but are reconstructable;
154
+ 3. compaction archives active events instead of discarding audit history;
155
+ 4. parallel same-version decisions are surfaced as conflicts; and
156
+ 5. exports join validation state with the source detection metadata.
157
+
158
+ ## Application Configuration
159
+
160
+ The immediate mode is activated explicitly:
161
+
162
+ ```text
163
+ BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED=true
164
+ BIRDNET_AUTH_MODE=hf_token
165
+ BIRDNET_HF_PROJECT_STATE_REPOS=project-org/audio-dataset_state
166
+ HF_TOKEN=<admin read/write token for startup recovery>
167
+ ```
168
+
169
+ Notes:
170
+
171
+ 1. `BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED=true` implies HF `_state` syncing and
172
+ HF Bucket validation routing for projects created in the mode.
173
+ 2. `BIRDNET_AUTH_MODE=hf_token` permits each team member to use their own
174
+ verified token. OAuth-only Bucket capability has not been proven.
175
+ 3. `HF_TOKEN` allows the Space to restore project configuration after a
176
+ restart before any user has logged in. It belongs to the administrator of
177
+ this specific deployment and must be stored only as a Space secret.
178
+ 4. `BIRDNET_HF_PROJECT_STATE_REPOS` is updated after each project is created,
179
+ so the Space can reload it after rebuilds.
180
+ 5. Project tokens stored in application state are disabled for new
181
+ trusted-team Bucket projects. Users supply their own authorization.
182
+ 6. The hosted OAuth button requests read access only in this interim mode;
183
+ it does not replace the personal token required for the proven Bucket
184
+ write workflow.
185
+
186
+ ## Administrator Workflow
187
+
188
+ ### Prerequisites
189
+
190
+ 1. Create a dedicated Hugging Face organization for the confidential project.
191
+ 2. Ensure the audio dataset exists in that namespace and is private.
192
+ 3. Ensure the administrator token can read/write resources in the
193
+ organization.
194
+ 4. Configure the Space secrets/variables above.
195
+
196
+ ### Create Project In The App
197
+
198
+ 1. Log into the app with the administrator's own HF token.
199
+ 2. Open **Admin > Project management**.
200
+ 3. Enter:
201
+ - project slug;
202
+ - display name;
203
+ - private dataset repo id, for example `project-org/audio-dataset`;
204
+ - app visibility `collaborative` for a team project.
205
+ 4. Select **Create Project**.
206
+ 5. The app verifies that the source dataset is private, organization-owned,
207
+ and uses app visibility `collaborative`.
208
+ 6. The app creates:
209
+ - `project-org/audio-dataset_validation_state` as a private Bucket;
210
+ - `project-org/audio-dataset_state` as a private state repository.
211
+ 7. Copy the new `_state` repo id into the Space variable
212
+ `BIRDNET_HF_PROJECT_STATE_REPOS`, comma-separated with existing projects.
213
+ 8. Restart/rebuild the Space and confirm the project is restored.
214
+
215
+ ### Add Trusted Validators
216
+
217
+ 1. Add the validator to the dedicated project organization in Hugging Face
218
+ with `write` role.
219
+ 2. Assign or invite the same HF username in the app to retain workflow roles
220
+ and UI filtering.
221
+ 3. Validator logs in using their own HF token.
222
+ 4. Validator opens **Projects > Private project storage access** and runs the
223
+ check, which writes and removes a small diagnostic marker without changing
224
+ validation records.
225
+ 5. Validator verifies that the private audio opens and submits a test
226
+ validation.
227
+
228
+ Both membership layers are intentionally required in this interim model:
229
+ Hugging Face controls actual private resource access, while app roles control
230
+ the validation interface.
231
+
232
+ ## Implementation Work Packages
233
+
234
+ ### Package 1 - Explicit Runtime Mode
235
+
236
+ Implementation:
237
+
238
+ 1. Add `BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED`.
239
+ 2. Make the mode enable project-state synchronization and Bucket validation
240
+ routing without changing existing deployments by default.
241
+ 3. Display active private HF storage status in Settings and app header.
242
+
243
+ Tests:
244
+
245
+ 1. environment flag parsing;
246
+ 2. default disabled behavior;
247
+ 3. UI construction smoke test.
248
+
249
+ ### Package 2 - Secure Project Provisioning
250
+
251
+ Implementation:
252
+
253
+ 1. During project creation, require an organization-owned private source
254
+ dataset and app visibility `collaborative`.
255
+ 2. Initialize the private Bucket in the dataset namespace.
256
+ 3. Store Bucket reference in the `_state` manifest.
257
+ 4. Initialize the private `_state` repository.
258
+ 5. Do not persist setup/project tokens for Bucket projects.
259
+ 6. Refuse automatic overwrite if either store contains unrelated content.
260
+
261
+ Tests:
262
+
263
+ 1. private dataset accepted and public dataset rejected;
264
+ 2. Bucket initialization and manifest linkage;
265
+ 3. existing safe state reuse and unsafe content refusal;
266
+ 4. setup token does not appear in serialized project state.
267
+
268
+ ### Package 3 - Trusted Validator Writes
269
+
270
+ Implementation:
271
+
272
+ 1. Route validation reads/writes to the Bucket for projects declaring it.
273
+ 2. Use the current signed-in user's HF token for writes.
274
+ 3. Never fall back to the administrator token for a validator write.
275
+ 4. Give clear access errors when organization permission is missing.
276
+
277
+ Tests:
278
+
279
+ 1. actor token routing;
280
+ 2. no fallback to admin token;
281
+ 3. disabled-backend hard failure;
282
+ 4. two-user Space proof with private audio and Bucket.
283
+
284
+ ### Package 4 - Recovery And Audit
285
+
286
+ Implementation:
287
+
288
+ 1. Reload project metadata from configured `_state` repositories on startup.
289
+ 2. Keep events append-oriented and snapshots reconstructable.
290
+ 3. Preserve conflict reporting and archive compaction.
291
+ 4. Expose CSV/XLSX exports from reconciled current state.
292
+
293
+ Tests:
294
+
295
+ 1. rebuild Space and restore project/ACL;
296
+ 2. restore snapshot from events;
297
+ 3. simultaneous different-audio validation;
298
+ 4. same-audio conflict;
299
+ 5. export before and after restart.
300
+
301
+ ### Package 5 - Operational Guardrails
302
+
303
+ Implementation:
304
+
305
+ 1. Add visible wording that trusted-team mode requires HF organization
306
+ membership separately from app invitations.
307
+ 2. Provide a collaborator-run Bucket read/write diagnostic that removes its
308
+ own marker and does not create a validation decision.
309
+ 3. Warn when no `_state` repository is configured for restart recovery.
310
+ 4. Document backup/export cadence.
311
+ 5. Document token revocation after team changes.
312
+ 6. Record that organization isolation is the privacy boundary.
313
+
314
+ ## Merge Gate For `main`
315
+
316
+ The branch may be merged only after:
317
+
318
+ 1. the existing automated test suite passes;
319
+ 2. a private test dataset in a dedicated organization is loaded in the Space;
320
+ 3. the administrator creates a project and sees both private state resources;
321
+ 4. the Space is rebuilt and restores the project from `_state`;
322
+ 5. administrator and second trusted account pass **Private project storage
323
+ access** using their own tokens;
324
+ 6. a second trusted account with organization `write` role loads private
325
+ audio and writes a validation using its own token;
326
+ 7. an account outside the organization cannot load audio or validation state;
327
+ 8. twenty rapid validations persist after refresh/rebuild;
328
+ 9. a CSV and XLSX export contain the expected validation decisions; and
329
+ 10. the administrator keeps a source backup of audio/index data.
330
+
331
+ ## Known Limitations
332
+
333
+ 1. Creating organizations and managing membership remain Hugging Face web UI
334
+ steps in the free supported workflow investigated to date.
335
+ 2. A trusted validator receives broad write permission in that project's
336
+ organization.
337
+ 3. The startup recovery secret is acceptable for the administrator's immediate
338
+ private deployment, but is not the distributed public-product model.
339
+ 4. OAuth-only access to private Buckets remains unproven; routine immediate
340
+ testing uses each member's own verified HF token.
341
+
342
+ ## Transition To Distribution Architecture
343
+
344
+ When BYO Supabase with Edge Function support is proven:
345
+
346
+ 1. retain the private HF audio dataset as source storage;
347
+ 2. export/replay Bucket validation events into the administrator-owned
348
+ Supabase database;
349
+ 3. replace organization-wide write membership with app-issued validation
350
+ authorization; and
351
+ 4. archive the HF Bucket as a historical backup or retain it read-only.
352
+
353
+ ## Official References
354
+
355
+ 1. Hugging Face Storage Buckets:
356
+ <https://huggingface.co/docs/hub/storage-buckets>
357
+ 2. Hugging Face dataset repositories:
358
+ <https://huggingface.co/docs/hub/datasets-overview>
359
+ 3. Hugging Face organizations:
360
+ <https://huggingface.co/docs/hub/organizations>
361
+ 4. Hugging Face OAuth:
362
+ <https://huggingface.co/docs/hub/oauth>
src/config/runtime_config.py CHANGED
@@ -33,6 +33,7 @@ class RuntimeConfig:
33
  auth_mode: str = "auto"
34
  hf_project_state_writes_enabled: bool = False
35
  hf_bucket_validations_enabled: bool = False
 
36
  hf_project_state_repos: tuple[str, ...] = ()
37
 
38
  @classmethod
@@ -111,6 +112,8 @@ class RuntimeConfig:
111
  hf_project_state_writes_enabled = raw_hf_project_state_writes in {"1", "true", "yes", "on"}
112
  raw_hf_bucket_validations = (os.getenv("BIRDNET_HF_BUCKET_VALIDATIONS_ENABLED") or "").strip().lower()
113
  hf_bucket_validations_enabled = raw_hf_bucket_validations in {"1", "true", "yes", "on"}
 
 
114
  raw_hf_project_state_repos = (os.getenv("BIRDNET_HF_PROJECT_STATE_REPOS") or "").strip()
115
  hf_project_state_repos = tuple(
116
  repo.strip()
@@ -146,6 +149,7 @@ class RuntimeConfig:
146
  auth_mode=auth_mode,
147
  hf_project_state_writes_enabled=hf_project_state_writes_enabled,
148
  hf_bucket_validations_enabled=hf_bucket_validations_enabled,
 
149
  hf_project_state_repos=hf_project_state_repos,
150
  )
151
 
 
33
  auth_mode: str = "auto"
34
  hf_project_state_writes_enabled: bool = False
35
  hf_bucket_validations_enabled: bool = False
36
+ hf_trusted_team_mode_enabled: bool = False
37
  hf_project_state_repos: tuple[str, ...] = ()
38
 
39
  @classmethod
 
112
  hf_project_state_writes_enabled = raw_hf_project_state_writes in {"1", "true", "yes", "on"}
113
  raw_hf_bucket_validations = (os.getenv("BIRDNET_HF_BUCKET_VALIDATIONS_ENABLED") or "").strip().lower()
114
  hf_bucket_validations_enabled = raw_hf_bucket_validations in {"1", "true", "yes", "on"}
115
+ raw_hf_trusted_team_mode = (os.getenv("BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED") or "").strip().lower()
116
+ hf_trusted_team_mode_enabled = raw_hf_trusted_team_mode in {"1", "true", "yes", "on"}
117
  raw_hf_project_state_repos = (os.getenv("BIRDNET_HF_PROJECT_STATE_REPOS") or "").strip()
118
  hf_project_state_repos = tuple(
119
  repo.strip()
 
149
  auth_mode=auth_mode,
150
  hf_project_state_writes_enabled=hf_project_state_writes_enabled,
151
  hf_bucket_validations_enabled=hf_bucket_validations_enabled,
152
+ hf_trusted_team_mode_enabled=hf_trusted_team_mode_enabled,
153
  hf_project_state_repos=hf_project_state_repos,
154
  )
155
 
src/repositories/hf_bucket_validation_repository.py CHANGED
@@ -123,6 +123,14 @@ class HfBucketCompactionResult:
123
  compacted_event_count: int
124
 
125
 
 
 
 
 
 
 
 
 
126
  class HfBucketValidationInitializer:
127
  """Create a private mutable validation store owned by the project admin."""
128
 
@@ -173,6 +181,88 @@ class HfBucketValidationInitializer:
173
  return HfBucketValidationInitResult(bucket_id=bucket_id, initialized=True, reused_existing=False)
174
 
175
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
176
  def _json_bytes(payload: object) -> bytes:
177
  return json.dumps(payload, ensure_ascii=True, indent=2, sort_keys=True).encode("utf-8")
178
 
 
123
  compacted_event_count: int
124
 
125
 
126
+ @dataclass(frozen=True)
127
+ class HfBucketPermissionProbeResult:
128
+ bucket_id: str
129
+ actor_username: str
130
+ diagnostic_path: str
131
+ verified_at: str
132
+
133
+
134
  class HfBucketValidationInitializer:
135
  """Create a private mutable validation store owned by the project admin."""
136
 
 
181
  return HfBucketValidationInitResult(bucket_id=bucket_id, initialized=True, reused_existing=False)
182
 
183
 
184
+ class HfBucketPermissionProbe:
185
+ """Verify a user's private Bucket access without changing validation state."""
186
+
187
+ def __init__(self, api: HfBucketFilesApi | None = None) -> None:
188
+ self._api = api or HuggingFaceBucketFilesApi()
189
+
190
+ def probe(
191
+ self,
192
+ *,
193
+ bucket_id: str,
194
+ actor_username: str,
195
+ token: str | None,
196
+ ) -> HfBucketPermissionProbeResult:
197
+ bucket = (bucket_id or "").strip()
198
+ actor = (actor_username or "").strip()
199
+ token_value = (token or "").strip()
200
+ if not bucket or not actor or not token_value:
201
+ raise HfBucketValidationError(
202
+ "Select a project and sign in with your own Hugging Face authorization before testing private storage access."
203
+ )
204
+
205
+ verified_at = datetime.now(UTC).isoformat()
206
+ diagnostic_path = f"diagnostics/access-proof/{uuid4()}.json"
207
+ payload = {
208
+ "schema_version": HF_BUCKET_VALIDATION_SCHEMA_VERSION,
209
+ "diagnostic_type": "private_bucket_access_proof",
210
+ "actor_username": actor,
211
+ "bucket_id": bucket,
212
+ "verified_at": verified_at,
213
+ "contains_project_data": False,
214
+ }
215
+ marker_written = False
216
+ try:
217
+ self._api.read_text(
218
+ bucket_id=bucket,
219
+ path_in_bucket="metadata/project.json",
220
+ token=token_value,
221
+ )
222
+ self._api.write_files(
223
+ bucket_id=bucket,
224
+ token=token_value,
225
+ files={diagnostic_path: _json_bytes(payload)},
226
+ )
227
+ marker_written = True
228
+ written_payload = json.loads(
229
+ self._api.read_text(
230
+ bucket_id=bucket,
231
+ path_in_bucket=diagnostic_path,
232
+ token=token_value,
233
+ )
234
+ )
235
+ if written_payload.get("actor_username") != actor:
236
+ raise HfBucketValidationError("The private storage diagnostic marker could not be verified.")
237
+ except HfBucketValidationError:
238
+ raise
239
+ except Exception as exc:
240
+ raise HfBucketValidationError(
241
+ f"Private Bucket read/write test failed for '{bucket}' using the signed-in Hugging Face account: {exc}"
242
+ ) from exc
243
+ finally:
244
+ if marker_written:
245
+ try:
246
+ self._api.write_files(
247
+ bucket_id=bucket,
248
+ token=token_value,
249
+ files={},
250
+ delete_paths=[diagnostic_path],
251
+ )
252
+ except Exception as exc:
253
+ raise HfBucketValidationError(
254
+ "Private Bucket access was verified, but the diagnostic marker could not be removed: "
255
+ f"{exc}"
256
+ ) from exc
257
+
258
+ return HfBucketPermissionProbeResult(
259
+ bucket_id=bucket,
260
+ actor_username=actor,
261
+ diagnostic_path=diagnostic_path,
262
+ verified_at=verified_at,
263
+ )
264
+
265
+
266
  def _json_bytes(payload: object) -> bytes:
267
  return json.dumps(payload, ensure_ascii=True, indent=2, sort_keys=True).encode("utf-8")
268
 
src/ui/app_factory.py CHANGED
@@ -26,7 +26,9 @@ from src.repositories.append_only_validation_repository import AppendOnlyValidat
26
  from src.repositories.in_memory_detection_repository import InMemoryDetectionRepository
27
  from src.repositories.hf_bucket_validation_repository import (
28
  HF_BUCKET_VALIDATION_BACKEND,
 
29
  HfBucketValidationError,
 
30
  )
31
  from src.repositories.project_aware_validation_repository import ProjectAwareValidationRepository
32
  from src.repositories.state_safety import (
@@ -1559,6 +1561,67 @@ def _is_running_in_hf_space() -> bool:
1559
  return bool((os.getenv("SPACE_ID") or os.getenv("SPACE_HOST") or "").strip())
1560
 
1561
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1562
  def _resolve_username_login_policy(runtime_config: RuntimeConfig) -> tuple[bool, str, str]:
1563
  mode = (runtime_config.auth_mode or "auto").strip().lower()
1564
  if mode == "username":
@@ -2904,6 +2967,8 @@ def create_app() -> gr.Blocks:
2904
 
2905
  projects_file_path, user_access_file_path, invites_file_path = _resolve_bootstrap_file_paths(runtime_config)
2906
  state_store, supabase_validation_repository, state_backend_message = _build_supabase_state(runtime_config)
 
 
2907
  allow_username_login, auth_mode_label, auth_mode_description = _resolve_username_login_policy(runtime_config)
2908
  bootstrap_warning = _bootstrap_auth_and_projects(
2909
  auth_service,
@@ -2938,6 +3003,8 @@ def create_app() -> gr.Blocks:
2938
  max_loaded_projects = 3
2939
  audio_service = AudioFetchService(EphemeralCacheManager(ttl_seconds=300, max_files=128))
2940
  base_validation_repository = supabase_validation_repository or AppendOnlyValidationRepository(base_dir=runtime_config.validation_base_dir)
 
 
2941
  validation_repository = ProjectAwareValidationRepository(
2942
  fallback_repository=base_validation_repository,
2943
  project_lookup=admin_manager.get_project,
@@ -2948,11 +3015,13 @@ def create_app() -> gr.Blocks:
2948
  or None
2949
  ),
2950
  actor_token_provider=lambda _project, username: auth_service.get_hf_token_for_user(username),
2951
- enable_hf_project_state=runtime_config.hf_project_state_writes_enabled,
2952
- enable_hf_bucket_validations=runtime_config.hf_bucket_validations_enabled,
2953
  )
2954
  validation_service = ValidationService(validation_repository)
2955
  hf_state_initializer = HfProjectStateStoreInitializer()
 
 
2956
  hf_state_connector = HfProjectStateStoreConnector()
2957
  hf_state_permission_probe = HfProjectStatePermissionProbe()
2958
  hf_state_sync = HfProjectStateStoreSync()
@@ -3124,7 +3193,7 @@ def create_app() -> gr.Blocks:
3124
  sync_project_slugs: set[str] | None = None,
3125
  archived_projects: list[Project] | None = None,
3126
  ) -> list[str]:
3127
- if not runtime_config.hf_project_state_writes_enabled:
3128
  return []
3129
 
3130
  user_access = auth_service.export_user_access_map(include_inactive=True)
@@ -3214,14 +3283,24 @@ def create_app() -> gr.Blocks:
3214
  section_header_html(
3215
  "Secure access",
3216
  "Sign in with your Hugging Face identity",
3217
- "Your account controls project access, validator attribution, and private dataset access.",
 
 
 
 
3218
  )
3219
  )
3220
  username_input, session_output, login_button, error_message = create_login_page(
3221
  auth_service,
3222
  allow_username_login=allow_username_login,
3223
  enable_oauth_login=_is_running_in_hf_space(),
3224
- auth_mode_label=auth_mode_description,
 
 
 
 
 
 
3225
  )
3226
 
3227
  # Store session ID when login succeeds
@@ -3300,6 +3379,16 @@ def create_app() -> gr.Blocks:
3300
  class_name="bn-panel-soft",
3301
  )
3302
  )
 
 
 
 
 
 
 
 
 
 
3303
  with gr.Row():
3304
  create_project_slug = gr.Textbox(
3305
  label="New Project Slug",
@@ -3319,7 +3408,11 @@ def create_app() -> gr.Blocks:
3319
  value="collaborative",
3320
  )
3321
  create_project_token = gr.Textbox(
3322
- label="Project HF Token (optional)",
 
 
 
 
3323
  placeholder="hf_xxx...",
3324
  type="password",
3325
  )
@@ -3373,7 +3466,8 @@ def create_app() -> gr.Blocks:
3373
  return "Visibility must be private or collaborative.", gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), session
3374
  if (
3375
  _is_running_in_hf_space()
3376
- and runtime_config.hf_project_state_writes_enabled
 
3377
  and getattr(session, "authentication_method", "") != "oauth"
3378
  ):
3379
  return (
@@ -3417,9 +3511,33 @@ def create_app() -> gr.Blocks:
3417
  dataset_repo_id=repo_id,
3418
  visibility=visibility_value,
3419
  owner_username=session.username,
3420
- dataset_token=explicit_project_token,
3421
  active=True,
3422
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3423
  try:
3424
  state_result = hf_state_initializer.initialize(
3425
  project=project,
@@ -3466,6 +3584,17 @@ def create_app() -> gr.Blocks:
3466
  project.state_schema_version = HF_PROJECT_STATE_SCHEMA_VERSION
3467
  project.state_status = "ready"
3468
  state_repo_action = "initialized" if state_result.initialized else "connected"
 
 
 
 
 
 
 
 
 
 
 
3469
 
3470
  created = admin_manager.register_project(project)
3471
  if not created:
@@ -3499,9 +3628,9 @@ def create_app() -> gr.Blocks:
3499
 
3500
  return (
3501
  (
3502
- f"Project '{slug}' created successfully. Private state repo {state_repo_action}: {state_result.state_repo_id}."
3503
  if persisted
3504
- else f"Project '{slug}' created with private state repo {state_repo_action} ({state_result.state_repo_id}). Could not persist bootstrap files: {persist_error}"
3505
  ),
3506
  _project_rows(),
3507
  gr.update(choices=admin_projects, value=slug),
@@ -3560,6 +3689,39 @@ def create_app() -> gr.Blocks:
3560
  session,
3561
  )
3562
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3563
  slug = loaded.project.project_slug
3564
  if admin_manager.get_project(slug) is not None:
3565
  return (
@@ -3703,6 +3865,18 @@ def create_app() -> gr.Blocks:
3703
  project = admin_manager.get_project(project_slug)
3704
  if project is None:
3705
  return "Select a valid project.", gr.update(), gr.update(), gr.update(), gr.update()
 
 
 
 
 
 
 
 
 
 
 
 
3706
 
3707
  if bool(clear_token):
3708
  project.dataset_token = None
@@ -4118,7 +4292,7 @@ def create_app() -> gr.Blocks:
4118
 
4119
  session_state.change(
4120
  fn=lambda s: (
4121
- gr.update(visible=bool(s is not None)),
4122
  gr.update(visible=bool(s is not None)),
4123
  gr.update(visible=bool(s is not None)),
4124
  gr.update(visible=bool(s is not None)),
@@ -4172,7 +4346,10 @@ def create_app() -> gr.Blocks:
4172
  interactive=False,
4173
  allow_custom_value=True,
4174
  )
4175
- with gr.Group(elem_classes=["bn-panel-soft"]):
 
 
 
4176
  gr.Markdown("### Private state OAuth diagnostic")
4177
  state_authorization_status = gr.Markdown(
4178
  "The current OAuth scope may allow Pull Requests rather than direct state writes. "
@@ -4182,6 +4359,19 @@ def create_app() -> gr.Blocks:
4182
  "Check OAuth state write capability",
4183
  elem_classes=["bn-soft-action"],
4184
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
4185
  invitations_info = gr.Markdown(value="")
4186
  invitations_overview = gr.HTML(value=invite_panel_html(0))
4187
  invite_selector = gr.Dropdown(choices=[], label="Pending Invites", interactive=False)
@@ -4435,6 +4625,28 @@ def create_app() -> gr.Blocks:
4435
  f"A private diagnostic write was confirmed in `{result.state_repo_id}`."
4436
  )
4437
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4438
  project_selector.change(
4439
  fn=update_selected_project,
4440
  inputs=[project_selector, session_state],
@@ -4446,6 +4658,11 @@ def create_app() -> gr.Blocks:
4446
  inputs=[project_selector, session_state],
4447
  outputs=[state_authorization_status],
4448
  )
 
 
 
 
 
4449
 
4450
  create_project_event.then(
4451
  fn=update_project_selector,
@@ -5793,11 +6010,18 @@ def create_app() -> gr.Blocks:
5793
  def _render_settings_health():
5794
  backend = (runtime_config.state_backend or "filesystem").strip().lower()
5795
  supabase_ready = bool(runtime_config.supabase_url and runtime_config.supabase_service_role_key)
5796
- state_label = "Supabase" if backend in {"supabase", "postgres", "postgresql"} and supabase_ready else "Filesystem"
5797
- state_tone = "ok" if state_label == "Supabase" else "warn"
 
 
 
 
5798
  normalized_bootstrap_dir = str(runtime_config.bootstrap_base_dir).replace("\\", "/").rstrip("/")
5799
  filesystem_durable = normalized_bootstrap_dir == "/data" or normalized_bootstrap_dir.startswith("/data/")
5800
- if state_label == "Supabase":
 
 
 
5801
  durability_label = "external durable"
5802
  durability_tone = "ok"
5803
  elif filesystem_durable:
@@ -5809,16 +6033,24 @@ def create_app() -> gr.Blocks:
5809
  demo_label = "enabled" if runtime_config.enable_demo_bootstrap else "disabled"
5810
  demo_tone = "warn" if runtime_config.enable_demo_bootstrap else "ok"
5811
  invite_email_label = "enabled" if runtime_config.invite_email_enabled and runtime_config.emailjs_enabled else "disabled"
5812
- hf_project_state_label = "enabled" if runtime_config.hf_project_state_writes_enabled else "disabled"
5813
- hf_bucket_validation_label = "legacy experimental" if runtime_config.hf_bucket_validations_enabled else "disabled"
 
 
 
 
5814
  hf_project_state_repos_label = str(len(runtime_config.hf_project_state_repos))
5815
  hf_space_label = os.getenv("SPACE_ID") or "local runtime"
5816
  health_html = settings_health_html(
5817
  [
5818
  ("State backend", state_label, state_tone),
5819
- ("HF project-state writes", hf_project_state_label, "ok" if runtime_config.hf_project_state_writes_enabled else "info"),
5820
- ("HF Bucket validations", hf_bucket_validation_label, "warn" if runtime_config.hf_bucket_validations_enabled else "info"),
5821
- ("HF project-state repos", hf_project_state_repos_label, "ok" if runtime_config.hf_project_state_repos else "info"),
 
 
 
 
5822
  ("Auth mode", auth_mode_label, "ok" if not allow_username_login else "warn"),
5823
  ("Destructive write guard", "enabled", "ok"),
5824
  ("Local JSON backups", "enabled" if state_label == "Filesystem" else "not used", "ok" if state_label == "Filesystem" else "info"),
@@ -5833,8 +6065,18 @@ def create_app() -> gr.Blocks:
5833
  ]
5834
  )
5835
  status_text = (
5836
- "Supabase persistence is active. Project and ACL removals are guarded against accidental broad overwrites."
5837
- if state_label == "Supabase"
 
 
 
 
 
 
 
 
 
 
5838
  else (
5839
  "Filesystem persistence is active with local JSON backups and destructive-write guards. "
5840
  "On free Spaces without persistent storage, local files are still not durable across rebuilds."
 
26
  from src.repositories.in_memory_detection_repository import InMemoryDetectionRepository
27
  from src.repositories.hf_bucket_validation_repository import (
28
  HF_BUCKET_VALIDATION_BACKEND,
29
+ HfBucketPermissionProbe,
30
  HfBucketValidationError,
31
+ HfBucketValidationInitializer,
32
  )
33
  from src.repositories.project_aware_validation_repository import ProjectAwareValidationRepository
34
  from src.repositories.state_safety import (
 
1561
  return bool((os.getenv("SPACE_ID") or os.getenv("SPACE_HOST") or "").strip())
1562
 
1563
 
1564
+ def _validate_hf_trusted_team_source_dataset(
1565
+ *,
1566
+ project: Project,
1567
+ token: str | None,
1568
+ api: HfApi | None = None,
1569
+ ) -> None:
1570
+ if (project.visibility or "").strip().lower() != "collaborative":
1571
+ raise HfBucketValidationError(
1572
+ "Trusted-team projects must use app visibility 'collaborative'. "
1573
+ "The Hugging Face dataset and storage remain private; collaborative visibility enables approved "
1574
+ "team members inside this app."
1575
+ )
1576
+ token_value = (token or "").strip()
1577
+ if not token_value:
1578
+ raise HfBucketValidationError("Trusted-team HF storage requires the administrator's Hugging Face authorization during project setup.")
1579
+
1580
+ client = api or HfApi()
1581
+ namespace = project.dataset_repo_id.split("/", 1)[0].strip()
1582
+ try:
1583
+ client.get_organization_overview(namespace, token=token_value)
1584
+ except Exception as exc:
1585
+ raise HfBucketValidationError(
1586
+ "Trusted-team HF storage requires the private dataset to be owned by a dedicated Hugging Face "
1587
+ f"organization. Could not verify organization '{namespace}': {exc}"
1588
+ ) from exc
1589
+ try:
1590
+ dataset_info = client.repo_info(
1591
+ repo_id=project.dataset_repo_id,
1592
+ repo_type="dataset",
1593
+ token=token_value,
1594
+ )
1595
+ except Exception as exc:
1596
+ raise HfBucketValidationError(f"Could not verify the Hugging Face dataset before creating private state: {exc}") from exc
1597
+ if not bool(getattr(dataset_info, "private", False)):
1598
+ raise HfBucketValidationError(
1599
+ "Trusted-team HF storage only accepts a private dataset. Make the dataset private in Hugging Face "
1600
+ "before creating the validation project."
1601
+ )
1602
+
1603
+
1604
+ def _initialize_hf_trusted_team_storage(
1605
+ *,
1606
+ project: Project,
1607
+ token: str | None,
1608
+ api: HfApi | None = None,
1609
+ bucket_initializer: HfBucketValidationInitializer | None = None,
1610
+ ):
1611
+ _validate_hf_trusted_team_source_dataset(project=project, token=token, api=api)
1612
+ token_value = (token or "").strip()
1613
+ initializer = bucket_initializer or HfBucketValidationInitializer()
1614
+ result = initializer.initialize(
1615
+ project_slug=project.project_slug,
1616
+ dataset_repo_id=project.dataset_repo_id,
1617
+ token=token_value,
1618
+ )
1619
+ project.dataset_token = None
1620
+ project.validation_backend = HF_BUCKET_VALIDATION_BACKEND
1621
+ project.validation_bucket_id = result.bucket_id
1622
+ return result
1623
+
1624
+
1625
  def _resolve_username_login_policy(runtime_config: RuntimeConfig) -> tuple[bool, str, str]:
1626
  mode = (runtime_config.auth_mode or "auto").strip().lower()
1627
  if mode == "username":
 
2967
 
2968
  projects_file_path, user_access_file_path, invites_file_path = _resolve_bootstrap_file_paths(runtime_config)
2969
  state_store, supabase_validation_repository, state_backend_message = _build_supabase_state(runtime_config)
2970
+ if runtime_config.hf_trusted_team_mode_enabled:
2971
+ state_backend_message = "HF trusted-team project storage enabled."
2972
  allow_username_login, auth_mode_label, auth_mode_description = _resolve_username_login_policy(runtime_config)
2973
  bootstrap_warning = _bootstrap_auth_and_projects(
2974
  auth_service,
 
3003
  max_loaded_projects = 3
3004
  audio_service = AudioFetchService(EphemeralCacheManager(ttl_seconds=300, max_files=128))
3005
  base_validation_repository = supabase_validation_repository or AppendOnlyValidationRepository(base_dir=runtime_config.validation_base_dir)
3006
+ hf_state_writes_enabled = runtime_config.hf_project_state_writes_enabled or runtime_config.hf_trusted_team_mode_enabled
3007
+ hf_bucket_validations_enabled = runtime_config.hf_bucket_validations_enabled or runtime_config.hf_trusted_team_mode_enabled
3008
  validation_repository = ProjectAwareValidationRepository(
3009
  fallback_repository=base_validation_repository,
3010
  project_lookup=admin_manager.get_project,
 
3015
  or None
3016
  ),
3017
  actor_token_provider=lambda _project, username: auth_service.get_hf_token_for_user(username),
3018
+ enable_hf_project_state=hf_state_writes_enabled,
3019
+ enable_hf_bucket_validations=hf_bucket_validations_enabled,
3020
  )
3021
  validation_service = ValidationService(validation_repository)
3022
  hf_state_initializer = HfProjectStateStoreInitializer()
3023
+ hf_bucket_initializer = HfBucketValidationInitializer()
3024
+ hf_bucket_permission_probe = HfBucketPermissionProbe()
3025
  hf_state_connector = HfProjectStateStoreConnector()
3026
  hf_state_permission_probe = HfProjectStatePermissionProbe()
3027
  hf_state_sync = HfProjectStateStoreSync()
 
3193
  sync_project_slugs: set[str] | None = None,
3194
  archived_projects: list[Project] | None = None,
3195
  ) -> list[str]:
3196
+ if not hf_state_writes_enabled:
3197
  return []
3198
 
3199
  user_access = auth_service.export_user_access_map(include_inactive=True)
 
3283
  section_header_html(
3284
  "Secure access",
3285
  "Sign in with your Hugging Face identity",
3286
+ (
3287
+ "Each trusted collaborator accesses private data and validation storage with their own Hugging Face token."
3288
+ if runtime_config.hf_trusted_team_mode_enabled
3289
+ else "Your account controls project access, validator attribution, and private dataset access."
3290
+ ),
3291
  )
3292
  )
3293
  username_input, session_output, login_button, error_message = create_login_page(
3294
  auth_service,
3295
  allow_username_login=allow_username_login,
3296
  enable_oauth_login=_is_running_in_hf_space(),
3297
+ auth_mode_label=(
3298
+ "Private HF trusted-team mode is active. Personal token login is the validated workflow; "
3299
+ "project data remain private to authorized Hugging Face organization members."
3300
+ if runtime_config.hf_trusted_team_mode_enabled
3301
+ else auth_mode_description
3302
+ ),
3303
+ trusted_team_mode=runtime_config.hf_trusted_team_mode_enabled,
3304
  )
3305
 
3306
  # Store session ID when login succeeds
 
3379
  class_name="bn-panel-soft",
3380
  )
3381
  )
3382
+ if runtime_config.hf_trusted_team_mode_enabled:
3383
+ gr.HTML(
3384
+ inline_hint_html(
3385
+ "Private HF trusted-team mode is active. Source datasets must be private. "
3386
+ "Choose collaborative visibility; the data remain private on Hugging Face. Before "
3387
+ "assigning validators here, add each trusted collaborator with write access to the "
3388
+ "project's dedicated Hugging Face organization.",
3389
+ "info",
3390
+ )
3391
+ )
3392
  with gr.Row():
3393
  create_project_slug = gr.Textbox(
3394
  label="New Project Slug",
 
3408
  value="collaborative",
3409
  )
3410
  create_project_token = gr.Textbox(
3411
+ label=(
3412
+ "One-time setup HF token (not stored)"
3413
+ if runtime_config.hf_trusted_team_mode_enabled
3414
+ else "Project HF Token (optional)"
3415
+ ),
3416
  placeholder="hf_xxx...",
3417
  type="password",
3418
  )
 
3466
  return "Visibility must be private or collaborative.", gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), gr.update(), session
3467
  if (
3468
  _is_running_in_hf_space()
3469
+ and hf_state_writes_enabled
3470
+ and not runtime_config.hf_trusted_team_mode_enabled
3471
  and getattr(session, "authentication_method", "") != "oauth"
3472
  ):
3473
  return (
 
3511
  dataset_repo_id=repo_id,
3512
  visibility=visibility_value,
3513
  owner_username=session.username,
3514
+ dataset_token=None if runtime_config.hf_trusted_team_mode_enabled else explicit_project_token,
3515
  active=True,
3516
  )
3517
+ bucket_result = None
3518
+ if runtime_config.hf_trusted_team_mode_enabled:
3519
+ try:
3520
+ bucket_result = _initialize_hf_trusted_team_storage(
3521
+ project=project,
3522
+ token=operation_token,
3523
+ bucket_initializer=hf_bucket_initializer,
3524
+ )
3525
+ except HfBucketValidationError as exc:
3526
+ return (
3527
+ f"Project was not created because its private validation Bucket could not be initialized: {exc}",
3528
+ _project_rows(),
3529
+ gr.update(),
3530
+ gr.update(),
3531
+ gr.update(),
3532
+ gr.update(),
3533
+ gr.update(),
3534
+ gr.update(),
3535
+ gr.update(),
3536
+ gr.update(),
3537
+ gr.update(),
3538
+ gr.update(),
3539
+ session,
3540
+ )
3541
  try:
3542
  state_result = hf_state_initializer.initialize(
3543
  project=project,
 
3584
  project.state_schema_version = HF_PROJECT_STATE_SCHEMA_VERSION
3585
  project.state_status = "ready"
3586
  state_repo_action = "initialized" if state_result.initialized else "connected"
3587
+ bucket_message = (
3588
+ f" Private validation Bucket ready: {bucket_result.bucket_id}."
3589
+ if bucket_result is not None
3590
+ else ""
3591
+ )
3592
+ recovery_message = (
3593
+ f" Add `{state_result.state_repo_id}` to `BIRDNET_HF_PROJECT_STATE_REPOS` before rebuilding "
3594
+ "the Space so this project is restored after restart."
3595
+ if runtime_config.hf_trusted_team_mode_enabled
3596
+ else ""
3597
+ )
3598
 
3599
  created = admin_manager.register_project(project)
3600
  if not created:
 
3628
 
3629
  return (
3630
  (
3631
+ f"Project '{slug}' created successfully. Private state repo {state_repo_action}: {state_result.state_repo_id}.{bucket_message}{recovery_message}"
3632
  if persisted
3633
+ else f"Project '{slug}' created with private state repo {state_repo_action} ({state_result.state_repo_id}).{bucket_message}{recovery_message} Could not persist bootstrap files: {persist_error}"
3634
  ),
3635
  _project_rows(),
3636
  gr.update(choices=admin_projects, value=slug),
 
3689
  session,
3690
  )
3691
 
3692
+ if runtime_config.hf_trusted_team_mode_enabled:
3693
+ if (
3694
+ (loaded.project.validation_backend or "").strip() != HF_BUCKET_VALIDATION_BACKEND
3695
+ or not (loaded.project.validation_bucket_id or "").strip()
3696
+ ):
3697
+ return (
3698
+ "Project state could not be connected in trusted-team mode because its manifest "
3699
+ "does not declare private Hugging Face Bucket validation storage.",
3700
+ _project_rows(),
3701
+ gr.update(),
3702
+ gr.update(),
3703
+ gr.update(),
3704
+ gr.update(),
3705
+ gr.update(),
3706
+ session,
3707
+ )
3708
+ try:
3709
+ _validate_hf_trusted_team_source_dataset(
3710
+ project=loaded.project,
3711
+ token=_session_hf_token(session),
3712
+ )
3713
+ except HfBucketValidationError as exc:
3714
+ return (
3715
+ f"Project state could not be connected in trusted-team mode: {exc}",
3716
+ _project_rows(),
3717
+ gr.update(),
3718
+ gr.update(),
3719
+ gr.update(),
3720
+ gr.update(),
3721
+ gr.update(),
3722
+ session,
3723
+ )
3724
+
3725
  slug = loaded.project.project_slug
3726
  if admin_manager.get_project(slug) is not None:
3727
  return (
 
3865
  project = admin_manager.get_project(project_slug)
3866
  if project is None:
3867
  return "Select a valid project.", gr.update(), gr.update(), gr.update(), gr.update()
3868
+ if (
3869
+ runtime_config.hf_trusted_team_mode_enabled
3870
+ and (project.validation_backend or "").strip() == HF_BUCKET_VALIDATION_BACKEND
3871
+ ):
3872
+ return (
3873
+ "Trusted-team HF projects never store a shared project token. Each collaborator must sign "
3874
+ "in using their own Hugging Face authorization.",
3875
+ gr.update(value=""),
3876
+ gr.update(value=False),
3877
+ _project_rows(),
3878
+ "",
3879
+ )
3880
 
3881
  if bool(clear_token):
3882
  project.dataset_token = None
 
4292
 
4293
  session_state.change(
4294
  fn=lambda s: (
4295
+ gr.update(visible=bool(s is not None and not runtime_config.hf_trusted_team_mode_enabled)),
4296
  gr.update(visible=bool(s is not None)),
4297
  gr.update(visible=bool(s is not None)),
4298
  gr.update(visible=bool(s is not None)),
 
4346
  interactive=False,
4347
  allow_custom_value=True,
4348
  )
4349
+ with gr.Group(
4350
+ visible=not runtime_config.hf_trusted_team_mode_enabled,
4351
+ elem_classes=["bn-panel-soft"],
4352
+ ):
4353
  gr.Markdown("### Private state OAuth diagnostic")
4354
  state_authorization_status = gr.Markdown(
4355
  "The current OAuth scope may allow Pull Requests rather than direct state writes. "
 
4359
  "Check OAuth state write capability",
4360
  elem_classes=["bn-soft-action"],
4361
  )
4362
+ with gr.Group(
4363
+ visible=runtime_config.hf_trusted_team_mode_enabled,
4364
+ elem_classes=["bn-panel-soft"],
4365
+ ):
4366
+ gr.Markdown("### Private project storage access")
4367
+ trusted_storage_status = gr.Markdown(
4368
+ "Run this check once for each trusted collaborator before validation. "
4369
+ "It verifies private validation storage write access with that user's own Hugging Face token."
4370
+ )
4371
+ test_trusted_storage_btn = gr.Button(
4372
+ "Check private validation storage access",
4373
+ elem_classes=["bn-soft-action"],
4374
+ )
4375
  invitations_info = gr.Markdown(value="")
4376
  invitations_overview = gr.HTML(value=invite_panel_html(0))
4377
  invite_selector = gr.Dropdown(choices=[], label="Pending Invites", interactive=False)
 
4625
  f"A private diagnostic write was confirmed in `{result.state_repo_id}`."
4626
  )
4627
 
4628
+ def test_trusted_storage_authorization(selected: str, session):
4629
+ if session is None:
4630
+ return "Login with your Hugging Face account before testing private validation storage."
4631
+ project_slug = (selected or "").strip()
4632
+ project = admin_manager.get_project(project_slug) if project_slug else None
4633
+ if project is None or project_slug not in session.authorized_projects:
4634
+ return "Select a project that is authorized for the signed-in account."
4635
+ if (project.validation_backend or "").strip() != HF_BUCKET_VALIDATION_BACKEND:
4636
+ return "This project does not use private Hugging Face Bucket validation storage."
4637
+ try:
4638
+ result = hf_bucket_permission_probe.probe(
4639
+ bucket_id=project.validation_bucket_id or "",
4640
+ actor_username=session.username,
4641
+ token=_session_hf_token(session),
4642
+ )
4643
+ except HfBucketValidationError as exc:
4644
+ return f"Private storage access test failed: {exc}"
4645
+ return (
4646
+ f"Private validation storage verified for **{result.actor_username}**. "
4647
+ f"Read/write access was confirmed in `{result.bucket_id}` and the diagnostic marker was removed."
4648
+ )
4649
+
4650
  project_selector.change(
4651
  fn=update_selected_project,
4652
  inputs=[project_selector, session_state],
 
4658
  inputs=[project_selector, session_state],
4659
  outputs=[state_authorization_status],
4660
  )
4661
+ test_trusted_storage_btn.click(
4662
+ fn=test_trusted_storage_authorization,
4663
+ inputs=[project_selector, session_state],
4664
+ outputs=[trusted_storage_status],
4665
+ )
4666
 
4667
  create_project_event.then(
4668
  fn=update_project_selector,
 
6010
  def _render_settings_health():
6011
  backend = (runtime_config.state_backend or "filesystem").strip().lower()
6012
  supabase_ready = bool(runtime_config.supabase_url and runtime_config.supabase_service_role_key)
6013
+ if runtime_config.hf_trusted_team_mode_enabled:
6014
+ state_label = "HF trusted-team"
6015
+ state_tone = "ok"
6016
+ else:
6017
+ state_label = "Supabase" if backend in {"supabase", "postgres", "postgresql"} and supabase_ready else "Filesystem"
6018
+ state_tone = "ok" if state_label == "Supabase" else "warn"
6019
  normalized_bootstrap_dir = str(runtime_config.bootstrap_base_dir).replace("\\", "/").rstrip("/")
6020
  filesystem_durable = normalized_bootstrap_dir == "/data" or normalized_bootstrap_dir.startswith("/data/")
6021
+ if state_label == "HF trusted-team":
6022
+ durability_label = "private HF state/Bucket"
6023
+ durability_tone = "ok"
6024
+ elif state_label == "Supabase":
6025
  durability_label = "external durable"
6026
  durability_tone = "ok"
6027
  elif filesystem_durable:
 
6033
  demo_label = "enabled" if runtime_config.enable_demo_bootstrap else "disabled"
6034
  demo_tone = "warn" if runtime_config.enable_demo_bootstrap else "ok"
6035
  invite_email_label = "enabled" if runtime_config.invite_email_enabled and runtime_config.emailjs_enabled else "disabled"
6036
+ hf_project_state_label = "enabled" if hf_state_writes_enabled else "disabled"
6037
+ hf_bucket_validation_label = (
6038
+ "trusted-team active"
6039
+ if runtime_config.hf_trusted_team_mode_enabled
6040
+ else ("legacy experimental" if runtime_config.hf_bucket_validations_enabled else "disabled")
6041
+ )
6042
  hf_project_state_repos_label = str(len(runtime_config.hf_project_state_repos))
6043
  hf_space_label = os.getenv("SPACE_ID") or "local runtime"
6044
  health_html = settings_health_html(
6045
  [
6046
  ("State backend", state_label, state_tone),
6047
+ ("HF project-state writes", hf_project_state_label, "ok" if hf_state_writes_enabled else "info"),
6048
+ ("HF Bucket validations", hf_bucket_validation_label, "ok" if runtime_config.hf_trusted_team_mode_enabled else ("warn" if runtime_config.hf_bucket_validations_enabled else "info")),
6049
+ (
6050
+ "HF project-state repos",
6051
+ hf_project_state_repos_label,
6052
+ "ok" if runtime_config.hf_project_state_repos else ("warn" if runtime_config.hf_trusted_team_mode_enabled else "info"),
6053
+ ),
6054
  ("Auth mode", auth_mode_label, "ok" if not allow_username_login else "warn"),
6055
  ("Destructive write guard", "enabled", "ok"),
6056
  ("Local JSON backups", "enabled" if state_label == "Filesystem" else "not used", "ok" if state_label == "Filesystem" else "info"),
 
6065
  ]
6066
  )
6067
  status_text = (
6068
+ (
6069
+ "HF trusted-team storage is active. Private dataset access and Bucket writes require each "
6070
+ "trusted collaborator's own Hugging Face permissions; "
6071
+ + (
6072
+ "no project-state repository is configured, so projects will not be restored after a rebuild."
6073
+ if not runtime_config.hf_project_state_repos
6074
+ else "configured state repositories will be used for restart recovery."
6075
+ )
6076
+ if state_label == "HF trusted-team"
6077
+ else "Supabase persistence is active. Project and ACL removals are guarded against accidental broad overwrites."
6078
+ )
6079
+ if state_label in {"HF trusted-team", "Supabase"}
6080
  else (
6081
  "Filesystem persistence is active with local JSON backups and destructive-write guards. "
6082
  "On free Spaces without persistent storage, local files are still not durable across rebuilds."
src/ui/login_page.py CHANGED
@@ -84,6 +84,7 @@ def create_login_page(
84
  allow_username_login: bool = True,
85
  enable_oauth_login: bool = False,
86
  auth_mode_label: str = "",
 
87
  ) -> Tuple[gr.Textbox, gr.Textbox, gr.Button, gr.Markdown]:
88
  """Create a Gradio login page with username input and session tracking.
89
 
@@ -98,34 +99,41 @@ def create_login_page(
98
  gr.Markdown("")
99
  with gr.Column(scale=6):
100
  gr.Markdown("# BirdNET Validation Platform")
101
- gr.Markdown("Sign in with your Hugging Face account to access project validation workflows.")
 
 
 
 
 
 
102
  if auth_mode_label:
103
  gr.Markdown(auth_mode_label)
104
 
105
- oauth_continue_button = None
106
- if enable_oauth_login:
107
  gr.Markdown(
108
- "**OAuth access required for private collaborative state.** "
109
- "Complete both steps below; signing into Hugging Face alone does not open an app session."
 
 
110
  )
111
- gr.LoginButton("1. Sign in with Hugging Face", logout_value="Sign out ({})")
112
- oauth_continue_button = gr.Button(
113
- "2. Enter workspace with OAuth authorization",
114
- variant="primary",
115
  )
116
-
117
- gr.Markdown(
118
- "Manual token access is available for development or legacy access, "
119
- "but cannot be used for the private collaborative state authorization test."
120
- )
121
  username_input = gr.Textbox(
122
  label="Username",
123
  placeholder="Enter your username" if allow_username_login else "Username login disabled in this deployment",
124
  lines=1,
125
  interactive=allow_username_login,
 
126
  )
127
  hf_token_input = gr.Textbox(
128
- label="Hugging Face Token (legacy/development only)",
 
 
 
 
129
  placeholder="hf_xxx...",
130
  type="password",
131
  lines=1,
@@ -133,7 +141,32 @@ def create_login_page(
133
 
134
  error_message = gr.Markdown()
135
 
136
- login_button = gr.Button("Login", variant="primary", scale=1)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
137
 
138
  session_output = gr.Textbox(
139
  label="Session ID",
 
84
  allow_username_login: bool = True,
85
  enable_oauth_login: bool = False,
86
  auth_mode_label: str = "",
87
+ trusted_team_mode: bool = False,
88
  ) -> Tuple[gr.Textbox, gr.Textbox, gr.Button, gr.Markdown]:
89
  """Create a Gradio login page with username input and session tracking.
90
 
 
99
  gr.Markdown("")
100
  with gr.Column(scale=6):
101
  gr.Markdown("# BirdNET Validation Platform")
102
+ gr.Markdown(
103
+ (
104
+ "Use your own Hugging Face authorization to access private project validation workflows."
105
+ if trusted_team_mode
106
+ else "Sign in with your Hugging Face account to access project validation workflows."
107
+ )
108
+ )
109
  if auth_mode_label:
110
  gr.Markdown(auth_mode_label)
111
 
112
+ if trusted_team_mode:
 
113
  gr.Markdown(
114
+ "### Personal token access\n"
115
+ "This is the validated access method for private trusted-team projects. "
116
+ "Use a token from your own account with permission to read the project's private dataset "
117
+ "and write to its private validation Bucket."
118
  )
119
+ else:
120
+ gr.Markdown(
121
+ "Manual token access is available for development or legacy access, "
122
+ "but cannot be used for the private collaborative state authorization test."
123
  )
 
 
 
 
 
124
  username_input = gr.Textbox(
125
  label="Username",
126
  placeholder="Enter your username" if allow_username_login else "Username login disabled in this deployment",
127
  lines=1,
128
  interactive=allow_username_login,
129
+ visible=allow_username_login,
130
  )
131
  hf_token_input = gr.Textbox(
132
+ label=(
133
+ "Personal Hugging Face Token"
134
+ if trusted_team_mode
135
+ else "Hugging Face Token (legacy/development only)"
136
+ ),
137
  placeholder="hf_xxx...",
138
  type="password",
139
  lines=1,
 
141
 
142
  error_message = gr.Markdown()
143
 
144
+ login_button = gr.Button(
145
+ "Enter with personal token" if trusted_team_mode else "Login",
146
+ variant="primary",
147
+ scale=1,
148
+ )
149
+
150
+ oauth_continue_button = None
151
+ if enable_oauth_login:
152
+ gr.Markdown(
153
+ (
154
+ "### Optional OAuth test\n"
155
+ "The hosted Space can sign you in with Hugging Face OAuth. "
156
+ "For this interim private-storage mode, complete the personal-token test above before "
157
+ "using real validation data because OAuth Bucket writes have not yet been proven."
158
+ if trusted_team_mode
159
+ else (
160
+ "**OAuth access required for private collaborative state.** "
161
+ "Complete both steps below; signing into Hugging Face alone does not open an app session."
162
+ )
163
+ )
164
+ )
165
+ gr.LoginButton("1. Sign in with Hugging Face", logout_value="Sign out ({})")
166
+ oauth_continue_button = gr.Button(
167
+ "2. Enter workspace with OAuth authorization",
168
+ variant="secondary" if trusted_team_mode else "primary",
169
+ )
170
 
171
  session_output = gr.Textbox(
172
  label="Session ID",
src/ui/theme.py CHANGED
@@ -1449,8 +1449,15 @@ select {
1449
 
1450
 
1451
  def app_header_html(state_backend_message: str = "") -> str:
1452
- backend_text = "Supabase ready" if "Supabase state backend enabled" in state_backend_message else "Filesystem state"
1453
- backend_class = "bn-pill-ok" if "Supabase state backend enabled" in state_backend_message else "bn-pill-warn"
 
 
 
 
 
 
 
1454
  return f"""
1455
  <div class="bn-app-header">
1456
  <div>
 
1449
 
1450
 
1451
  def app_header_html(state_backend_message: str = "") -> str:
1452
+ if "HF trusted-team project storage enabled" in state_backend_message:
1453
+ backend_text = "Private HF state ready"
1454
+ backend_class = "bn-pill-ok"
1455
+ elif "Supabase state backend enabled" in state_backend_message:
1456
+ backend_text = "Supabase ready"
1457
+ backend_class = "bn-pill-ok"
1458
+ else:
1459
+ backend_text = "Filesystem state"
1460
+ backend_class = "bn-pill-warn"
1461
  return f"""
1462
  <div class="bn-app-header">
1463
  <div>
tests/unit/test_app_factory_audio_helpers.py CHANGED
@@ -3,6 +3,7 @@ from datetime import date
3
  from pathlib import Path
4
  import hashlib
5
  import json
 
6
 
7
  import pandas as pd
8
  import pytest
@@ -35,11 +36,13 @@ from src.ui.app_factory import (
35
  _persist_bootstrap_state,
36
  _resolve_username_login_policy,
37
  _resolve_project_fetch_token,
 
38
  _write_validation_export,
39
  )
40
  from src.auth.auth_service import AuthService
41
  from src.config.runtime_config import RuntimeConfig
42
  from src.domain.models import Detection, Project, Role
 
43
  from src.repositories.state_safety import StateSafetyError
44
  from src.services.hf_project_state_store import HfProjectStateStoreLoadedProject
45
  from src.ui.admin_panel import AdminPanelManager
@@ -64,6 +67,38 @@ class FakeAudioService:
64
  self.cleaned.append(cache_key)
65
 
66
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
  class InaccessibleAudioService:
68
  def fetch(self, dataset_repo: str, audio_id: str, **kwargs: object) -> FakeFetchResult:
69
  _ = (dataset_repo, audio_id, kwargs)
@@ -197,6 +232,87 @@ class FakeQueueService:
197
  return FakeQueueService._Page()
198
 
199
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
200
  def test_extract_audio_id_from_list_rows() -> None:
201
  rows = [["k1", "audio_01", "sp", 0.9, 0.0, 1.0]]
202
  assert _extract_audio_id(rows, 0) == "audio_01"
 
3
  from pathlib import Path
4
  import hashlib
5
  import json
6
+ from types import SimpleNamespace
7
 
8
  import pandas as pd
9
  import pytest
 
36
  _persist_bootstrap_state,
37
  _resolve_username_login_policy,
38
  _resolve_project_fetch_token,
39
+ _initialize_hf_trusted_team_storage,
40
  _write_validation_export,
41
  )
42
  from src.auth.auth_service import AuthService
43
  from src.config.runtime_config import RuntimeConfig
44
  from src.domain.models import Detection, Project, Role
45
+ from src.repositories.hf_bucket_validation_repository import HfBucketValidationError, HfBucketValidationInitResult
46
  from src.repositories.state_safety import StateSafetyError
47
  from src.services.hf_project_state_store import HfProjectStateStoreLoadedProject
48
  from src.ui.admin_panel import AdminPanelManager
 
67
  self.cleaned.append(cache_key)
68
 
69
 
70
+ class FakeDatasetInfoApi:
71
+ def __init__(self, private: bool, organization_exists: bool = True) -> None:
72
+ self.private = private
73
+ self.organization_exists = organization_exists
74
+
75
+ def get_organization_overview(self, organization: str, token: str): # noqa: ANN001
76
+ assert organization == "project-org"
77
+ assert token == "hf_admin"
78
+ if not self.organization_exists:
79
+ raise RuntimeError("404 organization not found")
80
+ return SimpleNamespace(name=organization)
81
+
82
+ def repo_info(self, *, repo_id: str, repo_type: str, token: str): # noqa: ANN001
83
+ assert repo_id == "project-org/private-audio"
84
+ assert repo_type == "dataset"
85
+ assert token == "hf_admin"
86
+ return SimpleNamespace(private=self.private)
87
+
88
+
89
+ class FakeBucketInitializer:
90
+ def __init__(self) -> None:
91
+ self.calls: list[tuple[str, str, str | None]] = []
92
+
93
+ def initialize(self, *, project_slug: str, dataset_repo_id: str, token: str | None):
94
+ self.calls.append((project_slug, dataset_repo_id, token))
95
+ return HfBucketValidationInitResult(
96
+ bucket_id="project-org/private-audio_validation_state",
97
+ initialized=True,
98
+ reused_existing=False,
99
+ )
100
+
101
+
102
  class InaccessibleAudioService:
103
  def fetch(self, dataset_repo: str, audio_id: str, **kwargs: object) -> FakeFetchResult:
104
  _ = (dataset_repo, audio_id, kwargs)
 
232
  return FakeQueueService._Page()
233
 
234
 
235
+ def test_initialize_hf_trusted_team_storage_sets_bucket_without_persisting_shared_token() -> None:
236
+ project = Project(
237
+ project_slug="private-audio",
238
+ name="Private Audio",
239
+ dataset_repo_id="project-org/private-audio",
240
+ dataset_token="hf_shared_should_be_removed",
241
+ )
242
+ bucket_initializer = FakeBucketInitializer()
243
+
244
+ result = _initialize_hf_trusted_team_storage(
245
+ project=project,
246
+ token="hf_admin",
247
+ api=FakeDatasetInfoApi(private=True), # type: ignore[arg-type]
248
+ bucket_initializer=bucket_initializer, # type: ignore[arg-type]
249
+ )
250
+
251
+ assert result.bucket_id == "project-org/private-audio_validation_state"
252
+ assert project.validation_bucket_id == result.bucket_id
253
+ assert project.validation_backend == "hf_bucket"
254
+ assert project.dataset_token is None
255
+ assert bucket_initializer.calls == [("private-audio", "project-org/private-audio", "hf_admin")]
256
+
257
+
258
+ def test_initialize_hf_trusted_team_storage_rejects_public_source_dataset() -> None:
259
+ project = Project(
260
+ project_slug="private-audio",
261
+ name="Private Audio",
262
+ dataset_repo_id="project-org/private-audio",
263
+ )
264
+ bucket_initializer = FakeBucketInitializer()
265
+
266
+ with pytest.raises(HfBucketValidationError, match="only accepts a private dataset"):
267
+ _initialize_hf_trusted_team_storage(
268
+ project=project,
269
+ token="hf_admin",
270
+ api=FakeDatasetInfoApi(private=False), # type: ignore[arg-type]
271
+ bucket_initializer=bucket_initializer, # type: ignore[arg-type]
272
+ )
273
+
274
+ assert bucket_initializer.calls == []
275
+
276
+
277
+ def test_initialize_hf_trusted_team_storage_requires_organization_namespace() -> None:
278
+ project = Project(
279
+ project_slug="private-audio",
280
+ name="Private Audio",
281
+ dataset_repo_id="project-org/private-audio",
282
+ )
283
+ bucket_initializer = FakeBucketInitializer()
284
+
285
+ with pytest.raises(HfBucketValidationError, match="dedicated Hugging Face organization"):
286
+ _initialize_hf_trusted_team_storage(
287
+ project=project,
288
+ token="hf_admin",
289
+ api=FakeDatasetInfoApi(private=True, organization_exists=False), # type: ignore[arg-type]
290
+ bucket_initializer=bucket_initializer, # type: ignore[arg-type]
291
+ )
292
+
293
+ assert bucket_initializer.calls == []
294
+
295
+
296
+ def test_initialize_hf_trusted_team_storage_requires_collaborative_app_visibility() -> None:
297
+ project = Project(
298
+ project_slug="private-audio",
299
+ name="Private Audio",
300
+ dataset_repo_id="project-org/private-audio",
301
+ visibility="private",
302
+ )
303
+ bucket_initializer = FakeBucketInitializer()
304
+
305
+ with pytest.raises(HfBucketValidationError, match="visibility 'collaborative'"):
306
+ _initialize_hf_trusted_team_storage(
307
+ project=project,
308
+ token="hf_admin",
309
+ api=FakeDatasetInfoApi(private=True), # type: ignore[arg-type]
310
+ bucket_initializer=bucket_initializer, # type: ignore[arg-type]
311
+ )
312
+
313
+ assert bucket_initializer.calls == []
314
+
315
+
316
  def test_extract_audio_id_from_list_rows() -> None:
317
  rows = [["k1", "audio_01", "sp", 0.9, 0.0, 1.0]]
318
  assert _extract_audio_id(rows, 0) == "audio_01"
tests/unit/test_hf_bucket_validation_repository.py CHANGED
@@ -6,6 +6,7 @@ from src.domain.models import Validation
6
  from src.repositories.append_only_validation_repository import OptimisticLockError
7
  from src.repositories.hf_bucket_validation_repository import (
8
  HF_BUCKET_VALIDATION_BACKEND,
 
9
  HfBucketValidationError,
10
  HfBucketValidationInitializer,
11
  HfBucketValidationRepository,
@@ -99,6 +100,22 @@ def test_initializer_reuses_manifest_but_refuses_unknown_existing_data() -> None
99
  unsafe.initialize(project_slug="project-a", dataset_repo_id="owner/audio", token="hf_admin")
100
 
101
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
102
  def test_save_validation_writes_event_and_snapshot_without_repo_commit() -> None:
103
  api = FakeBucketApi({"snapshots/current.json": json.dumps({"project_slug": "project-a", "items": {}})})
104
  repository = HfBucketValidationRepository(bucket_id="owner/audio_validation_state", token="hf_user", api=api)
 
6
  from src.repositories.append_only_validation_repository import OptimisticLockError
7
  from src.repositories.hf_bucket_validation_repository import (
8
  HF_BUCKET_VALIDATION_BACKEND,
9
+ HfBucketPermissionProbe,
10
  HfBucketValidationError,
11
  HfBucketValidationInitializer,
12
  HfBucketValidationRepository,
 
100
  unsafe.initialize(project_slug="project-a", dataset_repo_id="owner/audio", token="hf_admin")
101
 
102
 
103
+ def test_bucket_permission_probe_reads_writes_verifies_and_removes_marker() -> None:
104
+ api = FakeBucketApi({"metadata/project.json": "{}"})
105
+ probe = HfBucketPermissionProbe(api=api)
106
+
107
+ result = probe.probe(
108
+ bucket_id="owner/audio_validation_state",
109
+ actor_username="validator-a",
110
+ token="hf_validator",
111
+ )
112
+
113
+ assert result.bucket_id == "owner/audio_validation_state"
114
+ assert result.actor_username == "validator-a"
115
+ assert result.diagnostic_path not in api.files
116
+ assert api.deletes[-1] == [result.diagnostic_path]
117
+
118
+
119
  def test_save_validation_writes_event_and_snapshot_without_repo_commit() -> None:
120
  api = FakeBucketApi({"snapshots/current.json": json.dumps({"project_slug": "project-a", "items": {}})})
121
  repository = HfBucketValidationRepository(bucket_id="owner/audio_validation_state", token="hf_user", api=api)
tests/unit/test_runtime_config.py CHANGED
@@ -31,3 +31,19 @@ def test_runtime_config_reads_bucket_validation_flag(monkeypatch) -> None: # no
31
  config = RuntimeConfig.from_env()
32
 
33
  assert config.hf_bucket_validations_enabled is True
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  config = RuntimeConfig.from_env()
32
 
33
  assert config.hf_bucket_validations_enabled is True
34
+
35
+
36
+ def test_runtime_config_reads_hf_trusted_team_mode_flag(monkeypatch) -> None: # noqa: ANN001
37
+ monkeypatch.setenv("BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED", "true")
38
+
39
+ config = RuntimeConfig.from_env()
40
+
41
+ assert config.hf_trusted_team_mode_enabled is True
42
+
43
+
44
+ def test_runtime_config_disables_hf_trusted_team_mode_by_default(monkeypatch) -> None: # noqa: ANN001
45
+ monkeypatch.delenv("BIRDNET_HF_TRUSTED_TEAM_MODE_ENABLED", raising=False)
46
+
47
+ config = RuntimeConfig.from_env()
48
+
49
+ assert config.hf_trusted_team_mode_enabled is False