File size: 18,212 Bytes
4b6e3ab
 
7af707c
4b6e3ab
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7af707c
 
 
 
 
 
 
 
4b6e3ab
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7af707c
4b6e3ab
7af707c
4b6e3ab
 
 
 
 
 
 
 
 
7af707c
 
 
 
 
 
 
 
 
4b6e3ab
7af707c
 
 
 
4b6e3ab
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7af707c
4b6e3ab
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7a8349e
 
 
 
 
 
 
 
 
 
 
4b6e3ab
 
 
 
 
 
 
 
 
7af707c
4b6e3ab
7af707c
 
 
 
 
4b6e3ab
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
0760ded
 
 
 
4b6e3ab
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
# Private HF State + OAuth + Batching Implementation Plan

Status: permission proof failed; retained as investigation record
Created: 2026-05-25
Target branch: `feature/project-state-security-privacy-review`

## Decision Summary

Adopt a private Hugging Face companion dataset repository (`*_state`) as the
project-owned durable state resource, authenticated through each user's Hugging
Face OAuth session and optimized for frequent validation through batched,
append-only writes.

This strategy is intended for both collaborative and private projects, with one
important boundary:

- The private `_state` repository protects project state, ACL, invites, and
  validation decisions.
- Access to audio files is independent. Validators of a private audio dataset
  must also have Hugging Face read access to that dataset.
- An app invitation is an application-level authorization rule; it does not by
  itself grant Hub permissions to an existing private audio dataset.

The experimental admin-owned private Bucket path must remain disabled by
default. The two-user test demonstrated that an invited validator could not
access the personal private Bucket using their own Hugging Face authorization.

The private personal `_state` OAuth path also cannot become the default
validation backend as designed. On 2026-05-25, an OAuth-authenticated admin
ran the permission diagnostic against an initialized private state repository.
The manifest was read, but a direct diagnostic commit was rejected with `403
Forbidden` and the Hub required `create_pr=1`. Pull Request-only contributions
are appropriate for reviewed content submissions, not a high-frequency
validation event stream.

## What Is Confirmed And What Requires Proof

### Confirmed by official documentation

1. Hugging Face OAuth supports `contribute-repos`, which can create repositories
   and access repositories created by the application, without granting access
   to other personal repositories.
2. Free users and free organizations include 100 GB of private Hub storage.
   Storage limits apply to datasets, repositories, and Buckets.
3. Git-backed Hub repositories are not intended for database-style high-write
   workloads; the Hub experience can degrade after thousands of commits.
4. Hub action limits include repository commits and are not documented as fixed
   public numbers. The general free-user API request allowance is measured in
   five-minute windows and may change.

### Permission proof result

The following proof was designed before implementation:

1. Admin A signs into the Space through OAuth.
2. Admin A creates a private `*_state` repository through the app.
3. Validator B signs into the same Space through OAuth.
4. Validator B accepts an app invitation.
5. Validator B reads the private state manifest and attempts one test
   validation write using only B's OAuth token.
6. Admin A reloads the project and observes that write.

Observed result:

1. A manual-token `upload_test5_state` attempt was invalid for testing OAuth
   scopes and was discarded as evidence.
2. Admin A then authenticated through OAuth and executed the diagnostic on
   `jrrribeiro/upload_test6_state`.
3. The diagnostic read the initialized repository but could not commit its
   proof file directly to `main`.
4. The Hub returned `403 Forbidden` and explicitly required `create_pr=1`.

Conclusion: this strategy failed before testing validator B. If even the
creating OAuth identity cannot perform direct state writes, `contribute-repos`
cannot support durable batched validation updates without a Pull Request merge
workflow or an administrator credential. Neither meets the target workflow.

## Target Ownership And Privacy Model

### State repository

For every project, the app creates:

```text
owner-or-org/audio_dataset_state
```

The repository is private regardless of whether the audio dataset is public,
gated, or private. It is owned by the project admin's Hugging Face namespace,
or by a project organization explicitly chosen by the admin. Its storage usage
belongs to that owner, not to the app maintainer.

### Project categories

| Project type | Audio dataset access | `_state` visibility | Validator requirement |
|---|---|---|---|
| Public collaborative | Public resolver access or authenticated read | Private | OAuth access to `_state` after permission proof |
| Gated collaborative | Approved gated access | Private | Own HF account approved for audio plus `_state` authorization |
| Private collaborative | Private repo read granted by owner/org | Private | Own HF account authorized for both audio and `_state` |
| Private individual | Admin-only private audio access | Private | Admin OAuth only |

The application may present invitations and roles, but must never suggest that
an invitation alone unlocks a private audio repository.

## Security Principles

1. No permanent administrator HF token is persisted in project records, state
   files, exports, logs, or browser-facing values.
2. Every write is attributed to the signed-in validator identity.
3. Collaborative state writes require that user's OAuth token; they never
   silently fall back to a stored administrator token.
4. `acl.json` governs application roles only after Hub authentication succeeds.
5. Private state recovery requires authenticated admin access and a recorded
   admin role in `acl.json`.
6. Events are append-only; snapshots are derived and recoverable.
7. Pending, locally accepted decisions must be visible to the user until
   durable synchronization is acknowledged.

## Target State Layout

Keep low-frequency control files small and human-inspectable:

```text
project.json
acl.json
invites.json
checkpoints/current.json
events/
  batches/
    <validator_username>/
      YYYYMMDD/
        <batch_id>.jsonl
manifests/
  batches.jsonl              # optional compact audit manifest
conflicts/
  <detection_key>.json       # optional materialized unresolved conflicts
```

### Control-plane data

`project.json`, `acl.json`, and `invites.json` are written only for project
administration operations. They must contain no token.

### Validation event batch

Each accepted decision is represented as an immutable event. A JSONL batch
contains multiple events from one authenticated validator:

```json
{
  "schema_version": 2,
  "event_id": "uuid",
  "batch_id": "uuid",
  "project_slug": "upload_test5",
  "detection_key": "stable-key",
  "status": "positive",
  "corrected_species": null,
  "notes": "",
  "validator": "validator_username",
  "client_sequence": 17,
  "expected_version": 0,
  "timestamp": "2026-05-25T12:00:00Z"
}
```

The server must derive validator identity from the authenticated session. It
must not trust a client-provided username.

## Batching And Durability Strategy

The existing `_state` repository implementation creates one Hub commit per
validation. Replace that behavior before real high-volume use.

### Outbox first

Batching creates a short interval before remote durability. A validation must
not disappear if the Space restarts during that interval.

Implement a per-user, per-project outbox:

1. Before advancing to the next audio, store the pending event in browser
   durable storage (IndexedDB preferred; local storage only for a small
   prototype).
2. Render the decision as `pending sync` in the interface.
3. Flush outbox events in batches to the private `_state` repository.
4. Remove local entries only after the remote batch commit is confirmed.
5. On browser reload or reconnect, restore and retry unacknowledged events.
6. Provide a visible `Synced`, `Syncing`, `Offline/Pending`, or `Conflict`
   status; never silently claim durability.

### Initial batching policy

Start conservatively and measure:

| Trigger | Initial value |
|---|---:|
| Batch size | 25 validation events |
| Flush interval while pending | 15 seconds |
| Immediate flush | project change, sign out, export request, manual Sync |
| Retry | exponential backoff for transient HTTP/429/5xx failures |
| Maximum local pending events | 1,000 before validation is paused for sync |

At one decision every 3 seconds, one active validator would produce about 20
events/minute. The initial policy should ordinarily reduce approximately 20
commits/minute to roughly 1 commit/minute per active validator, before further
measurement.

### Snapshot policy

Do not overwrite a shared full snapshot on every batch:

1. Validation batches are the durable source of truth.
2. Rebuild or incrementally reconcile current state from batch files.
3. Write `checkpoints/current.json` only periodically, such as every 20
   confirmed batches or on admin export/maintenance.
4. If parallel writers validate the same detection from the same base version,
   surface a conflict rather than discarding either event.

This prevents concurrent validators from overwriting one another's recent
decisions through frequent shared snapshot rewrites.

## Implementation Phases

## Phase 0 - Permission Proof And Architecture Gate

Objective: establish whether `contribute-repos` supports the required
two-account private `_state` collaboration.

Implementation status (2026-05-25): completed, failed permission gate.

Changes:

1. Disable Bucket creation for newly created test projects. Implemented.
2. Add a clearly labeled `State authorization test` action visible to project
   admins and invited validators in a test project.
   Implemented as **Private state authorization** in the **Projects** tab.
3. Record diagnostics without tokens: acting username, repository id,
   read/write result, HTTP category, and timestamp.
   Partially implemented: successful read/write proof writes actor, repo,
   project, timestamp, and diagnostic id; failed attempts are returned to the
   user without storing tokens.
4. Keep the test write isolated under `diagnostics/oauth-permission-proof/`.
   Implemented.
5. Reject state validation submissions that would fall back from a signed-in
   collaborator to an administrator token. Implemented.
6. Distinguish OAuth sessions from manual-token sessions and reject manual
   token creation/proof attempts in the hosted Space. Implemented after the
   first attempted test used manual tokens and therefore did not exercise
   `contribute-repos`.

Test note:

- `upload_test5_state` was created and tested through manual-token sessions;
  its failed collaborator write is not a valid OAuth permission proof.
- The next proof must create a new repository, such as `upload_test6_state`,
  after the admin completes both OAuth login steps in the hosted Space.

Tests:

1. Unit tests ensuring only the session OAuth token is passed to the state
   repository.
2. Unit test rejecting admin-token fallback for collaborator state writes.
3. Manual Space test with two free HF accounts and a fresh private `_state`
   repo.

Exit gate outcome:

- Failed. Do not proceed to the `_state` batching phases as a production
  backend.
- Preserve the code only as a permission diagnostic until the next backend
  strategy is chosen.
- Do not mask the failure using an administrator token.

## Phase 1 - Backend Selection And Removal Of Unsafe Fallbacks

Objective: make private `_state` the explicit validation backend for new
projects after the proof passes.

Changes:

1. Add a backend constant such as `hf_private_state_batched`.
2. New projects default to private `_state` validation storage; Bucket is shown
   as experimental/disabled and is not selected automatically.
3. Separate token policies:
   - audio read token: acting user's OAuth token;
   - state read/write token: acting user's OAuth token;
   - project creation token: creating admin's OAuth token.
4. Remove project `dataset_token` from the normal collaborative creation flow.
5. Reject collaborative state writes without an authenticated OAuth token.
6. Preserve old Bucket/Supabase projects as migration sources only.

Tests:

1. Router tests for each backend declaration.
2. Tests proving no project token or environment token is used for a
   collaborator write.
3. Tests for public, gated, and inaccessible private audio messages.

## Phase 2 - Batched Event Repository

Objective: replace one-commit-per-validation behavior.

Changes:

1. Implement `HfBatchedProjectStateValidationRepository`.
2. Add immutable JSONL batch commits partitioned by validator and date.
3. Add batch idempotency by `event_id` and `batch_id`.
4. Add event reconciliation, version conflict detection, and a bounded
   checkpoint writer.
5. Replace full-repository scans with checkpoint plus new-batch discovery.

Tests:

1. Serialize/deserialize event batches.
2. Retry the same batch without duplicate validations.
3. Merge batches from two validators.
4. Detect two incompatible decisions for one base version.
5. Recover a snapshot from batches after deleting/corrupting a checkpoint.
6. Assert commit count is proportional to batches, not decisions.

## Phase 3 - Durable Client Outbox And UX

Objective: maintain user trust during batching and transient network failures.

Changes:

1. Implement browser-side persistent outbox.
2. Add session handoff for flush/retry endpoints.
3. Show sync state near the validation action area and in Progress.
4. Add `Sync now` and prevent sign-out without an explicit pending-sync
   warning.
5. Exports must either flush first or identify unconfirmed local decisions
   clearly.

Tests:

1. Simulate failed network response and browser reload.
2. Confirm restored pending decision is resent once.
3. Confirm synced decision is removed from local storage.
4. Confirm project switching does not mix pending events.

## Phase 4 - Project Privacy And Access UX

Objective: make private/public access behavior understandable and safe.

Changes:

1. During project creation, distinguish:
   - audio visibility/access policy;
   - private validation-state policy.
2. Before validation starts, run an audio-read check and state-write
   authorization check for the signed-in user.
3. In Admin, show collaborator readiness statuses:
   - invited in app;
   - state access verified;
   - audio access verified;
   - ready to validate.
4. For private/gated audio, show instructions to grant Hub access externally;
   do not imply the app invitation performs it.

Tests:

1. Public audio/private state workflow.
2. Private audio with authorized validator.
3. Private audio without authorization.
4. Authorized state but inaccessible audio and the inverse.

## Phase 5 - Migration And Recovery

Objective: protect existing test/real work during transition.

Changes:

1. Classify current projects by backend: filesystem, Supabase, `_state`
   single-event commits, or Bucket.
2. Provide admin-only migration preview with counts and target state repo.
3. Import prior events into batch files without deleting the source.
4. Verify counts, latest statuses, versions, conflicts, and export output.
5. Switch backend only after verification succeeds.
6. Retain source as read-only recovery material until the admin confirms.

Tests:

1. Migration from existing `_state` event layout.
2. Migration from Supabase export fixture.
3. Migration abort leaves the original backend active.
4. CSV/XLSX equality before and after migration for confirmed state.

## Phase 6 - Scale And Multi-User Acceptance

Objective: qualify the backend before real scientific validation.

Test scenarios:

1. Two free HF accounts, one admin and one validator.
2. Public audio dataset with private `_state`.
3. Private audio dataset where both accounts have Hub access.
4. At least two concurrent validators operating on the same species queue.
5. Simulated validation rate of one decision every 3 seconds per validator.
6. Network failure, retry, browser refresh, and Space restart during pending
   sync.
7. Export while validation is ongoing.

Acceptance criteria:

1. No administrator token stored or transmitted to a validator.
2. No acknowledged validation lost after refresh or restart.
3. Duplicate retries do not duplicate decisions.
4. Conflicts are visible and exportable.
5. The number of commits stays near configured batch count, not click count.
6. The admin can recover a project solely from its private `_state` repo and
   authenticated HF access.
7. The final CSV/XLSX contains the complete detection table and all confirmed
   validation fields.

## Fallback Decision If OAuth Permission Proof Fails

If validator B cannot write an admin-created private `_state` repository with
`contribute-repos`, do not restore stored admin tokens as the default model.

Evaluate in this order:

1. A free Hugging Face organization owned by the project team, with the audio
   and `_state` resources owned by that organization and Hub permissions
   explicitly managed there. This keeps quota with the project owner but may
   expose broader organization write access than desired on the free tier.
2. Per-admin external database configuration, such as each project admin
   connecting their own Supabase project. This preserves cost ownership but
   introduces setup burden and free-tier database constraints.
3. A paid or institution-supported organization architecture only as an
   optional advanced deployment, never as the default requirement for users.

The organization-owned Storage Bucket investigation and permission-spike gate
are documented in
[`HF_ORGANIZATION_STORAGE_BACKEND_INVESTIGATION.md`](HF_ORGANIZATION_STORAGE_BACKEND_INVESTIGATION.md).

## Implementation Order

Do not implement batching first. The correct order is:

1. Phase 0 permission proof.
2. Phase 1 safe routing and removal of unsafe collaborative fallback.
3. Phase 2 batched remote event storage.
4. Phase 3 durable outbox.
5. Phase 4 access/readiness UX.
6. Phase 5 migration.
7. Phase 6 qualification.

This sequence avoids optimizing a storage route that may not satisfy the core
multi-account permission requirement.

## Official References

- Hugging Face OAuth scopes: <https://huggingface.co/docs/hub/oauth>
- Hugging Face storage limits: <https://huggingface.co/docs/hub/en/storage-limits>
- Hugging Face rate limits: <https://huggingface.co/docs/hub/en/rate-limits>
- Hugging Face organization security: <https://huggingface.co/docs/hub/en/organizations-security>