Spaces:
Sleeping
Sleeping
Sync app from GitHub Actions (feature/project-state-security-privacy-review)
Browse files- README.md +2 -4
- docs/TEMP_HF_ADMIN_OWNED_PRIVATE_STORAGE_DELIVERY_PLAN.md +2 -3
- docs/TEMP_PROJECT_STATE_SECURITY_REVIEW_PLAN.md +22 -4
- src/auth/auth_service.py +0 -25
- src/repositories/hf_bucket_validation_repository.py +0 -196
- src/services/hf_project_state_store.py +0 -88
- src/ui/app_factory.py +6 -157
- src/ui/login_page.py +31 -72
- tests/unit/test_app_factory_audio_helpers.py +3 -3
- tests/unit/test_auth_service.py +0 -17
- tests/unit/test_hf_bucket_validation_repository.py +0 -160
- tests/unit/test_hf_project_state_store.py +0 -66
- tests/unit/test_login_page.py +0 -23
README.md
CHANGED
|
@@ -197,7 +197,7 @@ Optional runtime settings:
|
|
| 197 |
- `BIRDNET_INVITE_TTL_HOURS` (default `72`)
|
| 198 |
- `BIRDNET_PAGE_SIZE` (default `25`)
|
| 199 |
- `BIRDNET_ENABLE_DEMO_BOOTSTRAP` (`false` in production)
|
| 200 |
-
- `BIRDNET_HF_PROJECT_STATE_WRITES_ENABLED=true` (
|
| 201 |
- `BIRDNET_HF_BUCKET_VALIDATIONS_ENABLED=true` (legacy experimental access for existing Bucket projects)
|
| 202 |
- `BIRDNET_HF_ADMIN_STORAGE_MODE_ENABLED=true` (private HF-only delivery mode for a small trusted team)
|
| 203 |
- `BIRDNET_HF_PROJECT_STATE_REPOS=jrrribeiro/audio-dataset_state` (optional manual recovery list; administrator-owned mode discovers companion state repos automatically)
|
|
@@ -216,7 +216,7 @@ BIRDNET_AUTH_MODE=hf_token
|
|
| 216 |
BIRDNET_HF_STORAGE_TOKEN=<administrator token stored only as a Space secret>
|
| 217 |
```
|
| 218 |
|
| 219 |
-
`HF_TOKEN` remains a compatibility fallback for the storage credential, but `BIRDNET_HF_STORAGE_TOKEN` is preferred because its purpose is explicit.
|
| 220 |
|
| 221 |
Projects previously created with a private `_state` repository in this mode can be recovered from the **Admin** tab using **Connect Existing State**. Sign in as an administrator recorded in its `acl.json`, then provide the `_state` repo id (for example, `jrrribeiro/audio_dataset_state`). Recovery accepts only a manifest with private Bucket validation storage and a private personal-namespace source dataset accessible to the configured storage account. The app loads the saved manifest, ACL, and pending invites.
|
| 222 |
|
|
@@ -228,8 +228,6 @@ Legacy Supabase state backend, available while migrating existing projects:
|
|
| 228 |
|
| 229 |
When Supabase is enabled, projects, ACL, invites, validation events, and current validation snapshots are stored in Supabase instead of `/data`. For eventual public distribution, the planned backend is administrator-owned Supabase provisioned through OAuth with an Edge Function validating Hugging Face identity; see `docs/TEMP_BYO_SUPABASE_EDGE_FUNCTION_HF_IDENTITY_PLAN.md`.
|
| 230 |
|
| 231 |
-
The **Private state OAuth diagnostic** action in **Projects** applies only to experimental direct `_state` writes. In administrator-owned storage mode, frequent validation state uses the Bucket rather than dataset repository commits.
|
| 232 |
-
|
| 233 |
Optional invite email settings:
|
| 234 |
|
| 235 |
- `BIRDNET_INVITE_EMAIL_ENABLED=true`
|
|
|
|
| 197 |
- `BIRDNET_INVITE_TTL_HOURS` (default `72`)
|
| 198 |
- `BIRDNET_PAGE_SIZE` (default `25`)
|
| 199 |
- `BIRDNET_ENABLE_DEMO_BOOTSTRAP` (`false` in production)
|
| 200 |
+
- `BIRDNET_HF_PROJECT_STATE_WRITES_ENABLED=true` (legacy migration support for existing `_state` projects)
|
| 201 |
- `BIRDNET_HF_BUCKET_VALIDATIONS_ENABLED=true` (legacy experimental access for existing Bucket projects)
|
| 202 |
- `BIRDNET_HF_ADMIN_STORAGE_MODE_ENABLED=true` (private HF-only delivery mode for a small trusted team)
|
| 203 |
- `BIRDNET_HF_PROJECT_STATE_REPOS=jrrribeiro/audio-dataset_state` (optional manual recovery list; administrator-owned mode discovers companion state repos automatically)
|
|
|
|
| 216 |
BIRDNET_HF_STORAGE_TOKEN=<administrator token stored only as a Space secret>
|
| 217 |
```
|
| 218 |
|
| 219 |
+
`HF_TOKEN` remains a compatibility fallback for the server-side storage credential, but `BIRDNET_HF_STORAGE_TOKEN` is preferred because its purpose is explicit. Users sign in through Hugging Face OAuth; no personal-token login is exposed in the production interface. On restart the app uses the protected credential to find the administrator's companion `*_state` repositories automatically; `BIRDNET_HF_PROJECT_STATE_REPOS` remains available as an explicit fallback or migration list.
|
| 220 |
|
| 221 |
Projects previously created with a private `_state` repository in this mode can be recovered from the **Admin** tab using **Connect Existing State**. Sign in as an administrator recorded in its `acl.json`, then provide the `_state` repo id (for example, `jrrribeiro/audio_dataset_state`). Recovery accepts only a manifest with private Bucket validation storage and a private personal-namespace source dataset accessible to the configured storage account. The app loads the saved manifest, ACL, and pending invites.
|
| 222 |
|
|
|
|
| 228 |
|
| 229 |
When Supabase is enabled, projects, ACL, invites, validation events, and current validation snapshots are stored in Supabase instead of `/data`. For eventual public distribution, the planned backend is administrator-owned Supabase provisioned through OAuth with an Edge Function validating Hugging Face identity; see `docs/TEMP_BYO_SUPABASE_EDGE_FUNCTION_HF_IDENTITY_PLAN.md`.
|
| 230 |
|
|
|
|
|
|
|
| 231 |
Optional invite email settings:
|
| 232 |
|
| 233 |
- `BIRDNET_INVITE_EMAIL_ENABLED=true`
|
docs/TEMP_HF_ADMIN_OWNED_PRIVATE_STORAGE_DELIVERY_PLAN.md
CHANGED
|
@@ -53,8 +53,7 @@ For public distribution to independent administrators, this model is not suffici
|
|
| 53 |
8. At startup the app automatically discovers the administrator's private companion `*_state` datasets for rebuild recovery; a manual repo list remains an optional fallback.
|
| 54 |
9. This mode takes precedence over configured Supabase persistence so durable project state does not remain dependent on the app operator's Supabase database.
|
| 55 |
10. Empty `*_state` repositories left by interrupted setup attempts are ignored during discovery until they contain a `project.json` manifest.
|
| 56 |
-
11. The
|
| 57 |
-
12. The backend storage health check now verifies the private audio dataset, recovered `_state` manifest and admin ACL, Bucket manifest, current snapshot, latest audit-event coverage, and a removable write proof before reporting the project as healthy.
|
| 58 |
|
| 59 |
## Space Configuration
|
| 60 |
|
|
@@ -71,7 +70,7 @@ Configure as a Secret:
|
|
| 71 |
BIRDNET_HF_STORAGE_TOKEN=<administrator credential with required private dataset/repo/Bucket read-write access>
|
| 72 |
```
|
| 73 |
|
| 74 |
-
Use a scoped credential limited to the administrator's required resources when Hugging Face token controls permit it. Do not share it with validators
|
| 75 |
|
| 76 |
## Acceptance Gate Before Real Validation
|
| 77 |
|
|
|
|
| 53 |
8. At startup the app automatically discovers the administrator's private companion `*_state` datasets for rebuild recovery; a manual repo list remains an optional fallback.
|
| 54 |
9. This mode takes precedence over configured Supabase persistence so durable project state does not remain dependent on the app operator's Supabase database.
|
| 55 |
10. Empty `*_state` repositories left by interrupted setup attempts are ignored during discovery until they contain a `project.json` manifest.
|
| 56 |
+
11. The storage integrity diagnostic was used during acceptance testing and removed from the end-user interface after successful validation; validators do not receive direct Bucket permissions.
|
|
|
|
| 57 |
|
| 58 |
## Space Configuration
|
| 59 |
|
|
|
|
| 70 |
BIRDNET_HF_STORAGE_TOKEN=<administrator credential with required private dataset/repo/Bucket read-write access>
|
| 71 |
```
|
| 72 |
|
| 73 |
+
Use a scoped credential limited to the administrator's required resources when Hugging Face token controls permit it. Do not share it with validators. End users authenticate through OAuth only.
|
| 74 |
|
| 75 |
## Acceptance Gate Before Real Validation
|
| 76 |
|
docs/TEMP_PROJECT_STATE_SECURITY_REVIEW_PLAN.md
CHANGED
|
@@ -970,9 +970,27 @@ Implemented on branch `feature/project-state-security-privacy-review`:
|
|
| 970 |
- The final health status is successful only after a temporary backend write can be read and removed from the private Bucket.
|
| 971 |
- Ordinary validation, export, and authorization flows are unchanged.
|
| 972 |
|
| 973 |
-
|
| 974 |
|
| 975 |
-
1.
|
| 976 |
-
2.
|
| 977 |
-
3.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 978 |
|
|
|
|
| 970 |
- The final health status is successful only after a temporary backend write can be read and removed from the private Bucket.
|
| 971 |
- Ordinary validation, export, and authorization flows are unchanged.
|
| 972 |
|
| 973 |
+
Acceptance performed:
|
| 974 |
|
| 975 |
+
1. Ran the health check for a real private project in the deployed Space.
|
| 976 |
+
2. Restarted the Space and confirmed recovery from `_state` and preserved validations.
|
| 977 |
+
3. Confirmed simultaneous authorized-user operation before removing the diagnostic UI from the product experience.
|
| 978 |
+
|
| 979 |
+
### 2026-05-27: Post-acceptance removal of diagnostic and token-login UI
|
| 980 |
+
|
| 981 |
+
Implemented after successful private-storage and restart testing:
|
| 982 |
+
|
| 983 |
+
1. Removed manual Hugging Face token login from the product interface.
|
| 984 |
+
- Hosted deployments now present Hugging Face OAuth as the only end-user authentication path.
|
| 985 |
+
- Local/demo username access remains available only for development.
|
| 986 |
+
- The protected backend storage secret remains unchanged and is never exposed as a login method.
|
| 987 |
+
|
| 988 |
+
2. Removed acceptance-only diagnostics from the Projects experience.
|
| 989 |
+
- Removed the private Bucket backend health button and its temporary marker flow.
|
| 990 |
+
- Removed the experimental direct `_state` OAuth write diagnostic.
|
| 991 |
+
- Kept the tested storage architecture, recovery, ACL enforcement, event compaction, and exports unchanged.
|
| 992 |
+
|
| 993 |
+
3. Reduced dormant code associated with those workflows.
|
| 994 |
+
- Removed the user-token authentication method.
|
| 995 |
+
- Removed diagnostic service classes and their narrowly scoped tests after acceptance was completed.
|
| 996 |
|
src/auth/auth_service.py
CHANGED
|
@@ -5,8 +5,6 @@ from datetime import UTC, datetime, timedelta
|
|
| 5 |
from typing import Dict, List, Optional
|
| 6 |
import uuid
|
| 7 |
|
| 8 |
-
from huggingface_hub import HfApi
|
| 9 |
-
|
| 10 |
from src.domain.models import Role, User
|
| 11 |
|
| 12 |
|
|
@@ -259,29 +257,6 @@ class AuthService:
|
|
| 259 |
self._sessions[session_id] = session
|
| 260 |
return session
|
| 261 |
|
| 262 |
-
def login_with_hf_token(self, token: str) -> tuple[Optional[Session], str]:
|
| 263 |
-
"""Authenticate using Hugging Face personal token and resolve username via whoami."""
|
| 264 |
-
token_value = (token or "").strip()
|
| 265 |
-
if not token_value:
|
| 266 |
-
return None, "Please provide a Hugging Face token"
|
| 267 |
-
|
| 268 |
-
try:
|
| 269 |
-
whoami = HfApi().whoami(token=token_value)
|
| 270 |
-
except Exception:
|
| 271 |
-
return None, "Invalid Hugging Face token or network error"
|
| 272 |
-
|
| 273 |
-
username = str(whoami.get("name") or "").strip()
|
| 274 |
-
if not username:
|
| 275 |
-
return None, "Unable to resolve Hugging Face username from token"
|
| 276 |
-
|
| 277 |
-
email_value = str(whoami.get("email") or "").strip() or None
|
| 278 |
-
return self.login_with_verified_hf_identity(
|
| 279 |
-
username=username,
|
| 280 |
-
token=token_value,
|
| 281 |
-
email=email_value,
|
| 282 |
-
authentication_method="hf_token",
|
| 283 |
-
)
|
| 284 |
-
|
| 285 |
def login_with_verified_hf_identity(
|
| 286 |
self,
|
| 287 |
*,
|
|
|
|
| 5 |
from typing import Dict, List, Optional
|
| 6 |
import uuid
|
| 7 |
|
|
|
|
|
|
|
| 8 |
from src.domain.models import Role, User
|
| 9 |
|
| 10 |
|
|
|
|
| 257 |
self._sessions[session_id] = session
|
| 258 |
return session
|
| 259 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 260 |
def login_with_verified_hf_identity(
|
| 261 |
self,
|
| 262 |
*,
|
src/repositories/hf_bucket_validation_repository.py
CHANGED
|
@@ -123,25 +123,6 @@ class HfBucketCompactionResult:
|
|
| 123 |
compacted_event_count: int
|
| 124 |
|
| 125 |
|
| 126 |
-
@dataclass(frozen=True)
|
| 127 |
-
class HfBucketPermissionProbeResult:
|
| 128 |
-
bucket_id: str
|
| 129 |
-
actor_username: str
|
| 130 |
-
diagnostic_path: str
|
| 131 |
-
verified_at: str
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
@dataclass(frozen=True)
|
| 135 |
-
class HfBucketHealthResult:
|
| 136 |
-
bucket_id: str
|
| 137 |
-
project_slug: str
|
| 138 |
-
dataset_repo_id: str
|
| 139 |
-
snapshot_item_count: int
|
| 140 |
-
active_event_count: int
|
| 141 |
-
archive_file_count: int
|
| 142 |
-
latest_event_at: str | None
|
| 143 |
-
|
| 144 |
-
|
| 145 |
class HfBucketValidationInitializer:
|
| 146 |
"""Create a private mutable validation store owned by the project admin."""
|
| 147 |
|
|
@@ -192,183 +173,6 @@ class HfBucketValidationInitializer:
|
|
| 192 |
return HfBucketValidationInitResult(bucket_id=bucket_id, initialized=True, reused_existing=False)
|
| 193 |
|
| 194 |
|
| 195 |
-
class HfBucketPermissionProbe:
|
| 196 |
-
"""Verify a user's private Bucket access without changing validation state."""
|
| 197 |
-
|
| 198 |
-
def __init__(self, api: HfBucketFilesApi | None = None) -> None:
|
| 199 |
-
self._api = api or HuggingFaceBucketFilesApi()
|
| 200 |
-
|
| 201 |
-
def probe(
|
| 202 |
-
self,
|
| 203 |
-
*,
|
| 204 |
-
bucket_id: str,
|
| 205 |
-
actor_username: str,
|
| 206 |
-
token: str | None,
|
| 207 |
-
) -> HfBucketPermissionProbeResult:
|
| 208 |
-
bucket = (bucket_id or "").strip()
|
| 209 |
-
actor = (actor_username or "").strip()
|
| 210 |
-
token_value = (token or "").strip()
|
| 211 |
-
if not bucket or not actor or not token_value:
|
| 212 |
-
raise HfBucketValidationError(
|
| 213 |
-
"Select a project and sign in with your own Hugging Face authorization before testing private storage access."
|
| 214 |
-
)
|
| 215 |
-
|
| 216 |
-
verified_at = datetime.now(UTC).isoformat()
|
| 217 |
-
diagnostic_path = f"diagnostics/access-proof/{uuid4()}.json"
|
| 218 |
-
payload = {
|
| 219 |
-
"schema_version": HF_BUCKET_VALIDATION_SCHEMA_VERSION,
|
| 220 |
-
"diagnostic_type": "private_bucket_access_proof",
|
| 221 |
-
"actor_username": actor,
|
| 222 |
-
"bucket_id": bucket,
|
| 223 |
-
"verified_at": verified_at,
|
| 224 |
-
"contains_project_data": False,
|
| 225 |
-
}
|
| 226 |
-
marker_written = False
|
| 227 |
-
try:
|
| 228 |
-
self._api.read_text(
|
| 229 |
-
bucket_id=bucket,
|
| 230 |
-
path_in_bucket="metadata/project.json",
|
| 231 |
-
token=token_value,
|
| 232 |
-
)
|
| 233 |
-
self._api.write_files(
|
| 234 |
-
bucket_id=bucket,
|
| 235 |
-
token=token_value,
|
| 236 |
-
files={diagnostic_path: _json_bytes(payload)},
|
| 237 |
-
)
|
| 238 |
-
marker_written = True
|
| 239 |
-
written_payload = json.loads(
|
| 240 |
-
self._api.read_text(
|
| 241 |
-
bucket_id=bucket,
|
| 242 |
-
path_in_bucket=diagnostic_path,
|
| 243 |
-
token=token_value,
|
| 244 |
-
)
|
| 245 |
-
)
|
| 246 |
-
if written_payload.get("actor_username") != actor:
|
| 247 |
-
raise HfBucketValidationError("The private storage diagnostic marker could not be verified.")
|
| 248 |
-
except HfBucketValidationError:
|
| 249 |
-
raise
|
| 250 |
-
except Exception as exc:
|
| 251 |
-
raise HfBucketValidationError(
|
| 252 |
-
f"Private Bucket read/write test failed for '{bucket}' using the supplied storage credential: {exc}"
|
| 253 |
-
) from exc
|
| 254 |
-
finally:
|
| 255 |
-
if marker_written:
|
| 256 |
-
try:
|
| 257 |
-
self._api.write_files(
|
| 258 |
-
bucket_id=bucket,
|
| 259 |
-
token=token_value,
|
| 260 |
-
files={},
|
| 261 |
-
delete_paths=[diagnostic_path],
|
| 262 |
-
)
|
| 263 |
-
except Exception as exc:
|
| 264 |
-
raise HfBucketValidationError(
|
| 265 |
-
"Private Bucket access was verified, but the diagnostic marker could not be removed: "
|
| 266 |
-
f"{exc}"
|
| 267 |
-
) from exc
|
| 268 |
-
|
| 269 |
-
return HfBucketPermissionProbeResult(
|
| 270 |
-
bucket_id=bucket,
|
| 271 |
-
actor_username=actor,
|
| 272 |
-
diagnostic_path=diagnostic_path,
|
| 273 |
-
verified_at=verified_at,
|
| 274 |
-
)
|
| 275 |
-
|
| 276 |
-
|
| 277 |
-
class HfBucketHealthInspector:
|
| 278 |
-
"""Read and validate a project's Bucket manifest, snapshot, and audit layout."""
|
| 279 |
-
|
| 280 |
-
def __init__(self, api: HfBucketFilesApi | None = None) -> None:
|
| 281 |
-
self._api = api or HuggingFaceBucketFilesApi()
|
| 282 |
-
|
| 283 |
-
def inspect(
|
| 284 |
-
self,
|
| 285 |
-
*,
|
| 286 |
-
project_slug: str,
|
| 287 |
-
dataset_repo_id: str,
|
| 288 |
-
bucket_id: str,
|
| 289 |
-
token: str | None,
|
| 290 |
-
) -> HfBucketHealthResult:
|
| 291 |
-
project = (project_slug or "").strip()
|
| 292 |
-
dataset = (dataset_repo_id or "").strip()
|
| 293 |
-
bucket = (bucket_id or "").strip()
|
| 294 |
-
token_value = (token or "").strip()
|
| 295 |
-
if not project or not dataset or not bucket or not token_value:
|
| 296 |
-
raise HfBucketValidationError(
|
| 297 |
-
"A project, dataset, validation bucket, and protected storage credential are required for health inspection."
|
| 298 |
-
)
|
| 299 |
-
|
| 300 |
-
try:
|
| 301 |
-
raw_manifest = self._api.read_text(
|
| 302 |
-
bucket_id=bucket,
|
| 303 |
-
path_in_bucket="metadata/project.json",
|
| 304 |
-
token=token_value,
|
| 305 |
-
)
|
| 306 |
-
raw_snapshot = self._api.read_text(
|
| 307 |
-
bucket_id=bucket,
|
| 308 |
-
path_in_bucket="snapshots/current.json",
|
| 309 |
-
token=token_value,
|
| 310 |
-
)
|
| 311 |
-
except Exception as exc:
|
| 312 |
-
raise HfBucketValidationError(f"Could not read Bucket health metadata for '{bucket}': {exc}") from exc
|
| 313 |
-
try:
|
| 314 |
-
manifest = json.loads(raw_manifest)
|
| 315 |
-
snapshot_payload = json.loads(raw_snapshot)
|
| 316 |
-
except json.JSONDecodeError as exc:
|
| 317 |
-
raise HfBucketValidationError(f"Bucket health metadata is not valid JSON for '{bucket}': {exc}") from exc
|
| 318 |
-
|
| 319 |
-
if not isinstance(manifest, dict):
|
| 320 |
-
raise HfBucketValidationError(f"Bucket manifest for '{bucket}' is not a JSON object.")
|
| 321 |
-
expected_manifest = {
|
| 322 |
-
"project_slug": project,
|
| 323 |
-
"dataset_repo_id": dataset,
|
| 324 |
-
"validation_backend": HF_BUCKET_VALIDATION_BACKEND,
|
| 325 |
-
"bucket_id": bucket,
|
| 326 |
-
}
|
| 327 |
-
mismatches = [
|
| 328 |
-
key for key, expected in expected_manifest.items()
|
| 329 |
-
if str(manifest.get(key) or "").strip() != expected
|
| 330 |
-
]
|
| 331 |
-
if mismatches:
|
| 332 |
-
raise HfBucketValidationError(
|
| 333 |
-
f"Bucket manifest for '{bucket}' does not match the selected project fields: {', '.join(mismatches)}."
|
| 334 |
-
)
|
| 335 |
-
|
| 336 |
-
snapshot_items = _extract_items(snapshot_payload, project)
|
| 337 |
-
if snapshot_items is None:
|
| 338 |
-
raise HfBucketValidationError(
|
| 339 |
-
f"Bucket snapshot for '{bucket}' is missing, malformed, or belongs to another project."
|
| 340 |
-
)
|
| 341 |
-
|
| 342 |
-
repository = HfBucketValidationRepository(bucket_id=bucket, token=token_value, api=self._api)
|
| 343 |
-
active_paths = repository._active_event_paths()
|
| 344 |
-
archive_paths = repository._archive_paths()
|
| 345 |
-
current_items = repository.load_current_snapshot(project)
|
| 346 |
-
latest_events = repository.list_recent_events(project, limit=1)
|
| 347 |
-
latest_event_at = (
|
| 348 |
-
str(latest_events[0].get("timestamp") or "").strip() or None
|
| 349 |
-
if latest_events
|
| 350 |
-
else None
|
| 351 |
-
)
|
| 352 |
-
if latest_events:
|
| 353 |
-
latest_event = latest_events[0]
|
| 354 |
-
latest_key = str(latest_event.get("detection_key") or "").strip()
|
| 355 |
-
latest_version = int(latest_event.get("new_version") or 0)
|
| 356 |
-
current_version = int(current_items.get(latest_key, {}).get("version") or 0)
|
| 357 |
-
if latest_key and latest_version > current_version:
|
| 358 |
-
raise HfBucketValidationError(
|
| 359 |
-
f"Bucket snapshot for '{bucket}' does not contain its latest audit event."
|
| 360 |
-
)
|
| 361 |
-
return HfBucketHealthResult(
|
| 362 |
-
bucket_id=bucket,
|
| 363 |
-
project_slug=project,
|
| 364 |
-
dataset_repo_id=dataset,
|
| 365 |
-
snapshot_item_count=len(current_items),
|
| 366 |
-
active_event_count=len(active_paths),
|
| 367 |
-
archive_file_count=len(archive_paths),
|
| 368 |
-
latest_event_at=latest_event_at,
|
| 369 |
-
)
|
| 370 |
-
|
| 371 |
-
|
| 372 |
def _json_bytes(payload: object) -> bytes:
|
| 373 |
return json.dumps(payload, ensure_ascii=True, indent=2, sort_keys=True).encode("utf-8")
|
| 374 |
|
|
|
|
| 123 |
compacted_event_count: int
|
| 124 |
|
| 125 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 126 |
class HfBucketValidationInitializer:
|
| 127 |
"""Create a private mutable validation store owned by the project admin."""
|
| 128 |
|
|
|
|
| 173 |
return HfBucketValidationInitResult(bucket_id=bucket_id, initialized=True, reused_existing=False)
|
| 174 |
|
| 175 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 176 |
def _json_bytes(payload: object) -> bytes:
|
| 177 |
return json.dumps(payload, ensure_ascii=True, indent=2, sort_keys=True).encode("utf-8")
|
| 178 |
|
src/services/hf_project_state_store.py
CHANGED
|
@@ -104,14 +104,6 @@ class HfProjectStateStoreSyncResult:
|
|
| 104 |
synced_paths: list[str]
|
| 105 |
|
| 106 |
|
| 107 |
-
@dataclass(frozen=True)
|
| 108 |
-
class HfProjectStatePermissionProbeResult:
|
| 109 |
-
state_repo_id: str
|
| 110 |
-
actor_username: str
|
| 111 |
-
diagnostic_path: str
|
| 112 |
-
verified_at: str
|
| 113 |
-
|
| 114 |
-
|
| 115 |
@dataclass(frozen=True)
|
| 116 |
class HfProjectStateStoreLoadedProject:
|
| 117 |
state_repo_id: str
|
|
@@ -609,86 +601,6 @@ class HfProjectStateStoreConnector:
|
|
| 609 |
return loaded
|
| 610 |
|
| 611 |
|
| 612 |
-
class HfProjectStatePermissionProbe:
|
| 613 |
-
"""Prove that the acting OAuth identity can read and write a private state repo."""
|
| 614 |
-
|
| 615 |
-
def __init__(
|
| 616 |
-
self,
|
| 617 |
-
*,
|
| 618 |
-
loader: HfProjectStateStoreLoader | None = None,
|
| 619 |
-
api: HfProjectStateApi | None = None,
|
| 620 |
-
) -> None:
|
| 621 |
-
self._loader = loader or HfProjectStateStoreLoader()
|
| 622 |
-
self._api = api or HfApi()
|
| 623 |
-
|
| 624 |
-
def probe(
|
| 625 |
-
self,
|
| 626 |
-
*,
|
| 627 |
-
project: Project,
|
| 628 |
-
actor_username: str,
|
| 629 |
-
token: str | None,
|
| 630 |
-
) -> HfProjectStatePermissionProbeResult:
|
| 631 |
-
actor = (actor_username or "").strip()
|
| 632 |
-
token_value = (token or "").strip()
|
| 633 |
-
state_repo_id = (project.state_repo_id or "").strip()
|
| 634 |
-
if not actor or not token_value:
|
| 635 |
-
raise HfProjectStateStoreError(
|
| 636 |
-
"Sign in with your Hugging Face OAuth account before testing private state authorization."
|
| 637 |
-
)
|
| 638 |
-
if not state_repo_id:
|
| 639 |
-
raise HfProjectStateStoreError("This project has no private `_state` repository to test.")
|
| 640 |
-
|
| 641 |
-
try:
|
| 642 |
-
loaded = self._loader.load_project_state(state_repo_id=state_repo_id, token=token_value)
|
| 643 |
-
except Exception as exc:
|
| 644 |
-
raise HfProjectStateStoreError(
|
| 645 |
-
f"State authorization read failed for {state_repo_id} using the signed-in account: {exc}"
|
| 646 |
-
) from exc
|
| 647 |
-
if loaded is None or loaded.project.project_slug != project.project_slug:
|
| 648 |
-
raise HfProjectStateStoreError("The private state manifest does not match the selected project.")
|
| 649 |
-
|
| 650 |
-
verified_at = datetime.now(UTC).isoformat()
|
| 651 |
-
diagnostic_id = str(uuid4())
|
| 652 |
-
diagnostic_path = f"diagnostics/oauth-permission-proof/{diagnostic_id}.json"
|
| 653 |
-
payload = {
|
| 654 |
-
"schema_version": HF_PROJECT_STATE_SCHEMA_VERSION,
|
| 655 |
-
"diagnostic_type": "oauth_permission_proof",
|
| 656 |
-
"project_slug": project.project_slug,
|
| 657 |
-
"state_repo_id": state_repo_id,
|
| 658 |
-
"actor_username": actor,
|
| 659 |
-
"verified_at": verified_at,
|
| 660 |
-
"diagnostic_id": diagnostic_id,
|
| 661 |
-
}
|
| 662 |
-
try:
|
| 663 |
-
self._api.create_commit(
|
| 664 |
-
repo_id=state_repo_id,
|
| 665 |
-
repo_type="dataset",
|
| 666 |
-
token=token_value,
|
| 667 |
-
commit_message=f"Verify OAuth state access for {actor}",
|
| 668 |
-
operations=[
|
| 669 |
-
CommitOperationAdd(path_in_repo=diagnostic_path, path_or_fileobj=_json_bytes(payload)),
|
| 670 |
-
],
|
| 671 |
-
)
|
| 672 |
-
except Exception as exc:
|
| 673 |
-
message = str(exc).lower()
|
| 674 |
-
if "create_pr=1" in message or "create a pull request" in message:
|
| 675 |
-
raise HfProjectStateStoreError(
|
| 676 |
-
"OAuth access to this private `_state` repository was confirmed only for Pull Request "
|
| 677 |
-
"contributions, not direct durable writes to `main`. This permission model is not sufficient "
|
| 678 |
-
"for high-frequency collaborative validation state without an administrator merge step."
|
| 679 |
-
) from exc
|
| 680 |
-
raise HfProjectStateStoreError(
|
| 681 |
-
f"State authorization write failed for {state_repo_id} using the signed-in account: {exc}"
|
| 682 |
-
) from exc
|
| 683 |
-
|
| 684 |
-
return HfProjectStatePermissionProbeResult(
|
| 685 |
-
state_repo_id=state_repo_id,
|
| 686 |
-
actor_username=actor,
|
| 687 |
-
diagnostic_path=diagnostic_path,
|
| 688 |
-
verified_at=verified_at,
|
| 689 |
-
)
|
| 690 |
-
|
| 691 |
-
|
| 692 |
class HfProjectStateStoreSync:
|
| 693 |
def __init__(self, api: HfProjectStateApi | None = None) -> None:
|
| 694 |
self._api = api or HfApi()
|
|
|
|
| 104 |
synced_paths: list[str]
|
| 105 |
|
| 106 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
@dataclass(frozen=True)
|
| 108 |
class HfProjectStateStoreLoadedProject:
|
| 109 |
state_repo_id: str
|
|
|
|
| 601 |
return loaded
|
| 602 |
|
| 603 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 604 |
class HfProjectStateStoreSync:
|
| 605 |
def __init__(self, api: HfProjectStateApi | None = None) -> None:
|
| 606 |
self._api = api or HfApi()
|
src/ui/app_factory.py
CHANGED
|
@@ -26,8 +26,6 @@ from src.repositories.append_only_validation_repository import AppendOnlyValidat
|
|
| 26 |
from src.repositories.in_memory_detection_repository import InMemoryDetectionRepository
|
| 27 |
from src.repositories.hf_bucket_validation_repository import (
|
| 28 |
HF_BUCKET_VALIDATION_BACKEND,
|
| 29 |
-
HfBucketHealthInspector,
|
| 30 |
-
HfBucketPermissionProbe,
|
| 31 |
HfBucketValidationError,
|
| 32 |
HfBucketValidationInitializer,
|
| 33 |
)
|
|
@@ -48,7 +46,6 @@ from src.services.hf_project_state_store import (
|
|
| 48 |
HfProjectStateStoreInitializer,
|
| 49 |
HfProjectStateStoreLoadedProject,
|
| 50 |
HfProjectStateStoreLoader,
|
| 51 |
-
HfProjectStatePermissionProbe,
|
| 52 |
HfProjectStateStoreSync,
|
| 53 |
)
|
| 54 |
from src.services.validation_service import ValidationService
|
|
@@ -1688,13 +1685,13 @@ def _resolve_username_login_policy(runtime_config: RuntimeConfig) -> tuple[bool,
|
|
| 1688 |
if mode == "username":
|
| 1689 |
return True, "OAuth or username", "OAuth is recommended; username-only login is enabled by explicit development configuration."
|
| 1690 |
if mode == "username_or_token":
|
| 1691 |
-
return True, "OAuth
|
| 1692 |
if mode == "hf_token":
|
| 1693 |
-
return False, "
|
| 1694 |
|
| 1695 |
if runtime_config.enable_demo_bootstrap or not _is_running_in_hf_space():
|
| 1696 |
-
return True, "auto: OAuth
|
| 1697 |
-
return False, "
|
| 1698 |
|
| 1699 |
|
| 1700 |
def _page_to_table(
|
|
@@ -3089,11 +3086,7 @@ def create_app() -> gr.Blocks:
|
|
| 3089 |
validation_service = ValidationService(validation_repository)
|
| 3090 |
hf_state_initializer = HfProjectStateStoreInitializer()
|
| 3091 |
hf_bucket_initializer = HfBucketValidationInitializer()
|
| 3092 |
-
hf_bucket_health_inspector = HfBucketHealthInspector()
|
| 3093 |
-
hf_bucket_permission_probe = HfBucketPermissionProbe()
|
| 3094 |
hf_state_connector = HfProjectStateStoreConnector()
|
| 3095 |
-
hf_state_health_loader = HfProjectStateStoreLoader()
|
| 3096 |
-
hf_state_permission_probe = HfProjectStatePermissionProbe()
|
| 3097 |
hf_state_sync = HfProjectStateStoreSync()
|
| 3098 |
report_cache: dict[tuple[str, int, int, str], tuple[float, tuple[object, ...]]] = {}
|
| 3099 |
report_cache_ttl_seconds = 30.0
|
|
@@ -3372,7 +3365,7 @@ def create_app() -> gr.Blocks:
|
|
| 3372 |
),
|
| 3373 |
)
|
| 3374 |
)
|
| 3375 |
-
|
| 3376 |
auth_service,
|
| 3377 |
allow_username_login=allow_username_login,
|
| 3378 |
enable_oauth_login=_is_running_in_hf_space(),
|
|
@@ -3558,8 +3551,7 @@ def create_app() -> gr.Blocks:
|
|
| 3558 |
):
|
| 3559 |
return (
|
| 3560 |
"Private `_state` test projects must be created through OAuth. Return to Login, complete "
|
| 3561 |
-
"both Hugging Face sign-in steps, and create a new test project
|
| 3562 |
-
"manual token login cannot prove `contribute-repos` authorization.",
|
| 3563 |
_project_rows(),
|
| 3564 |
gr.update(),
|
| 3565 |
gr.update(),
|
|
@@ -4435,37 +4427,6 @@ def create_app() -> gr.Blocks:
|
|
| 4435 |
interactive=False,
|
| 4436 |
allow_custom_value=True,
|
| 4437 |
)
|
| 4438 |
-
with gr.Group(
|
| 4439 |
-
visible=not runtime_config.hf_admin_storage_mode_enabled,
|
| 4440 |
-
elem_classes=["bn-panel-soft"],
|
| 4441 |
-
):
|
| 4442 |
-
gr.Markdown("### Private state OAuth diagnostic")
|
| 4443 |
-
state_authorization_status = gr.Markdown(
|
| 4444 |
-
"The current OAuth scope may allow Pull Requests rather than direct state writes. "
|
| 4445 |
-
"Run this diagnostic only when examining an experimental `_state` project."
|
| 4446 |
-
)
|
| 4447 |
-
test_state_authorization_btn = gr.Button(
|
| 4448 |
-
"Check OAuth state write capability",
|
| 4449 |
-
elem_classes=["bn-soft-action"],
|
| 4450 |
-
)
|
| 4451 |
-
with gr.Group(
|
| 4452 |
-
visible=runtime_config.hf_admin_storage_mode_enabled,
|
| 4453 |
-
elem_classes=["bn-panel-soft"],
|
| 4454 |
-
):
|
| 4455 |
-
gr.Markdown("### Private storage backend health")
|
| 4456 |
-
trusted_storage_health = gr.HTML(
|
| 4457 |
-
value=settings_health_html(
|
| 4458 |
-
[("Project integrity", "not checked", "info")]
|
| 4459 |
-
)
|
| 4460 |
-
)
|
| 4461 |
-
trusted_storage_status = gr.Markdown(
|
| 4462 |
-
"Administrators can verify that this Space can read and write the project's private "
|
| 4463 |
-
"validation storage. Validators do not require direct Hub storage access."
|
| 4464 |
-
)
|
| 4465 |
-
test_trusted_storage_btn = gr.Button(
|
| 4466 |
-
"Check backend storage access",
|
| 4467 |
-
elem_classes=["bn-soft-action"],
|
| 4468 |
-
)
|
| 4469 |
invitations_info = gr.Markdown(value="")
|
| 4470 |
invitations_overview = gr.HTML(value=invite_panel_html(0))
|
| 4471 |
invite_selector = gr.Dropdown(choices=[], label="Pending Invites", interactive=False)
|
|
@@ -4691,124 +4652,12 @@ def create_app() -> gr.Blocks:
|
|
| 4691 |
return selected, dataset_repo_id, project_overview_html(_project_rows(), session.authorized_projects, selected), project_context_html(project_row, role_label)
|
| 4692 |
return None, "", project_overview_html([], []), project_context_html(None)
|
| 4693 |
|
| 4694 |
-
def test_private_state_authorization(selected: str, session):
|
| 4695 |
-
if session is None:
|
| 4696 |
-
return "Login with your Hugging Face account before running the state authorization test."
|
| 4697 |
-
if getattr(session, "authentication_method", "") != "oauth":
|
| 4698 |
-
return (
|
| 4699 |
-
"This authorization proof requires OAuth. Return to Login and complete both steps: "
|
| 4700 |
-
"**1. Sign in with Hugging Face** and **2. Enter workspace with OAuth authorization**. "
|
| 4701 |
-
"Manual token login does not test `contribute-repos`."
|
| 4702 |
-
)
|
| 4703 |
-
project_slug = (selected or "").strip()
|
| 4704 |
-
project = admin_manager.get_project(project_slug) if project_slug else None
|
| 4705 |
-
if project is None or project_slug not in session.authorized_projects:
|
| 4706 |
-
return "Select a project that is authorized for the signed-in account."
|
| 4707 |
-
if (project.state_backend or "").strip() != HF_PROJECT_STATE_BACKEND:
|
| 4708 |
-
return "This project does not use a private Hugging Face `_state` repository."
|
| 4709 |
-
try:
|
| 4710 |
-
result = hf_state_permission_probe.probe(
|
| 4711 |
-
project=project,
|
| 4712 |
-
actor_username=session.username,
|
| 4713 |
-
token=_session_hf_token(session),
|
| 4714 |
-
)
|
| 4715 |
-
except HfProjectStateStoreError as exc:
|
| 4716 |
-
return f"State authorization test failed: {exc}"
|
| 4717 |
-
return (
|
| 4718 |
-
f"State authorization verified for **{result.actor_username}**. "
|
| 4719 |
-
f"A private diagnostic write was confirmed in `{result.state_repo_id}`."
|
| 4720 |
-
)
|
| 4721 |
-
|
| 4722 |
-
def test_trusted_storage_authorization(selected: str, session):
|
| 4723 |
-
def failure(message: str):
|
| 4724 |
-
return settings_health_html([("Project integrity", "check failed", "warn")]), message
|
| 4725 |
-
|
| 4726 |
-
if session is None:
|
| 4727 |
-
return failure("Login before testing private validation storage.")
|
| 4728 |
-
project_slug = (selected or "").strip()
|
| 4729 |
-
project = admin_manager.get_project(project_slug) if project_slug else None
|
| 4730 |
-
if project is None or project_slug not in session.authorized_projects:
|
| 4731 |
-
return failure("Select a project that is authorized for the signed-in account.")
|
| 4732 |
-
if not _is_admin_for_project(session, project_slug):
|
| 4733 |
-
return failure(
|
| 4734 |
-
"Only a project administrator can run this storage health check. "
|
| 4735 |
-
"Validators receive data through the app after assignment."
|
| 4736 |
-
)
|
| 4737 |
-
if (project.validation_backend or "").strip() != HF_BUCKET_VALIDATION_BACKEND:
|
| 4738 |
-
return failure("This project does not use private Hugging Face Bucket validation storage.")
|
| 4739 |
-
try:
|
| 4740 |
-
storage_token = _storage_hf_token()
|
| 4741 |
-
_validate_hf_admin_storage_source_dataset(project=project, token=storage_token)
|
| 4742 |
-
state_repo_id = (project.state_repo_id or "").strip()
|
| 4743 |
-
if not state_repo_id:
|
| 4744 |
-
raise HfProjectStateStoreError("This project does not declare a private `_state` repository.")
|
| 4745 |
-
loaded_state = hf_state_health_loader.load_project_state(
|
| 4746 |
-
state_repo_id=state_repo_id,
|
| 4747 |
-
token=storage_token,
|
| 4748 |
-
)
|
| 4749 |
-
if loaded_state is None:
|
| 4750 |
-
raise HfProjectStateStoreError("The private `_state` manifest is archived.")
|
| 4751 |
-
recovered_project = loaded_state.project
|
| 4752 |
-
if (
|
| 4753 |
-
recovered_project.project_slug != project.project_slug
|
| 4754 |
-
or recovered_project.dataset_repo_id != project.dataset_repo_id
|
| 4755 |
-
or recovered_project.validation_bucket_id != project.validation_bucket_id
|
| 4756 |
-
):
|
| 4757 |
-
raise HfProjectStateStoreError(
|
| 4758 |
-
"The private `_state` manifest does not match the active project's dataset or Bucket."
|
| 4759 |
-
)
|
| 4760 |
-
if loaded_state.user_access.get(session.username, {}).get(project_slug) != Role.admin:
|
| 4761 |
-
raise HfProjectStateStoreError(
|
| 4762 |
-
"The current administrator is not persisted as ADMIN in the private `_state` ACL."
|
| 4763 |
-
)
|
| 4764 |
-
bucket_health = hf_bucket_health_inspector.inspect(
|
| 4765 |
-
project_slug=project.project_slug,
|
| 4766 |
-
dataset_repo_id=project.dataset_repo_id,
|
| 4767 |
-
bucket_id=project.validation_bucket_id or "",
|
| 4768 |
-
token=storage_token,
|
| 4769 |
-
)
|
| 4770 |
-
result = hf_bucket_permission_probe.probe(
|
| 4771 |
-
bucket_id=project.validation_bucket_id or "",
|
| 4772 |
-
actor_username=session.username,
|
| 4773 |
-
token=storage_token,
|
| 4774 |
-
)
|
| 4775 |
-
except (HfBucketValidationError, HfProjectStateStoreError) as exc:
|
| 4776 |
-
return failure(f"Private storage integrity check failed: {exc}")
|
| 4777 |
-
health_html = settings_health_html(
|
| 4778 |
-
[
|
| 4779 |
-
("Audio dataset", "private and accessible", "ok"),
|
| 4780 |
-
("Project state", state_repo_id, "ok"),
|
| 4781 |
-
("Validation Bucket", bucket_health.bucket_id, "ok"),
|
| 4782 |
-
("Validated snapshot items", str(bucket_health.snapshot_item_count), "ok"),
|
| 4783 |
-
("Active audit events", str(bucket_health.active_event_count), "info"),
|
| 4784 |
-
("Archived audit batches", str(bucket_health.archive_file_count), "info"),
|
| 4785 |
-
("Latest validation event", bucket_health.latest_event_at or "none yet", "info"),
|
| 4786 |
-
("Backend write proof", "verified and removed", "ok"),
|
| 4787 |
-
]
|
| 4788 |
-
)
|
| 4789 |
-
return health_html, (
|
| 4790 |
-
f"Backend storage verified for **{result.actor_username}**. "
|
| 4791 |
-
f"The Space read and wrote `{result.bucket_id}` using its protected storage credential; "
|
| 4792 |
-
"the diagnostic marker was removed."
|
| 4793 |
-
)
|
| 4794 |
-
|
| 4795 |
project_selector.change(
|
| 4796 |
fn=update_selected_project,
|
| 4797 |
inputs=[project_selector, session_state],
|
| 4798 |
outputs=[selected_project_state, selected_dataset_repo_state, project_overview, project_context_display],
|
| 4799 |
)
|
| 4800 |
|
| 4801 |
-
test_state_authorization_btn.click(
|
| 4802 |
-
fn=test_private_state_authorization,
|
| 4803 |
-
inputs=[project_selector, session_state],
|
| 4804 |
-
outputs=[state_authorization_status],
|
| 4805 |
-
)
|
| 4806 |
-
test_trusted_storage_btn.click(
|
| 4807 |
-
fn=test_trusted_storage_authorization,
|
| 4808 |
-
inputs=[project_selector, session_state],
|
| 4809 |
-
outputs=[trusted_storage_health, trusted_storage_status],
|
| 4810 |
-
)
|
| 4811 |
-
|
| 4812 |
create_project_event.then(
|
| 4813 |
fn=update_project_selector,
|
| 4814 |
inputs=[session_state],
|
|
|
|
| 26 |
from src.repositories.in_memory_detection_repository import InMemoryDetectionRepository
|
| 27 |
from src.repositories.hf_bucket_validation_repository import (
|
| 28 |
HF_BUCKET_VALIDATION_BACKEND,
|
|
|
|
|
|
|
| 29 |
HfBucketValidationError,
|
| 30 |
HfBucketValidationInitializer,
|
| 31 |
)
|
|
|
|
| 46 |
HfProjectStateStoreInitializer,
|
| 47 |
HfProjectStateStoreLoadedProject,
|
| 48 |
HfProjectStateStoreLoader,
|
|
|
|
| 49 |
HfProjectStateStoreSync,
|
| 50 |
)
|
| 51 |
from src.services.validation_service import ValidationService
|
|
|
|
| 1685 |
if mode == "username":
|
| 1686 |
return True, "OAuth or username", "OAuth is recommended; username-only login is enabled by explicit development configuration."
|
| 1687 |
if mode == "username_or_token":
|
| 1688 |
+
return True, "OAuth or username", "OAuth is recommended; username fallback is enabled for local compatibility."
|
| 1689 |
if mode == "hf_token":
|
| 1690 |
+
return False, "Hugging Face OAuth", "Production identity mode: sign in securely with Hugging Face OAuth."
|
| 1691 |
|
| 1692 |
if runtime_config.enable_demo_bootstrap or not _is_running_in_hf_space():
|
| 1693 |
+
return True, "auto: OAuth or local username", "Development/demo identity mode: OAuth is preferred and local username login remains available."
|
| 1694 |
+
return False, "Hugging Face OAuth", "Production identity mode: sign in securely with Hugging Face OAuth."
|
| 1695 |
|
| 1696 |
|
| 1697 |
def _page_to_table(
|
|
|
|
| 3086 |
validation_service = ValidationService(validation_repository)
|
| 3087 |
hf_state_initializer = HfProjectStateStoreInitializer()
|
| 3088 |
hf_bucket_initializer = HfBucketValidationInitializer()
|
|
|
|
|
|
|
| 3089 |
hf_state_connector = HfProjectStateStoreConnector()
|
|
|
|
|
|
|
| 3090 |
hf_state_sync = HfProjectStateStoreSync()
|
| 3091 |
report_cache: dict[tuple[str, int, int, str], tuple[float, tuple[object, ...]]] = {}
|
| 3092 |
report_cache_ttl_seconds = 30.0
|
|
|
|
| 3365 |
),
|
| 3366 |
)
|
| 3367 |
)
|
| 3368 |
+
session_output, error_message = create_login_page(
|
| 3369 |
auth_service,
|
| 3370 |
allow_username_login=allow_username_login,
|
| 3371 |
enable_oauth_login=_is_running_in_hf_space(),
|
|
|
|
| 3551 |
):
|
| 3552 |
return (
|
| 3553 |
"Private `_state` test projects must be created through OAuth. Return to Login, complete "
|
| 3554 |
+
"both Hugging Face sign-in steps, and create a new test project.",
|
|
|
|
| 3555 |
_project_rows(),
|
| 3556 |
gr.update(),
|
| 3557 |
gr.update(),
|
|
|
|
| 4427 |
interactive=False,
|
| 4428 |
allow_custom_value=True,
|
| 4429 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4430 |
invitations_info = gr.Markdown(value="")
|
| 4431 |
invitations_overview = gr.HTML(value=invite_panel_html(0))
|
| 4432 |
invite_selector = gr.Dropdown(choices=[], label="Pending Invites", interactive=False)
|
|
|
|
| 4652 |
return selected, dataset_repo_id, project_overview_html(_project_rows(), session.authorized_projects, selected), project_context_html(project_row, role_label)
|
| 4653 |
return None, "", project_overview_html([], []), project_context_html(None)
|
| 4654 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4655 |
project_selector.change(
|
| 4656 |
fn=update_selected_project,
|
| 4657 |
inputs=[project_selector, session_state],
|
| 4658 |
outputs=[selected_project_state, selected_dataset_repo_state, project_overview, project_context_display],
|
| 4659 |
)
|
| 4660 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4661 |
create_project_event.then(
|
| 4662 |
fn=update_project_selector,
|
| 4663 |
inputs=[session_state],
|
src/ui/login_page.py
CHANGED
|
@@ -10,25 +10,18 @@ from src.auth.auth_service import AuthService
|
|
| 10 |
def perform_login(
|
| 11 |
auth_service: AuthService,
|
| 12 |
username: str,
|
| 13 |
-
hf_token: str,
|
| 14 |
*,
|
| 15 |
allow_username_login: bool = True,
|
| 16 |
) -> Tuple[str, str]:
|
| 17 |
-
"""Attempt login and return session
|
| 18 |
-
if hf_token and hf_token.strip():
|
| 19 |
-
session, message = auth_service.login_with_hf_token(hf_token)
|
| 20 |
-
if session is None:
|
| 21 |
-
return "", message
|
| 22 |
-
return session.session_id, message
|
| 23 |
-
|
| 24 |
if not allow_username_login:
|
| 25 |
return (
|
| 26 |
"",
|
| 27 |
-
"Username-only login is disabled for this deployment. Sign in with
|
| 28 |
)
|
| 29 |
|
| 30 |
if not username or not username.strip():
|
| 31 |
-
return "", "Please enter a username
|
| 32 |
|
| 33 |
username = username.strip()
|
| 34 |
session = auth_service.login(username)
|
|
@@ -85,14 +78,14 @@ def create_login_page(
|
|
| 85 |
enable_oauth_login: bool = False,
|
| 86 |
auth_mode_label: str = "",
|
| 87 |
admin_storage_mode: bool = False,
|
| 88 |
-
) -> Tuple[gr.Textbox, gr.
|
| 89 |
-
"""Create
|
| 90 |
|
| 91 |
Args:
|
| 92 |
auth_service: AuthService instance for login validation
|
| 93 |
|
| 94 |
Returns:
|
| 95 |
-
Tuple of (
|
| 96 |
"""
|
| 97 |
with gr.Row():
|
| 98 |
with gr.Column(scale=1):
|
|
@@ -109,82 +102,48 @@ def create_login_page(
|
|
| 109 |
if auth_mode_label:
|
| 110 |
gr.Markdown(auth_mode_label)
|
| 111 |
|
| 112 |
-
if admin_storage_mode:
|
| 113 |
-
gr.Markdown(
|
| 114 |
-
"### Identity verification\n"
|
| 115 |
-
"Use a token from your own account only to prove your identity. "
|
| 116 |
-
"You do not need direct Hugging Face access to the project's private dataset or validation storage."
|
| 117 |
-
)
|
| 118 |
-
else:
|
| 119 |
-
gr.Markdown(
|
| 120 |
-
"Manual token access is available for development or legacy access, "
|
| 121 |
-
"but cannot be used for the private collaborative state authorization test."
|
| 122 |
-
)
|
| 123 |
-
username_input = gr.Textbox(
|
| 124 |
-
label="Username",
|
| 125 |
-
placeholder="Enter your username" if allow_username_login else "Username login disabled in this deployment",
|
| 126 |
-
lines=1,
|
| 127 |
-
interactive=allow_username_login,
|
| 128 |
-
visible=allow_username_login,
|
| 129 |
-
)
|
| 130 |
-
hf_token_input = gr.Textbox(
|
| 131 |
-
label=(
|
| 132 |
-
"Personal Hugging Face Token (identity verification)"
|
| 133 |
-
if admin_storage_mode
|
| 134 |
-
else "Hugging Face Token (legacy/development only)"
|
| 135 |
-
),
|
| 136 |
-
placeholder="hf_xxx...",
|
| 137 |
-
type="password",
|
| 138 |
-
lines=1,
|
| 139 |
-
)
|
| 140 |
-
|
| 141 |
error_message = gr.Markdown()
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
scale=1,
|
| 147 |
)
|
| 148 |
|
| 149 |
oauth_continue_button = None
|
| 150 |
if enable_oauth_login:
|
| 151 |
gr.Markdown(
|
| 152 |
(
|
| 153 |
-
"
|
| 154 |
-
"The hosted Space can verify your Hugging Face identity without requiring a personal token. "
|
| 155 |
"Project access remains controlled by assignments and invitations inside this app."
|
| 156 |
if admin_storage_mode
|
| 157 |
-
else
|
| 158 |
-
"**OAuth access required for private collaborative state.** "
|
| 159 |
-
"Complete both steps below; signing into Hugging Face alone does not open an app session."
|
| 160 |
-
)
|
| 161 |
)
|
| 162 |
)
|
| 163 |
gr.LoginButton("1. Sign in with Hugging Face", logout_value="Sign out ({})")
|
| 164 |
oauth_continue_button = gr.Button(
|
| 165 |
-
"2. Enter workspace
|
| 166 |
variant="primary",
|
| 167 |
)
|
| 168 |
-
|
| 169 |
-
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 174 |
with gr.Column(scale=1):
|
| 175 |
gr.Markdown("")
|
| 176 |
|
| 177 |
-
login_button.click(
|
| 178 |
-
fn=lambda username, hf_token: perform_login(
|
| 179 |
-
auth_service,
|
| 180 |
-
username,
|
| 181 |
-
hf_token,
|
| 182 |
-
allow_username_login=allow_username_login,
|
| 183 |
-
),
|
| 184 |
-
inputs=[username_input, hf_token_input],
|
| 185 |
-
outputs=[session_output, error_message],
|
| 186 |
-
)
|
| 187 |
-
|
| 188 |
def handle_oauth_login(
|
| 189 |
profile: gr.OAuthProfile | None,
|
| 190 |
oauth_token: gr.OAuthToken | None,
|
|
@@ -198,4 +157,4 @@ def create_login_page(
|
|
| 198 |
outputs=[session_output, error_message],
|
| 199 |
)
|
| 200 |
|
| 201 |
-
return
|
|
|
|
| 10 |
def perform_login(
|
| 11 |
auth_service: AuthService,
|
| 12 |
username: str,
|
|
|
|
| 13 |
*,
|
| 14 |
allow_username_login: bool = True,
|
| 15 |
) -> Tuple[str, str]:
|
| 16 |
+
"""Attempt a development-only username login and return its session status."""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
if not allow_username_login:
|
| 18 |
return (
|
| 19 |
"",
|
| 20 |
+
"Username-only login is disabled for this deployment. Sign in with Hugging Face OAuth.",
|
| 21 |
)
|
| 22 |
|
| 23 |
if not username or not username.strip():
|
| 24 |
+
return "", "Please enter a username"
|
| 25 |
|
| 26 |
username = username.strip()
|
| 27 |
session = auth_service.login(username)
|
|
|
|
| 78 |
enable_oauth_login: bool = False,
|
| 79 |
auth_mode_label: str = "",
|
| 80 |
admin_storage_mode: bool = False,
|
| 81 |
+
) -> Tuple[gr.Textbox, gr.Markdown]:
|
| 82 |
+
"""Create an OAuth production login with an optional local username fallback.
|
| 83 |
|
| 84 |
Args:
|
| 85 |
auth_service: AuthService instance for login validation
|
| 86 |
|
| 87 |
Returns:
|
| 88 |
+
Tuple of (session_output, error_message)
|
| 89 |
"""
|
| 90 |
with gr.Row():
|
| 91 |
with gr.Column(scale=1):
|
|
|
|
| 102 |
if auth_mode_label:
|
| 103 |
gr.Markdown(auth_mode_label)
|
| 104 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
error_message = gr.Markdown()
|
| 106 |
+
session_output = gr.Textbox(
|
| 107 |
+
label="Session ID",
|
| 108 |
+
interactive=False,
|
| 109 |
+
visible=False,
|
|
|
|
| 110 |
)
|
| 111 |
|
| 112 |
oauth_continue_button = None
|
| 113 |
if enable_oauth_login:
|
| 114 |
gr.Markdown(
|
| 115 |
(
|
| 116 |
+
"The hosted Space verifies your Hugging Face identity securely. "
|
|
|
|
| 117 |
"Project access remains controlled by assignments and invitations inside this app."
|
| 118 |
if admin_storage_mode
|
| 119 |
+
else "Sign in securely with your Hugging Face identity to enter the workspace."
|
|
|
|
|
|
|
|
|
|
| 120 |
)
|
| 121 |
)
|
| 122 |
gr.LoginButton("1. Sign in with Hugging Face", logout_value="Sign out ({})")
|
| 123 |
oauth_continue_button = gr.Button(
|
| 124 |
+
"2. Enter workspace",
|
| 125 |
variant="primary",
|
| 126 |
)
|
| 127 |
+
elif allow_username_login:
|
| 128 |
+
gr.Markdown("Local development login")
|
| 129 |
+
username_input = gr.Textbox(
|
| 130 |
+
label="Username",
|
| 131 |
+
placeholder="Enter your username",
|
| 132 |
+
lines=1,
|
| 133 |
+
)
|
| 134 |
+
login_button = gr.Button("Login", variant="primary", scale=1)
|
| 135 |
+
login_button.click(
|
| 136 |
+
fn=lambda username: perform_login(
|
| 137 |
+
auth_service,
|
| 138 |
+
username,
|
| 139 |
+
allow_username_login=True,
|
| 140 |
+
),
|
| 141 |
+
inputs=[username_input],
|
| 142 |
+
outputs=[session_output, error_message],
|
| 143 |
+
)
|
| 144 |
with gr.Column(scale=1):
|
| 145 |
gr.Markdown("")
|
| 146 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
def handle_oauth_login(
|
| 148 |
profile: gr.OAuthProfile | None,
|
| 149 |
oauth_token: gr.OAuthToken | None,
|
|
|
|
| 157 |
outputs=[session_output, error_message],
|
| 158 |
)
|
| 159 |
|
| 160 |
+
return session_output, error_message
|
tests/unit/test_app_factory_audio_helpers.py
CHANGED
|
@@ -1571,7 +1571,7 @@ def test_bootstrap_auth_and_projects_recovers_emergency_admin_when_missing(tmp_p
|
|
| 1571 |
assert emergency_session.role.value == "admin"
|
| 1572 |
|
| 1573 |
|
| 1574 |
-
def
|
| 1575 |
monkeypatch.setenv("SPACE_ID", "owner/space")
|
| 1576 |
runtime_config = RuntimeConfig(
|
| 1577 |
detection_seed_path=None,
|
|
@@ -1591,8 +1591,8 @@ def test_auto_auth_policy_requires_hf_token_in_space_without_demo(monkeypatch: p
|
|
| 1591 |
allow_username, label, description = _resolve_username_login_policy(runtime_config)
|
| 1592 |
|
| 1593 |
assert allow_username is False
|
| 1594 |
-
assert "
|
| 1595 |
-
assert "
|
| 1596 |
|
| 1597 |
|
| 1598 |
def test_auto_auth_policy_allows_username_for_demo(tmp_path: Path) -> None:
|
|
|
|
| 1571 |
assert emergency_session.role.value == "admin"
|
| 1572 |
|
| 1573 |
|
| 1574 |
+
def test_auto_auth_policy_requires_oauth_in_space_without_demo(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
|
| 1575 |
monkeypatch.setenv("SPACE_ID", "owner/space")
|
| 1576 |
runtime_config = RuntimeConfig(
|
| 1577 |
detection_seed_path=None,
|
|
|
|
| 1591 |
allow_username, label, description = _resolve_username_login_policy(runtime_config)
|
| 1592 |
|
| 1593 |
assert allow_username is False
|
| 1594 |
+
assert "OAuth" in label
|
| 1595 |
+
assert "OAuth" in description
|
| 1596 |
|
| 1597 |
|
| 1598 |
def test_auto_auth_policy_allows_username_for_demo(tmp_path: Path) -> None:
|
tests/unit/test_auth_service.py
CHANGED
|
@@ -198,23 +198,6 @@ class TestAuthService:
|
|
| 198 |
|
| 199 |
assert auth_service.get_session(session.session_id) is None
|
| 200 |
|
| 201 |
-
def test_login_with_hf_token_stores_user_token_and_email(self, monkeypatch):
|
| 202 |
-
class FakeApi:
|
| 203 |
-
def whoami(self, token: str):
|
| 204 |
-
assert token == "hf_test_token"
|
| 205 |
-
return {"name": "hf_user", "email": "hf_user@example.org"}
|
| 206 |
-
|
| 207 |
-
monkeypatch.setattr("src.auth.auth_service.HfApi", lambda: FakeApi())
|
| 208 |
-
|
| 209 |
-
service = AuthService()
|
| 210 |
-
session, message = service.login_with_hf_token("hf_test_token")
|
| 211 |
-
|
| 212 |
-
assert session is not None
|
| 213 |
-
assert "Welcome" in message
|
| 214 |
-
assert session.authentication_method == "hf_token"
|
| 215 |
-
assert service.get_hf_token_for_user("hf_user") == "hf_test_token"
|
| 216 |
-
assert service.get_known_email_for_user("hf_user") == "hf_user@example.org"
|
| 217 |
-
|
| 218 |
def test_login_with_verified_hf_identity_uses_oauth_identity_without_whoami(self):
|
| 219 |
service = AuthService()
|
| 220 |
|
|
|
|
| 198 |
|
| 199 |
assert auth_service.get_session(session.session_id) is None
|
| 200 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 201 |
def test_login_with_verified_hf_identity_uses_oauth_identity_without_whoami(self):
|
| 202 |
service = AuthService()
|
| 203 |
|
tests/unit/test_hf_bucket_validation_repository.py
CHANGED
|
@@ -6,8 +6,6 @@ from src.domain.models import Validation
|
|
| 6 |
from src.repositories.append_only_validation_repository import OptimisticLockError
|
| 7 |
from src.repositories.hf_bucket_validation_repository import (
|
| 8 |
HF_BUCKET_VALIDATION_BACKEND,
|
| 9 |
-
HfBucketHealthInspector,
|
| 10 |
-
HfBucketPermissionProbe,
|
| 11 |
HfBucketValidationError,
|
| 12 |
HfBucketValidationInitializer,
|
| 13 |
HfBucketValidationRepository,
|
|
@@ -101,164 +99,6 @@ def test_initializer_reuses_manifest_but_refuses_unknown_existing_data() -> None
|
|
| 101 |
unsafe.initialize(project_slug="project-a", dataset_repo_id="owner/audio", token="hf_admin")
|
| 102 |
|
| 103 |
|
| 104 |
-
def test_bucket_permission_probe_reads_writes_verifies_and_removes_marker() -> None:
|
| 105 |
-
api = FakeBucketApi({"metadata/project.json": "{}"})
|
| 106 |
-
probe = HfBucketPermissionProbe(api=api)
|
| 107 |
-
|
| 108 |
-
result = probe.probe(
|
| 109 |
-
bucket_id="owner/audio_validation_state",
|
| 110 |
-
actor_username="validator-a",
|
| 111 |
-
token="hf_validator",
|
| 112 |
-
)
|
| 113 |
-
|
| 114 |
-
assert result.bucket_id == "owner/audio_validation_state"
|
| 115 |
-
assert result.actor_username == "validator-a"
|
| 116 |
-
assert result.diagnostic_path not in api.files
|
| 117 |
-
assert api.deletes[-1] == [result.diagnostic_path]
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
def test_bucket_health_inspector_validates_manifest_snapshot_and_event_layout() -> None:
|
| 121 |
-
api = FakeBucketApi(
|
| 122 |
-
{
|
| 123 |
-
"metadata/project.json": json.dumps(
|
| 124 |
-
{
|
| 125 |
-
"project_slug": "project-a",
|
| 126 |
-
"dataset_repo_id": "owner/audio",
|
| 127 |
-
"validation_backend": "hf_bucket",
|
| 128 |
-
"bucket_id": "owner/audio_validation_state",
|
| 129 |
-
}
|
| 130 |
-
),
|
| 131 |
-
"snapshots/current.json": json.dumps(
|
| 132 |
-
{
|
| 133 |
-
"project_slug": "project-a",
|
| 134 |
-
"items": {
|
| 135 |
-
"audio-a-0000000001": {"status": "positive", "version": 1},
|
| 136 |
-
},
|
| 137 |
-
}
|
| 138 |
-
),
|
| 139 |
-
"events/20260524/a.json": json.dumps(
|
| 140 |
-
{
|
| 141 |
-
"event_id": "active-a",
|
| 142 |
-
"project_slug": "project-a",
|
| 143 |
-
"detection_key": "audio-a-0000000001",
|
| 144 |
-
"status": "positive",
|
| 145 |
-
"validator": "validator-a",
|
| 146 |
-
"timestamp": "2026-05-24T12:00:00+00:00",
|
| 147 |
-
"new_version": 1,
|
| 148 |
-
}
|
| 149 |
-
),
|
| 150 |
-
"archives/events/20260523.jsonl": json.dumps(
|
| 151 |
-
{
|
| 152 |
-
"event_id": "archive-a",
|
| 153 |
-
"project_slug": "project-a",
|
| 154 |
-
"detection_key": "audio-b-0000000001",
|
| 155 |
-
"status": "negative",
|
| 156 |
-
"validator": "validator-b",
|
| 157 |
-
"timestamp": "2026-05-23T12:00:00+00:00",
|
| 158 |
-
"new_version": 1,
|
| 159 |
-
}
|
| 160 |
-
)
|
| 161 |
-
+ "\n",
|
| 162 |
-
}
|
| 163 |
-
)
|
| 164 |
-
|
| 165 |
-
result = HfBucketHealthInspector(api=api).inspect(
|
| 166 |
-
project_slug="project-a",
|
| 167 |
-
dataset_repo_id="owner/audio",
|
| 168 |
-
bucket_id="owner/audio_validation_state",
|
| 169 |
-
token="hf_admin",
|
| 170 |
-
)
|
| 171 |
-
|
| 172 |
-
assert result.snapshot_item_count == 1
|
| 173 |
-
assert result.active_event_count == 1
|
| 174 |
-
assert result.archive_file_count == 1
|
| 175 |
-
assert result.latest_event_at == "2026-05-24T12:00:00+00:00"
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
def test_bucket_health_inspector_rejects_manifest_for_another_project() -> None:
|
| 179 |
-
api = FakeBucketApi(
|
| 180 |
-
{
|
| 181 |
-
"metadata/project.json": json.dumps(
|
| 182 |
-
{
|
| 183 |
-
"project_slug": "another-project",
|
| 184 |
-
"dataset_repo_id": "owner/audio",
|
| 185 |
-
"validation_backend": "hf_bucket",
|
| 186 |
-
"bucket_id": "owner/audio_validation_state",
|
| 187 |
-
}
|
| 188 |
-
),
|
| 189 |
-
"snapshots/current.json": json.dumps({"project_slug": "project-a", "items": {}}),
|
| 190 |
-
}
|
| 191 |
-
)
|
| 192 |
-
|
| 193 |
-
with pytest.raises(HfBucketValidationError, match="does not match"):
|
| 194 |
-
HfBucketHealthInspector(api=api).inspect(
|
| 195 |
-
project_slug="project-a",
|
| 196 |
-
dataset_repo_id="owner/audio",
|
| 197 |
-
bucket_id="owner/audio_validation_state",
|
| 198 |
-
token="hf_admin",
|
| 199 |
-
)
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
def test_bucket_health_inspector_rejects_invalid_snapshot() -> None:
|
| 203 |
-
api = FakeBucketApi(
|
| 204 |
-
{
|
| 205 |
-
"metadata/project.json": json.dumps(
|
| 206 |
-
{
|
| 207 |
-
"project_slug": "project-a",
|
| 208 |
-
"dataset_repo_id": "owner/audio",
|
| 209 |
-
"validation_backend": "hf_bucket",
|
| 210 |
-
"bucket_id": "owner/audio_validation_state",
|
| 211 |
-
}
|
| 212 |
-
),
|
| 213 |
-
"snapshots/current.json": json.dumps({"project_slug": "another-project", "items": {}}),
|
| 214 |
-
}
|
| 215 |
-
)
|
| 216 |
-
|
| 217 |
-
with pytest.raises(HfBucketValidationError, match="snapshot"):
|
| 218 |
-
HfBucketHealthInspector(api=api).inspect(
|
| 219 |
-
project_slug="project-a",
|
| 220 |
-
dataset_repo_id="owner/audio",
|
| 221 |
-
bucket_id="owner/audio_validation_state",
|
| 222 |
-
token="hf_admin",
|
| 223 |
-
)
|
| 224 |
-
|
| 225 |
-
|
| 226 |
-
def test_bucket_health_inspector_rejects_snapshot_missing_latest_archived_event() -> None:
|
| 227 |
-
api = FakeBucketApi(
|
| 228 |
-
{
|
| 229 |
-
"metadata/project.json": json.dumps(
|
| 230 |
-
{
|
| 231 |
-
"project_slug": "project-a",
|
| 232 |
-
"dataset_repo_id": "owner/audio",
|
| 233 |
-
"validation_backend": "hf_bucket",
|
| 234 |
-
"bucket_id": "owner/audio_validation_state",
|
| 235 |
-
}
|
| 236 |
-
),
|
| 237 |
-
"snapshots/current.json": json.dumps({"project_slug": "project-a", "items": {}}),
|
| 238 |
-
"archives/events/20260524.jsonl": json.dumps(
|
| 239 |
-
{
|
| 240 |
-
"event_id": "archive-a",
|
| 241 |
-
"project_slug": "project-a",
|
| 242 |
-
"detection_key": "audio-a-0000000001",
|
| 243 |
-
"status": "positive",
|
| 244 |
-
"validator": "validator-a",
|
| 245 |
-
"timestamp": "2026-05-24T12:00:00+00:00",
|
| 246 |
-
"new_version": 1,
|
| 247 |
-
}
|
| 248 |
-
)
|
| 249 |
-
+ "\n",
|
| 250 |
-
}
|
| 251 |
-
)
|
| 252 |
-
|
| 253 |
-
with pytest.raises(HfBucketValidationError, match="latest audit event"):
|
| 254 |
-
HfBucketHealthInspector(api=api).inspect(
|
| 255 |
-
project_slug="project-a",
|
| 256 |
-
dataset_repo_id="owner/audio",
|
| 257 |
-
bucket_id="owner/audio_validation_state",
|
| 258 |
-
token="hf_admin",
|
| 259 |
-
)
|
| 260 |
-
|
| 261 |
-
|
| 262 |
def test_save_validation_writes_event_and_snapshot_without_repo_commit() -> None:
|
| 263 |
api = FakeBucketApi({"snapshots/current.json": json.dumps({"project_slug": "project-a", "items": {}})})
|
| 264 |
repository = HfBucketValidationRepository(bucket_id="owner/audio_validation_state", token="hf_user", api=api)
|
|
|
|
| 6 |
from src.repositories.append_only_validation_repository import OptimisticLockError
|
| 7 |
from src.repositories.hf_bucket_validation_repository import (
|
| 8 |
HF_BUCKET_VALIDATION_BACKEND,
|
|
|
|
|
|
|
| 9 |
HfBucketValidationError,
|
| 10 |
HfBucketValidationInitializer,
|
| 11 |
HfBucketValidationRepository,
|
|
|
|
| 99 |
unsafe.initialize(project_slug="project-a", dataset_repo_id="owner/audio", token="hf_admin")
|
| 100 |
|
| 101 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 102 |
def test_save_validation_writes_event_and_snapshot_without_repo_commit() -> None:
|
| 103 |
api = FakeBucketApi({"snapshots/current.json": json.dumps({"project_slug": "project-a", "items": {}})})
|
| 104 |
repository = HfBucketValidationRepository(bucket_id="owner/audio_validation_state", token="hf_user", api=api)
|
tests/unit/test_hf_project_state_store.py
CHANGED
|
@@ -8,8 +8,6 @@ from src.services.hf_project_state_store import (
|
|
| 8 |
HfProjectStateStoreError,
|
| 9 |
HfProjectStateStoreConnector,
|
| 10 |
HfProjectStateStoreInitializer,
|
| 11 |
-
HfProjectStatePermissionProbe,
|
| 12 |
-
HfProjectStateStoreLoadedProject,
|
| 13 |
HfProjectStateStoreLoader,
|
| 14 |
HfProjectStateStoreSync,
|
| 15 |
default_state_repo_id,
|
|
@@ -50,12 +48,6 @@ class FakeHfProjectStateReadApi:
|
|
| 50 |
return self.files[path]
|
| 51 |
|
| 52 |
|
| 53 |
-
class PullRequestOnlyHfProjectStateApi(FakeHfProjectStateApi):
|
| 54 |
-
def create_commit(self, **kwargs): # noqa: ANN001
|
| 55 |
-
_ = kwargs
|
| 56 |
-
raise RuntimeError("403 Forbidden: pass create_pr=1 as a query parameter to create a Pull Request.")
|
| 57 |
-
|
| 58 |
-
|
| 59 |
def _project() -> Project:
|
| 60 |
return Project(
|
| 61 |
project_slug="upload-test2",
|
|
@@ -264,64 +256,6 @@ def test_sync_project_state_requires_token() -> None:
|
|
| 264 |
)
|
| 265 |
|
| 266 |
|
| 267 |
-
def test_permission_probe_writes_isolated_diagnostic_with_acting_token_only() -> None:
|
| 268 |
-
fake_api = FakeHfProjectStateApi()
|
| 269 |
-
project = _project()
|
| 270 |
-
project.state_repo_id = "jrrribeiro/upload_test2_state"
|
| 271 |
-
|
| 272 |
-
class FakeLoader:
|
| 273 |
-
def load_project_state(self, *, state_repo_id: str, token: str): # noqa: ANN001
|
| 274 |
-
assert state_repo_id == project.state_repo_id
|
| 275 |
-
assert token == "hf_validator"
|
| 276 |
-
return HfProjectStateStoreLoadedProject(
|
| 277 |
-
state_repo_id=state_repo_id,
|
| 278 |
-
project=project,
|
| 279 |
-
user_access={},
|
| 280 |
-
pending_invites={},
|
| 281 |
-
)
|
| 282 |
-
|
| 283 |
-
probe = HfProjectStatePermissionProbe(loader=FakeLoader(), api=fake_api) # type: ignore[arg-type]
|
| 284 |
-
result = probe.probe(project=project, actor_username="validator-a", token="hf_validator")
|
| 285 |
-
|
| 286 |
-
assert result.actor_username == "validator-a"
|
| 287 |
-
assert result.diagnostic_path.startswith("diagnostics/oauth-permission-proof/")
|
| 288 |
-
commit = fake_api.commits[0]
|
| 289 |
-
assert commit["token"] == "hf_validator"
|
| 290 |
-
assert commit["operations"][0].path_in_repo == result.diagnostic_path
|
| 291 |
-
serialized = commit["operations"][0].path_or_fileobj.decode("utf-8")
|
| 292 |
-
assert "validator-a" in serialized
|
| 293 |
-
assert "hf_validator" not in serialized
|
| 294 |
-
|
| 295 |
-
|
| 296 |
-
def test_permission_probe_requires_personal_token() -> None:
|
| 297 |
-
project = _project()
|
| 298 |
-
project.state_repo_id = "jrrribeiro/upload_test2_state"
|
| 299 |
-
probe = HfProjectStatePermissionProbe(api=FakeHfProjectStateApi())
|
| 300 |
-
|
| 301 |
-
with pytest.raises(HfProjectStateStoreError, match="OAuth"):
|
| 302 |
-
probe.probe(project=project, actor_username="validator-a", token="")
|
| 303 |
-
|
| 304 |
-
|
| 305 |
-
def test_permission_probe_explains_pull_request_only_oauth_access() -> None:
|
| 306 |
-
project = _project()
|
| 307 |
-
project.state_repo_id = "jrrribeiro/upload_test2_state"
|
| 308 |
-
|
| 309 |
-
class FakeLoader:
|
| 310 |
-
def load_project_state(self, *, state_repo_id: str, token: str): # noqa: ANN001
|
| 311 |
-
_ = token
|
| 312 |
-
return HfProjectStateStoreLoadedProject(
|
| 313 |
-
state_repo_id=state_repo_id,
|
| 314 |
-
project=project,
|
| 315 |
-
user_access={},
|
| 316 |
-
pending_invites={},
|
| 317 |
-
)
|
| 318 |
-
|
| 319 |
-
probe = HfProjectStatePermissionProbe(loader=FakeLoader(), api=PullRequestOnlyHfProjectStateApi()) # type: ignore[arg-type]
|
| 320 |
-
|
| 321 |
-
with pytest.raises(HfProjectStateStoreError, match="Pull Request contributions"):
|
| 322 |
-
probe.probe(project=project, actor_username="jrrribeiro", token="oauth_token")
|
| 323 |
-
|
| 324 |
-
|
| 325 |
def test_load_project_state_recovers_project_acl_and_invites() -> None:
|
| 326 |
loader = HfProjectStateStoreLoader(
|
| 327 |
api=FakeHfProjectStateReadApi(
|
|
|
|
| 8 |
HfProjectStateStoreError,
|
| 9 |
HfProjectStateStoreConnector,
|
| 10 |
HfProjectStateStoreInitializer,
|
|
|
|
|
|
|
| 11 |
HfProjectStateStoreLoader,
|
| 12 |
HfProjectStateStoreSync,
|
| 13 |
default_state_repo_id,
|
|
|
|
| 48 |
return self.files[path]
|
| 49 |
|
| 50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
def _project() -> Project:
|
| 52 |
return Project(
|
| 53 |
project_slug="upload-test2",
|
|
|
|
| 256 |
)
|
| 257 |
|
| 258 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 259 |
def test_load_project_state_recovers_project_acl_and_invites() -> None:
|
| 260 |
loader = HfProjectStateStoreLoader(
|
| 261 |
api=FakeHfProjectStateReadApi(
|
tests/unit/test_login_page.py
CHANGED
|
@@ -10,7 +10,6 @@ def test_perform_login_blocks_username_when_disabled() -> None:
|
|
| 10 |
session_id, message = perform_login(
|
| 11 |
auth_service,
|
| 12 |
"admin_user",
|
| 13 |
-
"",
|
| 14 |
allow_username_login=False,
|
| 15 |
)
|
| 16 |
|
|
@@ -25,7 +24,6 @@ def test_perform_login_allows_username_when_enabled() -> None:
|
|
| 25 |
session_id, message = perform_login(
|
| 26 |
auth_service,
|
| 27 |
"admin_user",
|
| 28 |
-
"",
|
| 29 |
allow_username_login=True,
|
| 30 |
)
|
| 31 |
|
|
@@ -33,27 +31,6 @@ def test_perform_login_allows_username_when_enabled() -> None:
|
|
| 33 |
assert "Welcome, admin_user" in message
|
| 34 |
|
| 35 |
|
| 36 |
-
def test_perform_login_allows_hf_token_when_username_disabled(monkeypatch) -> None: # noqa: ANN001
|
| 37 |
-
class FakeApi:
|
| 38 |
-
def whoami(self, token: str):
|
| 39 |
-
assert token == "hf_valid"
|
| 40 |
-
return {"name": "hf_user", "email": "hf_user@example.org"}
|
| 41 |
-
|
| 42 |
-
monkeypatch.setattr("src.auth.auth_service.HfApi", lambda: FakeApi())
|
| 43 |
-
auth_service = AuthService()
|
| 44 |
-
|
| 45 |
-
session_id, message = perform_login(
|
| 46 |
-
auth_service,
|
| 47 |
-
"",
|
| 48 |
-
"hf_valid",
|
| 49 |
-
allow_username_login=False,
|
| 50 |
-
)
|
| 51 |
-
|
| 52 |
-
assert session_id
|
| 53 |
-
assert "Welcome, hf_user" in message
|
| 54 |
-
assert auth_service.get_session(session_id).authentication_method == "hf_token"
|
| 55 |
-
|
| 56 |
-
|
| 57 |
def test_perform_oauth_login_uses_verified_profile_and_token() -> None:
|
| 58 |
class OAuthToken:
|
| 59 |
token = "oauth_token"
|
|
|
|
| 10 |
session_id, message = perform_login(
|
| 11 |
auth_service,
|
| 12 |
"admin_user",
|
|
|
|
| 13 |
allow_username_login=False,
|
| 14 |
)
|
| 15 |
|
|
|
|
| 24 |
session_id, message = perform_login(
|
| 25 |
auth_service,
|
| 26 |
"admin_user",
|
|
|
|
| 27 |
allow_username_login=True,
|
| 28 |
)
|
| 29 |
|
|
|
|
| 31 |
assert "Welcome, admin_user" in message
|
| 32 |
|
| 33 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
def test_perform_oauth_login_uses_verified_profile_and_token() -> None:
|
| 35 |
class OAuthToken:
|
| 36 |
token = "oauth_token"
|