Spaces:
Sleeping
Sleeping
Sync app from GitHub Actions (feature/project-state-security-privacy-review)
Browse files
README.md
CHANGED
|
@@ -200,22 +200,23 @@ Optional runtime settings:
|
|
| 200 |
- `BIRDNET_HF_PROJECT_STATE_WRITES_ENABLED=true` (experimental `_state` diagnostics and recovery only while a new collaborative backend is selected)
|
| 201 |
- `BIRDNET_HF_BUCKET_VALIDATIONS_ENABLED=true` (legacy experimental access for existing Bucket projects)
|
| 202 |
- `BIRDNET_HF_ADMIN_STORAGE_MODE_ENABLED=true` (private HF-only delivery mode for a small trusted team)
|
| 203 |
-
- `BIRDNET_HF_PROJECT_STATE_REPOS=jrrribeiro/audio-dataset_state` (
|
| 204 |
|
| 205 |
The HF administrator-owned storage mode is intended for confidential projects run by a small trusted research team. The source audio dataset must already be private in the administrator's personal Hugging Face namespace. When a project is created, the app creates a private `_state` repository for project/ACL/invite recovery and a private Storage Bucket for validation events and snapshots in that same namespace. Select `collaborative` inside the app: this enables app-managed validators and does not make any Hugging Face resource public.
|
| 206 |
|
| 207 |
The Space stores the administrator storage credential as a protected secret. Validators log in with their own Hugging Face identity and are invited or assigned inside the app; they do not need repository or Bucket permissions, organization membership, or the administrator token. Resource operations are performed server-side only after the app confirms the user's project role.
|
| 208 |
|
|
|
|
|
|
|
| 209 |
Recommended administrator-owned Space settings:
|
| 210 |
|
| 211 |
```text
|
| 212 |
BIRDNET_HF_ADMIN_STORAGE_MODE_ENABLED=true
|
| 213 |
BIRDNET_AUTH_MODE=hf_token
|
| 214 |
-
BIRDNET_HF_PROJECT_STATE_REPOS=jrrribeiro/audio-dataset_state
|
| 215 |
BIRDNET_HF_STORAGE_TOKEN=<administrator token stored only as a Space secret>
|
| 216 |
```
|
| 217 |
|
| 218 |
-
`HF_TOKEN` remains a compatibility fallback for the storage credential, but `BIRDNET_HF_STORAGE_TOKEN` is preferred because its purpose is explicit. Never enter this administrator token in an ordinary user login form.
|
| 219 |
|
| 220 |
Projects previously created with a private `_state` repository in this mode can be recovered from the **Admin** tab using **Connect Existing State**. Sign in as an administrator recorded in its `acl.json`, then provide the `_state` repo id (for example, `jrrribeiro/audio_dataset_state`). Recovery accepts only a manifest with private Bucket validation storage and a private personal-namespace source dataset accessible to the configured storage account. The app loads the saved manifest, ACL, and pending invites.
|
| 221 |
|
|
|
|
| 200 |
- `BIRDNET_HF_PROJECT_STATE_WRITES_ENABLED=true` (experimental `_state` diagnostics and recovery only while a new collaborative backend is selected)
|
| 201 |
- `BIRDNET_HF_BUCKET_VALIDATIONS_ENABLED=true` (legacy experimental access for existing Bucket projects)
|
| 202 |
- `BIRDNET_HF_ADMIN_STORAGE_MODE_ENABLED=true` (private HF-only delivery mode for a small trusted team)
|
| 203 |
+
- `BIRDNET_HF_PROJECT_STATE_REPOS=jrrribeiro/audio-dataset_state` (optional manual recovery list; administrator-owned mode discovers companion state repos automatically)
|
| 204 |
|
| 205 |
The HF administrator-owned storage mode is intended for confidential projects run by a small trusted research team. The source audio dataset must already be private in the administrator's personal Hugging Face namespace. When a project is created, the app creates a private `_state` repository for project/ACL/invite recovery and a private Storage Bucket for validation events and snapshots in that same namespace. Select `collaborative` inside the app: this enables app-managed validators and does not make any Hugging Face resource public.
|
| 206 |
|
| 207 |
The Space stores the administrator storage credential as a protected secret. Validators log in with their own Hugging Face identity and are invited or assigned inside the app; they do not need repository or Bucket permissions, organization membership, or the administrator token. Resource operations are performed server-side only after the app confirms the user's project role.
|
| 208 |
|
| 209 |
+
When this mode is enabled it takes precedence over any existing Supabase settings: project ACL/invites are restored from private `*_state` repositories and validations are read and written in the project's private Bucket.
|
| 210 |
+
|
| 211 |
Recommended administrator-owned Space settings:
|
| 212 |
|
| 213 |
```text
|
| 214 |
BIRDNET_HF_ADMIN_STORAGE_MODE_ENABLED=true
|
| 215 |
BIRDNET_AUTH_MODE=hf_token
|
|
|
|
| 216 |
BIRDNET_HF_STORAGE_TOKEN=<administrator token stored only as a Space secret>
|
| 217 |
```
|
| 218 |
|
| 219 |
+
`HF_TOKEN` remains a compatibility fallback for the storage credential, but `BIRDNET_HF_STORAGE_TOKEN` is preferred because its purpose is explicit. Never enter this administrator token in an ordinary user login form. On restart the app uses the protected credential to find the administrator's companion `*_state` repositories automatically; `BIRDNET_HF_PROJECT_STATE_REPOS` remains available as an explicit fallback or migration list. An administrator may run **Projects > Private storage backend health** to verify the protected backend credential with a temporary marker that is immediately removed.
|
| 220 |
|
| 221 |
Projects previously created with a private `_state` repository in this mode can be recovered from the **Admin** tab using **Connect Existing State**. Sign in as an administrator recorded in its `acl.json`, then provide the `_state` repo id (for example, `jrrribeiro/audio_dataset_state`). Recovery accepts only a manifest with private Bucket validation storage and a private personal-namespace source dataset accessible to the configured storage account. The app loads the saved manifest, ACL, and pending invites.
|
| 222 |
|
docs/TEMP_HF_ADMIN_OWNED_PRIVATE_STORAGE_DELIVERY_PLAN.md
CHANGED
|
@@ -50,7 +50,9 @@ For public distribution to independent administrators, this model is not suffici
|
|
| 50 |
5. Project manifests persist the Bucket reference; ACL and invite changes sync to `_state`.
|
| 51 |
6. Validation Bucket reads and writes route through the backend storage credential while retaining the actual validator username in each event.
|
| 52 |
7. Dataset metadata and audio fetches route through the backend credential only after server-side authorization of the session and project role.
|
| 53 |
-
8.
|
|
|
|
|
|
|
| 54 |
|
| 55 |
## Space Configuration
|
| 56 |
|
|
@@ -59,7 +61,6 @@ Configure:
|
|
| 59 |
```text
|
| 60 |
BIRDNET_HF_ADMIN_STORAGE_MODE_ENABLED=true
|
| 61 |
BIRDNET_AUTH_MODE=hf_token
|
| 62 |
-
BIRDNET_HF_PROJECT_STATE_REPOS=jrrribeiro/audio-dataset_state
|
| 63 |
```
|
| 64 |
|
| 65 |
Configure as a Secret:
|
|
@@ -74,8 +75,8 @@ Use a scoped credential limited to the administrator's required resources when H
|
|
| 74 |
|
| 75 |
1. Administrator creates a private test dataset in their personal namespace.
|
| 76 |
2. Administrator creates the app project; the app reports the generated private `_state` repository and validation Bucket.
|
| 77 |
-
3. Administrator
|
| 78 |
-
4. Project is restored with its ACL and Bucket reference intact.
|
| 79 |
5. Administrator assigns or invites a second Hugging Face account only through the app.
|
| 80 |
6. The second account logs in with its own identity, opens the project, loads private audio, and submits validations without Hub resource permissions.
|
| 81 |
7. The second account cannot access the private audio dataset, `_state`, or Bucket directly on Hugging Face.
|
|
|
|
| 50 |
5. Project manifests persist the Bucket reference; ACL and invite changes sync to `_state`.
|
| 51 |
6. Validation Bucket reads and writes route through the backend storage credential while retaining the actual validator username in each event.
|
| 52 |
7. Dataset metadata and audio fetches route through the backend credential only after server-side authorization of the session and project role.
|
| 53 |
+
8. At startup the app automatically discovers the administrator's private companion `*_state` datasets for rebuild recovery; a manual repo list remains an optional fallback.
|
| 54 |
+
9. This mode takes precedence over configured Supabase persistence so durable project state does not remain dependent on the app operator's Supabase database.
|
| 55 |
+
10. The administrator can run a backend storage health check; validators do not test or receive direct Bucket permissions.
|
| 56 |
|
| 57 |
## Space Configuration
|
| 58 |
|
|
|
|
| 61 |
```text
|
| 62 |
BIRDNET_HF_ADMIN_STORAGE_MODE_ENABLED=true
|
| 63 |
BIRDNET_AUTH_MODE=hf_token
|
|
|
|
| 64 |
```
|
| 65 |
|
| 66 |
Configure as a Secret:
|
|
|
|
| 75 |
|
| 76 |
1. Administrator creates a private test dataset in their personal namespace.
|
| 77 |
2. Administrator creates the app project; the app reports the generated private `_state` repository and validation Bucket.
|
| 78 |
+
3. Administrator restarts the Space without manually registering the new `_state` repository.
|
| 79 |
+
4. Project is discovered and restored with its ACL and Bucket reference intact.
|
| 80 |
5. Administrator assigns or invites a second Hugging Face account only through the app.
|
| 81 |
6. The second account logs in with its own identity, opens the project, loads private audio, and submits validations without Hub resource permissions.
|
| 82 |
7. The second account cannot access the private audio dataset, `_state`, or Bucket directly on Hugging Face.
|
src/ui/app_factory.py
CHANGED
|
@@ -1388,6 +1388,32 @@ def _load_hf_project_state_repos(
|
|
| 1388 |
return loaded, errors
|
| 1389 |
|
| 1390 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1391 |
def _resolve_bootstrap_file_paths(runtime_config: RuntimeConfig) -> tuple[Path, Path, Path]:
|
| 1392 |
bootstrap_dir = Path(runtime_config.bootstrap_base_dir)
|
| 1393 |
projects_path = Path(runtime_config.projects_file_path) if runtime_config.projects_file_path else (bootstrap_dir / "projects.json")
|
|
@@ -1462,6 +1488,7 @@ def _bootstrap_auth_and_projects(
|
|
| 1462 |
state_store: SupabaseBootstrapStore | None = None,
|
| 1463 |
hf_project_state_token: str | None = None,
|
| 1464 |
hf_project_state_loader: HfProjectStateStoreLoader | None = None,
|
|
|
|
| 1465 |
) -> str:
|
| 1466 |
if state_store is not None:
|
| 1467 |
projects = state_store.load_projects()
|
|
@@ -1472,8 +1499,17 @@ def _bootstrap_auth_and_projects(
|
|
| 1472 |
user_access = _load_user_access_from_file(user_access_file_path or runtime_config.user_access_file_path)
|
| 1473 |
pending_invites = _load_pending_invites_from_file(invites_file_path or runtime_config.invites_file_path)
|
| 1474 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1475 |
hf_loaded_state, hf_load_errors = _load_hf_project_state_repos(
|
| 1476 |
-
state_repo_ids=
|
| 1477 |
token=hf_project_state_token,
|
| 1478 |
loader=hf_project_state_loader,
|
| 1479 |
)
|
|
@@ -1532,6 +1568,8 @@ def _bootstrap_auth_and_projects(
|
|
| 1532 |
)
|
| 1533 |
|
| 1534 |
warnings = [emergency_admin_message] if emergency_admin_message else []
|
|
|
|
|
|
|
| 1535 |
if hf_load_errors:
|
| 1536 |
warnings.append("⚠️ Some HF project-state repos could not be loaded: " + " | ".join(hf_load_errors))
|
| 1537 |
if hf_loaded_state:
|
|
@@ -2981,6 +3019,8 @@ def create_app() -> gr.Blocks:
|
|
| 2981 |
projects_file_path, user_access_file_path, invites_file_path = _resolve_bootstrap_file_paths(runtime_config)
|
| 2982 |
state_store, supabase_validation_repository, state_backend_message = _build_supabase_state(runtime_config)
|
| 2983 |
if runtime_config.hf_admin_storage_mode_enabled:
|
|
|
|
|
|
|
| 2984 |
state_backend_message = "HF admin-owned private storage enabled."
|
| 2985 |
allow_username_login, auth_mode_label, auth_mode_description = _resolve_username_login_policy(runtime_config)
|
| 2986 |
bootstrap_warning = _bootstrap_auth_and_projects(
|
|
@@ -3624,8 +3664,7 @@ def create_app() -> gr.Blocks:
|
|
| 3624 |
else ""
|
| 3625 |
)
|
| 3626 |
recovery_message = (
|
| 3627 |
-
|
| 3628 |
-
"the Space so this project is restored after restart."
|
| 3629 |
if runtime_config.hf_admin_storage_mode_enabled
|
| 3630 |
else ""
|
| 3631 |
)
|
|
@@ -6116,7 +6155,11 @@ def create_app() -> gr.Blocks:
|
|
| 6116 |
if runtime_config.hf_admin_storage_mode_enabled
|
| 6117 |
else ("legacy experimental" if runtime_config.hf_bucket_validations_enabled else "disabled")
|
| 6118 |
)
|
| 6119 |
-
hf_project_state_repos_label =
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6120 |
hf_space_label = os.getenv("SPACE_ID") or "local runtime"
|
| 6121 |
health_html = settings_health_html(
|
| 6122 |
[
|
|
@@ -6126,7 +6169,7 @@ def create_app() -> gr.Blocks:
|
|
| 6126 |
(
|
| 6127 |
"HF project-state repos",
|
| 6128 |
hf_project_state_repos_label,
|
| 6129 |
-
"ok" if runtime_config.
|
| 6130 |
),
|
| 6131 |
("Auth mode", auth_mode_label, "ok" if not allow_username_login else "warn"),
|
| 6132 |
("Destructive write guard", "enabled", "ok"),
|
|
@@ -6145,11 +6188,7 @@ def create_app() -> gr.Blocks:
|
|
| 6145 |
(
|
| 6146 |
"HF administrator-owned storage is active. Private dataset and Bucket access use only the "
|
| 6147 |
"protected Space storage credential; validator identities are authorized inside this app; "
|
| 6148 |
-
|
| 6149 |
-
"no project-state repository is configured, so projects will not be restored after a rebuild."
|
| 6150 |
-
if not runtime_config.hf_project_state_repos
|
| 6151 |
-
else "configured state repositories will be used for restart recovery."
|
| 6152 |
-
)
|
| 6153 |
if state_label == "HF admin-owned private storage"
|
| 6154 |
else "Supabase persistence is active. Project and ACL removals are guarded against accidental broad overwrites."
|
| 6155 |
)
|
|
|
|
| 1388 |
return loaded, errors
|
| 1389 |
|
| 1390 |
|
| 1391 |
+
def _discover_hf_admin_state_repos(
|
| 1392 |
+
*,
|
| 1393 |
+
token: str | None,
|
| 1394 |
+
api: HfApi | None = None,
|
| 1395 |
+
) -> tuple[tuple[str, ...], str]:
|
| 1396 |
+
"""Find companion state datasets owned by the configured administrator account."""
|
| 1397 |
+
token_value = (token or "").strip()
|
| 1398 |
+
if not token_value:
|
| 1399 |
+
return (), "Administrator storage mode is enabled, but no HF storage secret is configured for state recovery."
|
| 1400 |
+
client = api or HfApi()
|
| 1401 |
+
try:
|
| 1402 |
+
identity = client.whoami(token=token_value)
|
| 1403 |
+
owner = str(identity.get("name") if isinstance(identity, dict) else "").strip()
|
| 1404 |
+
if not owner:
|
| 1405 |
+
return (), "Could not determine the administrator storage account for automatic state recovery."
|
| 1406 |
+
datasets = client.list_datasets(author=owner, token=token_value)
|
| 1407 |
+
repo_ids = []
|
| 1408 |
+
for dataset in datasets:
|
| 1409 |
+
repo_id = str(getattr(dataset, "id", None) or getattr(dataset, "repo_id", None) or "").strip()
|
| 1410 |
+
if repo_id.casefold().startswith(f"{owner}/".casefold()) and repo_id.casefold().endswith("_state"):
|
| 1411 |
+
repo_ids.append(repo_id)
|
| 1412 |
+
return tuple(sorted(set(repo_ids))), ""
|
| 1413 |
+
except Exception as exc:
|
| 1414 |
+
return (), f"Could not discover private companion state repos automatically: {exc}"
|
| 1415 |
+
|
| 1416 |
+
|
| 1417 |
def _resolve_bootstrap_file_paths(runtime_config: RuntimeConfig) -> tuple[Path, Path, Path]:
|
| 1418 |
bootstrap_dir = Path(runtime_config.bootstrap_base_dir)
|
| 1419 |
projects_path = Path(runtime_config.projects_file_path) if runtime_config.projects_file_path else (bootstrap_dir / "projects.json")
|
|
|
|
| 1488 |
state_store: SupabaseBootstrapStore | None = None,
|
| 1489 |
hf_project_state_token: str | None = None,
|
| 1490 |
hf_project_state_loader: HfProjectStateStoreLoader | None = None,
|
| 1491 |
+
hf_project_state_discovery_api: HfApi | None = None,
|
| 1492 |
) -> str:
|
| 1493 |
if state_store is not None:
|
| 1494 |
projects = state_store.load_projects()
|
|
|
|
| 1499 |
user_access = _load_user_access_from_file(user_access_file_path or runtime_config.user_access_file_path)
|
| 1500 |
pending_invites = _load_pending_invites_from_file(invites_file_path or runtime_config.invites_file_path)
|
| 1501 |
|
| 1502 |
+
configured_state_repos = tuple(runtime_config.hf_project_state_repos)
|
| 1503 |
+
discovery_message = ""
|
| 1504 |
+
discovered_state_repos: tuple[str, ...] = ()
|
| 1505 |
+
if runtime_config.hf_admin_storage_mode_enabled:
|
| 1506 |
+
discovered_state_repos, discovery_message = _discover_hf_admin_state_repos(
|
| 1507 |
+
token=hf_project_state_token,
|
| 1508 |
+
api=hf_project_state_discovery_api,
|
| 1509 |
+
)
|
| 1510 |
+
state_repo_ids = tuple(dict.fromkeys([*configured_state_repos, *discovered_state_repos]))
|
| 1511 |
hf_loaded_state, hf_load_errors = _load_hf_project_state_repos(
|
| 1512 |
+
state_repo_ids=state_repo_ids,
|
| 1513 |
token=hf_project_state_token,
|
| 1514 |
loader=hf_project_state_loader,
|
| 1515 |
)
|
|
|
|
| 1568 |
)
|
| 1569 |
|
| 1570 |
warnings = [emergency_admin_message] if emergency_admin_message else []
|
| 1571 |
+
if discovery_message:
|
| 1572 |
+
warnings.append(f"⚠️ {discovery_message}")
|
| 1573 |
if hf_load_errors:
|
| 1574 |
warnings.append("⚠️ Some HF project-state repos could not be loaded: " + " | ".join(hf_load_errors))
|
| 1575 |
if hf_loaded_state:
|
|
|
|
| 3019 |
projects_file_path, user_access_file_path, invites_file_path = _resolve_bootstrap_file_paths(runtime_config)
|
| 3020 |
state_store, supabase_validation_repository, state_backend_message = _build_supabase_state(runtime_config)
|
| 3021 |
if runtime_config.hf_admin_storage_mode_enabled:
|
| 3022 |
+
state_store = None
|
| 3023 |
+
supabase_validation_repository = None
|
| 3024 |
state_backend_message = "HF admin-owned private storage enabled."
|
| 3025 |
allow_username_login, auth_mode_label, auth_mode_description = _resolve_username_login_policy(runtime_config)
|
| 3026 |
bootstrap_warning = _bootstrap_auth_and_projects(
|
|
|
|
| 3664 |
else ""
|
| 3665 |
)
|
| 3666 |
recovery_message = (
|
| 3667 |
+
" This private state repository will be discovered automatically after a Space restart."
|
|
|
|
| 3668 |
if runtime_config.hf_admin_storage_mode_enabled
|
| 3669 |
else ""
|
| 3670 |
)
|
|
|
|
| 6155 |
if runtime_config.hf_admin_storage_mode_enabled
|
| 6156 |
else ("legacy experimental" if runtime_config.hf_bucket_validations_enabled else "disabled")
|
| 6157 |
)
|
| 6158 |
+
hf_project_state_repos_label = (
|
| 6159 |
+
"automatic discovery"
|
| 6160 |
+
if runtime_config.hf_admin_storage_mode_enabled
|
| 6161 |
+
else str(len(runtime_config.hf_project_state_repos))
|
| 6162 |
+
)
|
| 6163 |
hf_space_label = os.getenv("SPACE_ID") or "local runtime"
|
| 6164 |
health_html = settings_health_html(
|
| 6165 |
[
|
|
|
|
| 6169 |
(
|
| 6170 |
"HF project-state repos",
|
| 6171 |
hf_project_state_repos_label,
|
| 6172 |
+
"ok" if runtime_config.hf_admin_storage_mode_enabled or runtime_config.hf_project_state_repos else "info",
|
| 6173 |
),
|
| 6174 |
("Auth mode", auth_mode_label, "ok" if not allow_username_login else "warn"),
|
| 6175 |
("Destructive write guard", "enabled", "ok"),
|
|
|
|
| 6188 |
(
|
| 6189 |
"HF administrator-owned storage is active. Private dataset and Bucket access use only the "
|
| 6190 |
"protected Space storage credential; validator identities are authorized inside this app; "
|
| 6191 |
+
"private companion state repositories are discovered automatically for restart recovery."
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6192 |
if state_label == "HF admin-owned private storage"
|
| 6193 |
else "Supabase persistence is active. Project and ACL removals are guarded against accidental broad overwrites."
|
| 6194 |
)
|
tests/unit/test_app_factory_audio_helpers.py
CHANGED
|
@@ -33,6 +33,7 @@ from src.ui.app_factory import (
|
|
| 33 |
_load_projects_from_file,
|
| 34 |
_load_user_access_from_file,
|
| 35 |
_bootstrap_auth_and_projects,
|
|
|
|
| 36 |
_persist_bootstrap_state,
|
| 37 |
_resolve_username_login_policy,
|
| 38 |
_resolve_project_fetch_token,
|
|
@@ -96,6 +97,21 @@ class FakeBucketInitializer:
|
|
| 96 |
)
|
| 97 |
|
| 98 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 99 |
class InaccessibleAudioService:
|
| 100 |
def fetch(self, dataset_repo: str, audio_id: str, **kwargs: object) -> FakeFetchResult:
|
| 101 |
_ = (dataset_repo, audio_id, kwargs)
|
|
@@ -310,6 +326,16 @@ def test_initialize_hf_admin_storage_requires_collaborative_app_visibility() ->
|
|
| 310 |
assert bucket_initializer.calls == []
|
| 311 |
|
| 312 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 313 |
def test_extract_audio_id_from_list_rows() -> None:
|
| 314 |
rows = [["k1", "audio_01", "sp", 0.9, 0.0, 1.0]]
|
| 315 |
assert _extract_audio_id(rows, 0) == "audio_01"
|
|
|
|
| 33 |
_load_projects_from_file,
|
| 34 |
_load_user_access_from_file,
|
| 35 |
_bootstrap_auth_and_projects,
|
| 36 |
+
_discover_hf_admin_state_repos,
|
| 37 |
_persist_bootstrap_state,
|
| 38 |
_resolve_username_login_policy,
|
| 39 |
_resolve_project_fetch_token,
|
|
|
|
| 97 |
)
|
| 98 |
|
| 99 |
|
| 100 |
+
class FakeStateDiscoveryApi:
|
| 101 |
+
def whoami(self, token: str): # noqa: ANN001
|
| 102 |
+
assert token == "hf_storage"
|
| 103 |
+
return {"name": "jrrribeiro"}
|
| 104 |
+
|
| 105 |
+
def list_datasets(self, *, author: str, token: str): # noqa: ANN001
|
| 106 |
+
assert author == "jrrribeiro"
|
| 107 |
+
assert token == "hf_storage"
|
| 108 |
+
return [
|
| 109 |
+
SimpleNamespace(id="jrrribeiro/project-a_state"),
|
| 110 |
+
SimpleNamespace(id="jrrribeiro/audio-source"),
|
| 111 |
+
SimpleNamespace(id="someone-else/project-b_state"),
|
| 112 |
+
]
|
| 113 |
+
|
| 114 |
+
|
| 115 |
class InaccessibleAudioService:
|
| 116 |
def fetch(self, dataset_repo: str, audio_id: str, **kwargs: object) -> FakeFetchResult:
|
| 117 |
_ = (dataset_repo, audio_id, kwargs)
|
|
|
|
| 326 |
assert bucket_initializer.calls == []
|
| 327 |
|
| 328 |
|
| 329 |
+
def test_discover_hf_admin_state_repos_filters_personal_companion_datasets() -> None:
|
| 330 |
+
repo_ids, warning = _discover_hf_admin_state_repos(
|
| 331 |
+
token="hf_storage",
|
| 332 |
+
api=FakeStateDiscoveryApi(), # type: ignore[arg-type]
|
| 333 |
+
)
|
| 334 |
+
|
| 335 |
+
assert warning == ""
|
| 336 |
+
assert repo_ids == ("jrrribeiro/project-a_state",)
|
| 337 |
+
|
| 338 |
+
|
| 339 |
def test_extract_audio_id_from_list_rows() -> None:
|
| 340 |
rows = [["k1", "audio_01", "sp", 0.9, 0.0, 1.0]]
|
| 341 |
assert _extract_audio_id(rows, 0) == "audio_01"
|