ignatius commited on
Commit
2cf6688
·
2 Parent(s): c1d045b96a766a

Merge branch 'main' of ../curatorbot_new

Browse files

# Conflicts:
# .github/workflows/deploy-space.yml
# curatorbot/config.py
# docs/deploy-huggingface.md
# docs/real-data-runbook.md

.github/workflows/deploy-space.yml CHANGED
@@ -1,6 +1,6 @@
1
  name: deploy to Hugging Face Space
2
  # Pushes main to the Space after tests pass.
3
- # Needs repository secret HF_TOKEN (write access) and variable HF_SPACE (e.g. IgnatiusEzeani/curatorbot).
4
  on:
5
  workflow_run:
6
  workflows: [tests]
 
1
  name: deploy to Hugging Face Space
2
  # Pushes main to the Space after tests pass.
3
+ # Needs repository secret HF_TOKEN (write access) and variable HF_SPACE (e.g. ignatius/curatorbot).
4
  on:
5
  workflow_run:
6
  workflows: [tests]
curatorbot/config.py CHANGED
@@ -26,7 +26,7 @@ class Settings:
26
  llm_max_tokens: int = field(default_factory=lambda: int(os.getenv("CURATORBOT_LLM_MAX_TOKENS", "6000")))
27
  ner_model: str = field(default_factory=lambda: os.getenv("CURATORBOT_NER_MODEL", "en_core_web_sm"))
28
  data_dir: Path = field(default_factory=lambda: Path(os.getenv("CURATORBOT_DATA_DIR", str(Path(__file__).resolve().parent.parent / "data" / "interviews"))))
29
- # Private Hugging Face dataset holding real briefs (e.g. IgnatiusEzeani/curatorbot-data).
30
  # Downloaded at start-up so partner material never enters the code repository.
31
  data_repo: str = field(default_factory=lambda: os.getenv("CURATORBOT_DATA_REPO", ""))
32
  private_data_dir: Path = field(default_factory=lambda: Path(os.getenv("CURATORBOT_PRIVATE_DATA_DIR", str(Path.home() / ".cache" / "curatorbot-data"))))
 
26
  llm_max_tokens: int = field(default_factory=lambda: int(os.getenv("CURATORBOT_LLM_MAX_TOKENS", "6000")))
27
  ner_model: str = field(default_factory=lambda: os.getenv("CURATORBOT_NER_MODEL", "en_core_web_sm"))
28
  data_dir: Path = field(default_factory=lambda: Path(os.getenv("CURATORBOT_DATA_DIR", str(Path(__file__).resolve().parent.parent / "data" / "interviews"))))
29
+ # Private Hugging Face dataset holding real briefs (e.g. ignatius/curatorbot-data).
30
  # Downloaded at start-up so partner material never enters the code repository.
31
  data_repo: str = field(default_factory=lambda: os.getenv("CURATORBOT_DATA_REPO", ""))
32
  private_data_dir: Path = field(default_factory=lambda: Path(os.getenv("CURATORBOT_PRIVATE_DATA_DIR", str(Path.home() / ".cache" / "curatorbot-data"))))
docs/deploy-huggingface.md CHANGED
@@ -4,23 +4,23 @@ The repository is a ready Docker Space: the YAML block at the top of `README.md`
4
 
5
  **Plan needed.** Since mid-2026, Hugging Face requires a paid plan to create Docker (and Gradio) Spaces: PRO ($9 a month) for a personal account, Team ($20 per user a month) for an organisation. Static Spaces stay free. The hardware itself (CPU basic, 2 vCPU, 16 GB) has no hourly cost. Check [huggingface.co/pricing](https://huggingface.co/pricing) for current prices.
6
 
7
- This project hosts the Space on the lead's personal account (`IgnatiusEzeani/curatorbot`) with PRO, while the code stays in the `SpaceTimeNarratives` GitHub organisation. To move it to an organisation later, take a Team plan, create the Space there and change the `HF_SPACE` variable.
8
 
9
  ## One-time setup
10
 
11
  1. **Subscribe to PRO** on the personal Hugging Face account (Settings > Billing, or huggingface.co/pro).
12
- 2. **Create the Space.** New Space: owner `IgnatiusEzeani`, name `curatorbot`, SDK **Docker**, blank template, hardware **CPU basic**, visibility **Private**.
13
  3. **Add secrets** (Space > Settings > Variables and secrets):
14
  - `ANTHROPIC_API_KEY` (secret), or `OPENAI_API_KEY` with variable `CURATORBOT_LLM_PROVIDER=openai`
15
  - `CURATORBOT_DEMO_PASSCODE` (secret): protects your API credit on a public or shared Space
16
- - for real interviews: secret `HF_TOKEN` (read access to the private dataset) and variable `CURATORBOT_DATA_REPO=IgnatiusEzeani/curatorbot-data` (see `docs/real-data-runbook.md`)
17
  - optional variables: `CURATORBOT_LLM_MODEL`, `CURATORBOT_ENABLE_ASR=1` for short audio clips (slow on CPU basic), `CURATORBOT_MAX_CHARS`
18
- 4. **Connect GitHub.** Create a Hugging Face access token with **write** permission. In the GitHub repo (Settings > Secrets and variables > Actions) add secret `HF_TOKEN` (that token) and variable `HF_SPACE=IgnatiusEzeani/curatorbot`. The `deploy to Hugging Face Space` workflow then pushes `main` to the Space after tests pass. Run it once by hand from the Actions tab.
19
 
20
  To deploy without GitHub Actions:
21
 
22
  ```bash
23
- git remote add space https://huggingface.co/spaces/IgnatiusEzeani/curatorbot
24
  git push space main
25
  ```
26
 
 
4
 
5
  **Plan needed.** Since mid-2026, Hugging Face requires a paid plan to create Docker (and Gradio) Spaces: PRO ($9 a month) for a personal account, Team ($20 per user a month) for an organisation. Static Spaces stay free. The hardware itself (CPU basic, 2 vCPU, 16 GB) has no hourly cost. Check [huggingface.co/pricing](https://huggingface.co/pricing) for current prices.
6
 
7
+ This project hosts the Space on the lead's personal Hugging Face account (`ignatius/curatorbot`; the GitHub account is `IgnatiusEzeani`) with PRO, while the code stays in the `SpaceTimeNarratives` GitHub organisation. To move it to an organisation later, take a Team plan, create the Space there and change the `HF_SPACE` variable.
8
 
9
  ## One-time setup
10
 
11
  1. **Subscribe to PRO** on the personal Hugging Face account (Settings > Billing, or huggingface.co/pro).
12
+ 2. **Create the Space.** New Space: owner `ignatius`, name `curatorbot`, SDK **Docker**, blank template, hardware **CPU basic**, visibility **Private**.
13
  3. **Add secrets** (Space > Settings > Variables and secrets):
14
  - `ANTHROPIC_API_KEY` (secret), or `OPENAI_API_KEY` with variable `CURATORBOT_LLM_PROVIDER=openai`
15
  - `CURATORBOT_DEMO_PASSCODE` (secret): protects your API credit on a public or shared Space
16
+ - for real interviews: secret `HF_TOKEN` (read access to the private dataset) and variable `CURATORBOT_DATA_REPO=ignatius/curatorbot-data` (see `docs/real-data-runbook.md`)
17
  - optional variables: `CURATORBOT_LLM_MODEL`, `CURATORBOT_ENABLE_ASR=1` for short audio clips (slow on CPU basic), `CURATORBOT_MAX_CHARS`
18
+ 4. **Connect GitHub.** Create a Hugging Face access token with **write** permission. In the GitHub repo (Settings > Secrets and variables > Actions) add secret `HF_TOKEN` (that token) and variable `HF_SPACE=ignatius/curatorbot`. The `deploy to Hugging Face Space` workflow then pushes `main` to the Space after tests pass. Run it once by hand from the Actions tab.
19
 
20
  To deploy without GitHub Actions:
21
 
22
  ```bash
23
+ git remote add space https://huggingface.co/spaces/ignatius/curatorbot
24
  git push space main
25
  ```
26
 
docs/real-data-runbook.md CHANGED
@@ -51,14 +51,14 @@ Released transcripts carry no timestamps, so the brief cites turn numbers (p001,
51
 
52
  One-time setup:
53
 
54
- 1. On huggingface.co: **New → Dataset**, owner `IgnatiusEzeani`, name `curatorbot-data`, **Private**.
55
  2. Create a Hugging Face token with **read** access (a fine-grained token limited to that dataset is best).
56
- 3. In the Space's settings add the secret `HF_TOKEN` (that token) and the variable `CURATORBOT_DATA_REPO` = `IgnatiusEzeani/curatorbot-data`.
57
 
58
  For each interview, `prepare_interview.py` writes `data/interviews/<ID>/brief.json` (and `backtest.json` for released interviews); the Colab notebook also copies them to `CuratorBot/<ID>/out/` in Drive. Upload that folder to the dataset, keeping the interview id as the folder name:
59
 
60
  ```bash
61
- huggingface-cli upload IgnatiusEzeani/curatorbot-data data/interviews/<ID> <ID> --repo-type dataset
62
  ```
63
 
64
  or drag the two files into a new `<ID>/` folder on the dataset's **Files** page. Then either restart the Space, or fetch without restarting:
 
51
 
52
  One-time setup:
53
 
54
+ 1. On huggingface.co: **New → Dataset**, owner `ignatius`, name `curatorbot-data`, **Private**.
55
  2. Create a Hugging Face token with **read** access (a fine-grained token limited to that dataset is best).
56
+ 3. In the Space's settings add the secret `HF_TOKEN` (that token) and the variable `CURATORBOT_DATA_REPO` = `ignatius/curatorbot-data`.
57
 
58
  For each interview, `prepare_interview.py` writes `data/interviews/<ID>/brief.json` (and `backtest.json` for released interviews); the Colab notebook also copies them to `CuratorBot/<ID>/out/` in Drive. Upload that folder to the dataset, keeping the interview id as the folder name:
59
 
60
  ```bash
61
+ huggingface-cli upload ignatius/curatorbot-data data/interviews/<ID> <ID> --repo-type dataset
62
  ```
63
 
64
  or drag the two files into a new `<ID>/` folder on the dataset's **Files** page. Then either restart the Space, or fetch without restarting: