File size: 7,199 Bytes
4046a11
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
# Token Security Guide

This guide explains how to securely manage Hugging Face tokens in BirdNET Uploader.

## Token Resolution Order

The uploader resolves tokens in this order (highest to lowest priority):

1. **`--token` CLI option** – Use `--token hf_...` in the command
2. **`HF_TOKEN` environment variable** – Set `HF_TOKEN=hf_...` before running the CLI
3. **Keyring secure storage** – Use `birdnet-uploader login` to store the token securely

## Usage Methods

### Method 1: CLI Option (Least Secure for Scripting)

```bash
birdnet-uploader upload \
  --repo-id user/dataset \
  --segments ./segments \
  --token hf_aBcDeF123456
```

**Warning**: Token appears in shell history. Use only for interactive use or in CI/CD secrets.

### Method 2: Environment Variable (Recommended for Docker/CI)

**Local machine:**
```bash
export HF_TOKEN=hf_aBcDeF123456
birdnet-uploader upload --repo-id user/dataset --segments ./segments
```

**Docker:**
```bash
docker run --rm \
  -e HF_TOKEN="hf_aBcDeF123456" \
  -v C:/segments:C:/data/segments \
  -v C:/uploads:C:/data/sessions \
  birdnet-uploader:latest \
  upload --repo-id user/dataset --segments C:/data/segments
```

**GitHub Actions:**
```yaml
- name: Upload to Hugging Face
  env:
    HF_TOKEN: ${{ secrets.HF_TOKEN }}
  run: |
    birdnet-uploader upload \
      --repo-id user/dataset \
      --segments ./segments
```

**Docker Compose:**
```yaml
services:
  uploader:
    image: birdnet-uploader:latest
    environment:
      - HF_TOKEN=${HF_TOKEN}
    volumes:
      - ./segments:/data/segments
      - ./sessions:/data/sessions
```

### Method 3: Keyring Secure Storage (Recommended for Interactive Use)

Store token securely using your OS credential manager:

```bash
# Store token in keyring (Credential Manager on Windows, Keychain on macOS, Secret Service on Linux)
birdnet-uploader login
# Prompted for token, stored securely

# Use without --token option (reads from keyring)
birdnet-uploader upload --repo-id user/dataset --segments ./segments
```

**Verify stored token:**
```bash
# On Windows PowerShell
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor [System.Net.SecurityProtocolType]::Tls12
# Token is stored in Credential Manager under "birdnet-uploader" / "hf_token"
```

**Clear stored token:**
```bash
# Using a logout command (if implemented) or manually via Credential Manager:
# Windows: Control Panel > Credential Manager > Windows Credentials > birdnet-uploader
```

## Security Best Practices

### βœ… Do This

- βœ… Use `HF_TOKEN` environment variable in Docker/CI environments
- βœ… Use keyring storage (`birdnet-uploader login`) for local interactive use
- βœ… Store tokens in CI/CD secret managers (GitHub Secrets, GitLab CI Variables, etc.)
- βœ… Use Docker secrets for orchestrated environments (Kubernetes, Docker Swarm)
- βœ… Rotate tokens periodically (every 3-6 months)
- βœ… Use read-only tokens when possible (limited to specific datasets)

### ❌ Don't Do This

- ❌ Hardcode tokens in scripts or configuration files
- ❌ Commit tokens to version control (even in `.env` files)
- ❌ Pass tokens as CLI arguments in shared environments (appears in process lists)
- ❌ Log or print tokens (the CLI never displays tokens in output)
- ❌ Share tokens via email or unencrypted communication
- ❌ Use the same token across multiple services/projects

## Container-Specific Guidance

### Windows Container Example

```dockerfile
# Dockerfile (do NOT include token)
FROM birdnet-uploader:latest

WORKDIR C:\uploads
```

**Run with token:**
```powershell
docker run --rm `
  -e HF_TOKEN="hf_aBcDeF123456" `
  -v "C:\segments:C:\data\segments" `
  -v "C:\sessions:C:\data\sessions" `
  birdnet-uploader:latest `
  upload --repo-id user/dataset --segments C:\data\segments
```

### Docker Compose with Secrets

```yaml
version: '3.8'
services:
  uploader:
    image: birdnet-uploader:latest
    environment:
      - HF_TOKEN=/run/secrets/hf_token
    secrets:
      - hf_token
    volumes:
      - ./segments:/data/segments
      - ./sessions:/data/sessions

secrets:
  hf_token:
    external: true
```

Create and run:
```bash
echo "hf_aBcDeF123456" | docker secret create hf_token -
docker-compose up
```

## Token Permissions

When creating your Hugging Face token:

1. Go to [Hugging Face Settings > Access Tokens](https://huggingface.co/settings/tokens)
2. Create a **"Fine-grained"** token with minimal required permissions:
   - **Permissions**: `repo.content.write`, `repo.content.read`
   - **Repositories**: Specify only the dataset repo you're uploading to
   - **Expiration**: Set to 3-6 months

This limits the blast radius if the token is compromised.

## Troubleshooting

### "No stored token found" error

```bash
# Solution 1: Use HF_TOKEN environment variable
export HF_TOKEN=hf_aBcDeF123456

# Solution 2: Use CLI --token option
birdnet-uploader upload --repo-id user/dataset --segments ./segments --token hf_aBcDeF123456

# Solution 3: Store token via login
birdnet-uploader login
# Enter token when prompted
```

### Token validation failed

1. Verify token format starts with `hf_`
2. Check token hasn't expired (renew at https://huggingface.co/settings/tokens)
3. Verify token has `repo.content.write` permission on target dataset
4. Test token with: `curl -H "Authorization: Bearer hf_aBcDeF123456" https://huggingface.co/api/user`

### Token leaks in logs

The CLI is designed never to log or display tokens. If you see a token in logs:

1. **Never commit the leaked token** – regenerate immediately
2. **Rotate the token** at https://huggingface.co/settings/tokens
3. **Report to Hugging Face** if suspicious activity detected

## Environment Variable Reference

| Variable | Purpose | Default | Example |
|----------|---------|---------|---------|
| `HF_TOKEN` | Authentication token | (not set) | `hf_aBcDeF123456` |
| `BIRDNET_UPLOADER_DATA_DIR` | Session/cache root (container) | `~/.birdnet-uploader` | `C:\data` |
| `BIRDNET_UPLOADER_SESSION_DIR` | Override session directory | `$DATA_DIR/sessions` | `/tmp/sessions` |
| `BIRDNET_UPLOADER_CACHE_DIR` | Override cache directory | `$DATA_DIR/cache` | `/tmp/cache` |

## FAQ

**Q: Is the token stored after using `--token` CLI option?**
A: No. The `--token` option is used only for that command. Use `birdnet-uploader login` to store permanently.

**Q: Can I use different tokens for different datasets?**
A: Currently, only one token is stored in keyring. For multiple datasets, use `HF_TOKEN` env var or `--token` option per command.

**Q: What happens if I expose HF_TOKEN in docker run history?**
A: It's visible in `docker history` and shell history. Use Docker secrets or `.env` files with `docker-compose` for better isolation.

**Q: How do I rotate tokens without stopping the container?**
A: Set a new `HF_TOKEN` env var and restart the container, or pass `--token` directly to each command.

**Q: Are tokens encrypted in keyring storage?**
A: Yes. Keyring uses OS-level credential storage:
- **Windows**: Credential Manager (DPAPI encrypted)
- **macOS**: Keychain (encrypted)
- **Linux**: Secret Service or pass (encrypted, if backend supports it)