Spaces:
Sleeping
Sleeping
File size: 7,199 Bytes
4046a11 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 | # Token Security Guide
This guide explains how to securely manage Hugging Face tokens in BirdNET Uploader.
## Token Resolution Order
The uploader resolves tokens in this order (highest to lowest priority):
1. **`--token` CLI option** β Use `--token hf_...` in the command
2. **`HF_TOKEN` environment variable** β Set `HF_TOKEN=hf_...` before running the CLI
3. **Keyring secure storage** β Use `birdnet-uploader login` to store the token securely
## Usage Methods
### Method 1: CLI Option (Least Secure for Scripting)
```bash
birdnet-uploader upload \
--repo-id user/dataset \
--segments ./segments \
--token hf_aBcDeF123456
```
**Warning**: Token appears in shell history. Use only for interactive use or in CI/CD secrets.
### Method 2: Environment Variable (Recommended for Docker/CI)
**Local machine:**
```bash
export HF_TOKEN=hf_aBcDeF123456
birdnet-uploader upload --repo-id user/dataset --segments ./segments
```
**Docker:**
```bash
docker run --rm \
-e HF_TOKEN="hf_aBcDeF123456" \
-v C:/segments:C:/data/segments \
-v C:/uploads:C:/data/sessions \
birdnet-uploader:latest \
upload --repo-id user/dataset --segments C:/data/segments
```
**GitHub Actions:**
```yaml
- name: Upload to Hugging Face
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
birdnet-uploader upload \
--repo-id user/dataset \
--segments ./segments
```
**Docker Compose:**
```yaml
services:
uploader:
image: birdnet-uploader:latest
environment:
- HF_TOKEN=${HF_TOKEN}
volumes:
- ./segments:/data/segments
- ./sessions:/data/sessions
```
### Method 3: Keyring Secure Storage (Recommended for Interactive Use)
Store token securely using your OS credential manager:
```bash
# Store token in keyring (Credential Manager on Windows, Keychain on macOS, Secret Service on Linux)
birdnet-uploader login
# Prompted for token, stored securely
# Use without --token option (reads from keyring)
birdnet-uploader upload --repo-id user/dataset --segments ./segments
```
**Verify stored token:**
```bash
# On Windows PowerShell
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor [System.Net.SecurityProtocolType]::Tls12
# Token is stored in Credential Manager under "birdnet-uploader" / "hf_token"
```
**Clear stored token:**
```bash
# Using a logout command (if implemented) or manually via Credential Manager:
# Windows: Control Panel > Credential Manager > Windows Credentials > birdnet-uploader
```
## Security Best Practices
### β
Do This
- β
Use `HF_TOKEN` environment variable in Docker/CI environments
- β
Use keyring storage (`birdnet-uploader login`) for local interactive use
- β
Store tokens in CI/CD secret managers (GitHub Secrets, GitLab CI Variables, etc.)
- β
Use Docker secrets for orchestrated environments (Kubernetes, Docker Swarm)
- β
Rotate tokens periodically (every 3-6 months)
- β
Use read-only tokens when possible (limited to specific datasets)
### β Don't Do This
- β Hardcode tokens in scripts or configuration files
- β Commit tokens to version control (even in `.env` files)
- β Pass tokens as CLI arguments in shared environments (appears in process lists)
- β Log or print tokens (the CLI never displays tokens in output)
- β Share tokens via email or unencrypted communication
- β Use the same token across multiple services/projects
## Container-Specific Guidance
### Windows Container Example
```dockerfile
# Dockerfile (do NOT include token)
FROM birdnet-uploader:latest
WORKDIR C:\uploads
```
**Run with token:**
```powershell
docker run --rm `
-e HF_TOKEN="hf_aBcDeF123456" `
-v "C:\segments:C:\data\segments" `
-v "C:\sessions:C:\data\sessions" `
birdnet-uploader:latest `
upload --repo-id user/dataset --segments C:\data\segments
```
### Docker Compose with Secrets
```yaml
version: '3.8'
services:
uploader:
image: birdnet-uploader:latest
environment:
- HF_TOKEN=/run/secrets/hf_token
secrets:
- hf_token
volumes:
- ./segments:/data/segments
- ./sessions:/data/sessions
secrets:
hf_token:
external: true
```
Create and run:
```bash
echo "hf_aBcDeF123456" | docker secret create hf_token -
docker-compose up
```
## Token Permissions
When creating your Hugging Face token:
1. Go to [Hugging Face Settings > Access Tokens](https://huggingface.co/settings/tokens)
2. Create a **"Fine-grained"** token with minimal required permissions:
- **Permissions**: `repo.content.write`, `repo.content.read`
- **Repositories**: Specify only the dataset repo you're uploading to
- **Expiration**: Set to 3-6 months
This limits the blast radius if the token is compromised.
## Troubleshooting
### "No stored token found" error
```bash
# Solution 1: Use HF_TOKEN environment variable
export HF_TOKEN=hf_aBcDeF123456
# Solution 2: Use CLI --token option
birdnet-uploader upload --repo-id user/dataset --segments ./segments --token hf_aBcDeF123456
# Solution 3: Store token via login
birdnet-uploader login
# Enter token when prompted
```
### Token validation failed
1. Verify token format starts with `hf_`
2. Check token hasn't expired (renew at https://huggingface.co/settings/tokens)
3. Verify token has `repo.content.write` permission on target dataset
4. Test token with: `curl -H "Authorization: Bearer hf_aBcDeF123456" https://huggingface.co/api/user`
### Token leaks in logs
The CLI is designed never to log or display tokens. If you see a token in logs:
1. **Never commit the leaked token** β regenerate immediately
2. **Rotate the token** at https://huggingface.co/settings/tokens
3. **Report to Hugging Face** if suspicious activity detected
## Environment Variable Reference
| Variable | Purpose | Default | Example |
|----------|---------|---------|---------|
| `HF_TOKEN` | Authentication token | (not set) | `hf_aBcDeF123456` |
| `BIRDNET_UPLOADER_DATA_DIR` | Session/cache root (container) | `~/.birdnet-uploader` | `C:\data` |
| `BIRDNET_UPLOADER_SESSION_DIR` | Override session directory | `$DATA_DIR/sessions` | `/tmp/sessions` |
| `BIRDNET_UPLOADER_CACHE_DIR` | Override cache directory | `$DATA_DIR/cache` | `/tmp/cache` |
## FAQ
**Q: Is the token stored after using `--token` CLI option?**
A: No. The `--token` option is used only for that command. Use `birdnet-uploader login` to store permanently.
**Q: Can I use different tokens for different datasets?**
A: Currently, only one token is stored in keyring. For multiple datasets, use `HF_TOKEN` env var or `--token` option per command.
**Q: What happens if I expose HF_TOKEN in docker run history?**
A: It's visible in `docker history` and shell history. Use Docker secrets or `.env` files with `docker-compose` for better isolation.
**Q: How do I rotate tokens without stopping the container?**
A: Set a new `HF_TOKEN` env var and restart the container, or pass `--token` directly to each command.
**Q: Are tokens encrypted in keyring storage?**
A: Yes. Keyring uses OS-level credential storage:
- **Windows**: Credential Manager (DPAPI encrypted)
- **macOS**: Keychain (encrypted)
- **Linux**: Secret Service or pass (encrypted, if backend supports it)
|