Spaces:
Sleeping
Download docs/USAGE.md from jrrribeiro/BirdNET-Uploader-App: direct link, hf CLI and curl.
- Browser
- Download file 9.13 kB
-
https://huggingface.co/spaces/jrrribeiro/BirdNET-Uploader-App/resolve/main/docs/USAGE.md
- Command line
-
hf download hf://spaces/jrrribeiro/BirdNET-Uploader-App/docs/USAGE.md
-
curl -L -o USAGE.md https://huggingface.co/spaces/jrrribeiro/BirdNET-Uploader-App/resolve/main/docs/USAGE.md
A newer version of the Gradio SDK is available: 6.32.0
BirdNET Uploader Usage Guide
This guide covers how to download, install, and use the BirdNET Uploader CLI for uploading audio datasets to Hugging Face.
Installation
Option 1: Windows Standalone (Recommended)
- Download the latest release from Releases
- Extract the ZIP file to a folder
- No Python installation required!
- Run:
birdnet-uploader.exe --help
Option 2: Python + Pip
pip install birdnet-uploader
birdnet-uploader --help
Option 3: Docker
docker pull birdnet-uploader:latest
docker run --rm \
-e HF_TOKEN="hf_your_token" \
-v C:\segments:C:\data\segments \
birdnet-uploader:latest \
upload --repo-id user/dataset --segments C:\data\segments
Option 4: Build from Source
git clone https://github.com/jrrribeiro/BirdNET-Uploader-App
cd BirdNET-Uploader-App
pip install -r requirements.txt
python -m src.uploader_cli.main --help
Quick Start (5 Minutes)
Step 1: Authenticate
Store your Hugging Face token securely:
birdnet-uploader login
# Follow the prompt to enter your token
# Token is stored securely in your OS credential manager
Step 2: Create/Validate Dataset
birdnet-uploader init-repo --repo-id username/my-dataset --private
# Creates or validates the dataset exists
Step 3: Prepare Audio Files
Organize your audio files in a folder:
C:\my-segments\
βββ species-A\
β βββ recording-001.wav
β βββ recording-002.wav
β βββ ...
βββ species-B\
β βββ recording-003.wav
β βββ ...
βββ detections.csv (optional)
Step 4: Dry Run (Recommended)
Preview what will be uploaded without actually uploading:
birdnet-uploader upload \
--repo-id username/my-dataset \
--segments C:\my-segments \
--dry-run
Step 5: Upload
birdnet-uploader upload \
--repo-id username/my-dataset \
--segments C:\my-segments \
--csv detections.csv
Done! Audio files are now in audio/ and CSV is in index/detections.csv.
Common Workflows
Upload with Progress Tracking
birdnet-uploader upload \
--repo-id username/my-dataset \
--segments C:\my-segments \
--workers 4
The --workers option enables parallel uploads (faster on multi-core systems).
Resume Interrupted Upload
# Check existing sessions
ls ~/.birdnet-uploader/sessions/
# Resume a specific session
birdnet-uploader resume upload-20260429T120000Z
# Or start fresh
birdnet-uploader upload --repo-id username/my-dataset --segments C:\my-segments
Upload with Custom Token (CI/CD)
birdnet-uploader upload \
--repo-id username/my-dataset \
--segments C:\my-segments \
--token hf_your_token
Batch Upload Multiple Folders
$folders = @("C:\data\2024", "C:\data\2025")
foreach ($folder in $folders) {
birdnet-uploader upload `
--repo-id username/my-dataset `
--segments $folder
}
Upload Only CSV (Index Only)
birdnet-uploader upload \
--repo-id username/my-dataset \
--segments C:\dummy-empty \
--csv detections.csv \
--dry-run
Token Management
Method 1: Stored Token (Recommended for Local Use)
# Store token securely (OS Credential Manager)
birdnet-uploader login
# Use stored token automatically
birdnet-uploader upload --repo-id username/my-dataset --segments C:\segments
Method 2: Environment Variable (Docker/CI)
# Linux/macOS
export HF_TOKEN="hf_..."
birdnet-uploader upload ...
# Windows PowerShell
$env:HF_TOKEN = "hf_..."
birdnet-uploader.exe upload ...
# Windows CMD
set HF_TOKEN=hf_...
birdnet-uploader.exe upload ...
Method 3: CLI Option (One-Off)
birdnet-uploader upload \
--repo-id username/my-dataset \
--segments C:\segments \
--token hf_...
Advanced Usage
Parallel Uploads
Speed up uploads on multi-core systems:
birdnet-uploader upload \
--repo-id username/my-dataset \
--segments C:\segments \
--workers 8
Recommended values:
- 4-8 workers for residential/office internet
- 16-32 workers for datacenter/gigabit internet
Custom Dataset Path
Upload to a subdirectory in the dataset:
birdnet-uploader upload \
--repo-id username/my-dataset \
--segments C:\segments \
--remote-base audio/my-collection
Files uploaded to: audio/my-collection/... instead of audio/...
Session Management
# List all sessions
ls ~/.birdnet-uploader/sessions/
# View session checkpoint
birdnet-uploader resume upload-20260429T120000Z
# Remove old session (careful!)
rm -r ~/.birdnet-uploader/sessions/upload-20260101T000000Z
Clear Deduplication Cache
If you need to re-upload files that were marked as already uploaded:
# Remove cache (forces re-upload check)
rm -r ~/.birdnet-uploader/cache/
# Then retry upload
birdnet-uploader upload --repo-id username/my-dataset --segments C:\segments
Troubleshooting
"No stored token found"
Problem: You haven't logged in yet.
Solution:
birdnet-uploader login
# Enter your Hugging Face token when prompted
"Token validation failed"
Problem: Token is invalid or expired.
Solution:
- Go to https://huggingface.co/settings/tokens
- Create a new token
- Run
birdnet-uploader loginand enter the new token - Or use
--token hf_new_tokendirectly
"Dataset not found"
Problem: Dataset doesn't exist or you don't have access.
Solution:
# Create the dataset
birdnet-uploader init-repo --repo-id username/my-dataset
# Or verify access
curl -H "Authorization: Bearer hf_..." \
https://huggingface.co/api/datasets/username/my-dataset
"Upload stuck or slow"
Problem: Network issues or single-threaded upload.
Solution:
# Use parallel workers
birdnet-uploader upload \
--repo-id username/my-dataset \
--segments C:\segments \
--workers 4
"DLL not found" (Windows executable)
Problem: Missing runtime dependencies in bundled version.
Solution:
- Download latest release (should include all DLLs)
- Or run from Python:
python -m src.uploader_cli.main upload ... - Or report issue on GitHub
Out of Disk Space
Problem: Destination dataset is full.
Solution:
- Check dataset size:
huggingface_hub - Delete old uploads if not needed
- Create a new dataset:
birdnet-uploader init-repo --repo-id username/my-dataset-2
Dataset Structure
After upload, your Hugging Face dataset will look like:
my-dataset/
βββ audio/
β βββ species-A/
β β βββ recording-001.wav
β β βββ recording-002.wav
β β βββ ...
β βββ species-B/
β βββ recording-003.wav
β βββ ...
βββ index/
β βββ manifest.json # Index metadata
β βββ detections.csv # Your CSV (if provided)
β βββ shards/
β βββ detections_0.parquet
β βββ detections_1.parquet
βββ README.md
Performance Tips
- Use SSD for local segments: Faster scanning and hashing
- Enable parallel uploads:
--workers 4or higher - Stable network: Wired internet > WiFi
- Large batches: Consider
--batch-size 100for many files - Monitor system resources: CPU and memory usage during upload
Security
- Never hardcode tokens: Use
birdnet-uploader loginorHF_TOKENenv var - Rotate tokens: Generate new token every 3-6 months at https://huggingface.co/settings/tokens
- Limit token scope: Use fine-grained tokens with minimal permissions
- Don't share tokens: Each user should have their own token
- Use dataset privacy: Set
--privatewhen creating repos with sensitive data
Getting Help
- Check TOKEN_SECURITY.md for token security best practices
- Check TOKEN_USAGE_EXAMPLES.md for Docker/CI/CD examples
- Check CI_CD_PIPELINE.md for build and release info
- Report issues: Create a GitHub issue with error logs and reproduction steps
FAQ
Q: Can I cancel an upload? A: Yes, press Ctrl+C. The session checkpoint saves progress, so you can resume later with the same command.
Q: How large can the audio collection be? A: Hugging Face datasets support large collections (100GB+). Upload speed depends on your internet.
Q: Can multiple users upload to the same dataset? A: Yes, but they need write access. Use separate tokens for each user to track uploads separately.
Q: Does it support audio formats other than WAV?
A: Yes: WAV, MP3, FLAC, OGG, M4A. Extend AUDIO_EXTENSIONS in src/uploader_cli/config.py if needed.
Q: Can I upload without audio files (CSV only)?
A: Yes, but you need to provide a dummy --segments directory (can be empty).
Q: How do I verify upload integrity? A: The uploader computes SHA-256 hashes and deduplicates against remote index. Files are verified on upload.