AGENTS.md

AGENTS.md — ReClip Telegram Bot

Compact instruction file for OpenCode sessions working in this repository.

Project Overview

Self-hosted Telegram bot that downloads media from YouTube, TikTok, Instagram, and 1000+ other sites. Built from four Dockerized Python services:

Service	Tech	Port	Role
reclip/	Flask + yt-dlp	8899	Download engine with REST API
bot/	python-telegram-bot + httpx	—	Telegram bot client
dashboard/	FastAPI + aiosqlite + Jinja2	8080	Admin panel with stats
telegram-bot-api	aiogram/telegram-bot-api	8081	Self-hosted Bot API for 2GB uploads

Running Locally (No Docker)

Start services in this order — each needs its own terminal:

# 1. Reclip (download engine)
cd reclip
pip install flask yt-dlp
python app.py                      # listens on 8899

# 2. Dashboard (admin panel)
cd dashboard
pip install -r requirements.txt
ADMIN_PASSWORD=changeme DB_PATH=./reclip.db \
  uvicorn main:create_app --factory --host 0.0.0.0 --port 8080

# 3. Bot (Telegram client)
cd bot
pip install -r requirements.txt
BOT_TOKEN=<token> RECLIP_URL=http://localhost:8899 \
  DOWNLOADS_PATH=../reclip/downloads python bot.py

The bot waits up to 60s for the self-hosted Bot API server on startup. In Docker it talks to http://telegram-bot-api:8081; locally you need that container running too, or the bot will warn and start anyway (uploads >20MB will fail).

Running Tests

# Bot tests
cd bot
pip install -r requirements.txt
pip install pytest pytest-asyncio
python -m pytest tests/ -v

# Dashboard tests
cd dashboard
pip install -r requirements.txt
pip install pytest pytest-asyncio httpx
python -m pytest tests/ -v

Critical: Dashboard tests must set DB_PATH, ADMIN_PASSWORD, and SECRET_KEY before importing db.py or main.py. The test files do this via os.environ at the top of the file. If you add new dashboard test files, copy that pattern or imports will create the DB in the wrong place.

Architecture Notes

Reclip API Surface (reclip/app.py)

• POST /api/info — returns title, thumbnail, duration, uploader, extractor, and available quality formats
• POST /api/download — spawns a threaded yt-dlp download, returns job_id
• GET /api/status/<job_id> — returns status (downloading | done | error), progress dict, file metadata
• GET /api/file/<job_id> — serves finished file as attachment

Bot → Dashboard Events (bot/event_client.py)

The bot sends fire-and-forget HTTP POSTs to DASHBOARD_URL/api/events with type: download_start | progress | download_done | download_error. These are best-effort; failures are logged at DEBUG and ignored.

Download Flow

1. User sends URL → bot calls reclip_client.get_info()
2. Bot shows thumbnail + format buttons (MP4 / MP3)
3. MP4 path shows quality picker; MP3 path starts audio download directly
4. Bot calls start_download() → polls poll_status() every 2s
5. Progress updates edit the Telegram message in real time
6. On status: done, bot uploads the file via the self-hosted Bot API

Post-Processing in Reclip

• yt-dlp downloads to DOWNLOADS_PATH/<job_id>.<ext>
• AV1/VP9/VP8 videos are transcoded to H264 via ffmpeg (Telegram compatibility)
• Other MP4s get -movflags +faststart via ffmpeg copy
• File duration/width/height extracted with ffprobe

Concurrency & Limits

• Reclip uses threading.Semaphore(MAX_CONCURRENT_DOWNLOADS) (default 3)
• Each download has a hard timeout of DOWNLOAD_TIMEOUT seconds (default 900)
• Bot has a 10-minute in-memory session TTL (STATE_TTL = 600)

Cleanup (bot/cleanup.py)

Background asyncio task that runs every CLEANUP_INTERVAL_SECONDS (default 300):

1. Deletes files older than CLEANUP_MAX_AGE_HOURS (default 1)
2. If disk usage exceeds CLEANUP_MAX_DISK_MB (default 5000), deletes oldest files first

Environment Variables

Required:

• BOT_TOKEN — from @BotFather
• TELEGRAM_API_ID / TELEGRAM_API_HASH — from my.telegram.org
• DASHBOARD_PASSWORD — dashboard login password

Commonly overridden for local dev:

• RECLIP_URL=http://localhost:8899
• TELEGRAM_BOT_API_URL=http://localhost:8081
• DOWNLOADS_PATH=./downloads
• DB_PATH=./reclip.db
• SECRET_KEY=change-me (dashboard cookie signing)

See .env.example for the full list with defaults.

Code Conventions

• No linting / formatting / typechecking tooling is configured. There is no black, ruff, mypy, or pre-commit setup.
• Imports use sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) in test files to reach parent modules.
• Dashboard DB uses SQLite with WAL mode (PRAGMA journal_mode=WAL).
• Auth is session-cookie based (itsdangerous URLSafeTimedSerializer), not JWT or OAuth.
• All event_client calls are fire-and-forget with a 2-second timeout; they must never crash the bot.

Common Gotchas

• Shared volume: downloads is a Docker shared volume between reclip, bot, and dashboard. Locally, all three services must point to the same filesystem path.
• Bot API server first startup: The self-hosted Bot API downloads Telegram server data on first run. This can take 30–60 seconds. The bot polls wait_for_bot_api() for up to 60s.
• Dashboard DB path: _db_path() re-reads DB_PATH from env on every call so tests can override it dynamically.
• Reclip job state is in-memory: jobs = {} in reclip/app.py is not persistent. Restarting reclip loses active download state.
• FFmpeg dependency: The reclip Dockerfile installs ffmpeg. Locally you need it installed (apt install ffmpeg or equivalent).