Spaces:
Running
Running
Commit History
-
Implement image support in proxy with vision-aware routing
- Add Anthropic→OpenAI image block conversion with size validation - Update ModelCapabilities with granular vision support fields - Support base64-encoded images with memory optimization - Add 20MB image size limit guard in conversion - Clear base64 data from request blocks to reduce memory footprint - Update mistral-large-3-675b model entry with vision capabilities - Enable vision-aware model routing for image requests Closes: image upload support in Claude Code proxy Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>574e4e7 -
NIM speed optimization — adaptive rate limiting and increased throughput
- Add AdaptiveRateLimiter with auto-backoff on 429s (starts at 100 req/min, backs off to 10, recovers gradually) - Increase NIM rate limit to 100 req/min with 40 max concurrency (configurable via NIM_RATE_LIMIT, NIM_MAX_CONCURRENCY) - Tighten NIM timeouts: connect=8s, first_chunk=20s, fallback_first_chunk=12s - Lock-free fast path in StrictSlidingWindowLimiter (reduces contention under load) - Faster retry delays: base=0.3s, max=20s, jitter=0.1s - Max out HTTP connection pool: 100 keepalive, 500 connections, 5s expiry - Aggressive HTTP pool warmup on startup (forces TCP+TLS connection establishment) - Record 429s to ModelHealthTracker for adaptive recovery - Update CLAUDE.md with per-provider health params and session cleanup notes Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>aa9c0b0
-
Speed up NIM provider with failure tracking and faster timeouts
- Add ModelHealthTracker for per-model failure tracking (30s TTL) - Faster timeouts: connect=10s, first_chunk=30s, fallback=20s - Auto model pre-checks health before attempting requests - Record failures on timeout/rate-limit for smart fallback - Skip unhealthy models in both router and services Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>a5ea640
-
Speed optimizations and enhanced auto-model fallback routing
- Connection pool tuning: increased max_connections (100→200), max_keepalive (20→50), reduced keepalive_expiry (30s→15s) - NIM concurrency: increased max_concurrency for better throughput - Faster timeouts: connect (30→15s), first_chunk (60→45s), fallback (60→30s) - Retry backoff: reduced base_delay (1→0.5s), max_delay (60→30s) - New trivial request fast-path for hi/ok/status checks - Auto model now routes to all 7 NIM models by default - Pre-flight rate limit check before trying each candidate - Timeout/transient errors trigger automatic fallback to next model - Import cleanup for httpx/asyncio at module level Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>ebba9d6 -
Fix: sessions tracking now works correctly in dashboard and JSON
- Added `track_request` calls in services.py for both single-model and fallback paths - Sessions now show in `/admin` and `/api/admin/sessions` - Added `/` root endpoint to show active sessions count Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>ef123a8 -
Deploy claude-code-nvidia proxy to Hugging Face Spaces
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>0157ac7