Spaces:
Paused
Paused
Commit History
-
Integrate mistral-common for correct Devstral tokenization
gary-boon +1: Claude Opus 4.5Root cause: Devstral's Tekken tokenizer is incompatible with HuggingFace's standard tokenization. When [INST]/[/INST] are formatted as text and tokenized with HF, the model receives corrupted tokens -> garbage output. Solution: - Add mistral-common>=1.5.0 dependency - Create MistralTokenizerWrapper using MistralTokenizer.from_hf_hub() - Use encode_chat_completion() for correct Tekken token encoding - Relax numpy/pydantic version constraints for compatibility The mistral-common library produces correct token sequences by encoding chat messages directly to token IDs, bypassing text-based formatting. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>ed06dcb -
Remove mistral_common to fix dependency conflict
gary-boon +1: Claude Opus 4.5- Remove mistral_common from requirements (conflicts with numpy/pydantic) - Simplify prompt_formatter to use manual Mistral format - Format: [INST] {system}\n\n{user} [/INST] (no <s>, tokenizer adds BOS) - Keep recommended_temperature=0.15 for Devstral 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>3d9d9ee -
Use mistral_common for proper Devstral prompt formatting
gary-boon +1: Claude Opus 4.5- Add mistral_common>=1.5.0 dependency - Update prompt_formatter to use MistralTokenizer for Devstral - Add recommended_temperature to model configs (0.15 for Devstral) - Remove hardcoded <s> from manual format (tokenizer adds BOS) The mistral_common library provides the correct chat template encoding that Devstral expects, which should fix the garbage token output. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>3e80769 -
Add system prompt support for instruction-tuned models
gary-boon +1: Claude Opus 4.5- Add prompt_style and system_prompt fields to ModelConfig - Create prompt_formatter.py service for unified prompt handling - Update research endpoint to use formatter with proper system prompts - Devstral now receives proper system + user message format This fixes the garbage token output from Devstral by properly formatting prompts for instruction-tuned models. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>2860768 -
fix: Simpler prompt format and temperature=0 for Devstral
gary-boon +1: Claude Opus 4.5- Remove newlines from instruction (they become <SPECIAL_*> tokens) - Add explicit <s> BOS token to manual format - Use temperature=0 for fully deterministic output - Clearer instruction: "Continue this Python code:" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>76020ee -
fix: Sanitize JSON response for NaN/Inf float values
gary-boon +1: Claude Opus 4.5Add sanitize_for_json() helper that recursively traverses response and replaces NaN/Infinity values with null. Fixes JSON serialization errors when attention/entropy calculations produce invalid floats. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>99f6209 -
fix: Check chat_template is set before using apply_chat_template
gary-boon +1: Claude Opus 4.5The tokenizer has the apply_chat_template method but no template configured. Now we check tokenizer.chat_template is not None before attempting to use it, falling back to manual [INST] format. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>474927d -
fix: Add chat template support for Devstral instruct model
gary-boon +1: Claude Opus 4.5- Add uses_chat_template field to model configs - Apply tokenizer.apply_chat_template() for instruct models - Fallback to manual [INST] format if needed - Lower temperature to 0.15 for instruct models (better code output) Fixes garbage token generation when using Devstral without proper instruction formatting. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>8d85da8
-
fix: Convert bfloat16 to float32 for numpy compatibility
gary-boon +1: Claude Opus 4.5NumPy doesn't support bfloat16 directly. Add .float() conversion before .numpy() calls in attention/activation extraction. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>cb6f39c -
fix: Use eager attention for output_attentions support
gary-boon +1: Claude Opus 4.5Devstral defaults to SDPA (flash attention) which doesn't support output_attentions=True. Force eager attention to enable capturing attention weights for visualization. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>5333b21 -
fix: Skip heavy ML deps in CI security checks
gary-boon +1: Claude Opus 4.5Security scanning tools (Snyk, Bandit, Safety) don't need torch/ transformers installed - they do static analysis of code and requirements.txt. Installing torch>=2.3 with CUDA deps was exhausting CI disk space (~3GB+). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>ba27c0c -
fix: Update torch to 2.3+ for transformers compatibility
gary-boon +1: Claude Opus 4.5transformers 4.44+ requires torch.utils._pytree.register_pytree_node which was added in torch 2.3.0. Old torch 2.1.0 only had the underscore-prefixed _register_pytree_node. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>1b73605 -
fix: Update transformers for Devstral support
gary-boon +1: Claude Opus 4.5Devstral-Small requires transformers>=4.44 for proper Mistral architecture support. Old version (4.36.2) caused shape mismatch errors during model loading. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>b788304 -
docs: Mark GPU HF Space Devstral deployment complete
gary-boon +1: Claude Opus 4.5- A100 (80GB VRAM) configured - DEFAULT_MODEL=devstral-small set - Phase 2c fully complete 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>65c6e2e -
docs: Update phased plan with Phase 2/2b/2c completion status
gary-boon +1: Claude Opus 4.5- Mark Phase 2 (Devstral backend support) as COMPLETE - Mark Phase 2b (Frontend dynamic layers) as COMPLETE - Mark Phase 2c (API route conversion) as COMPLETE (partial) - Note Spark toggle deferred until PyTorch GB10 support - Update tag references 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>688efad -
Add vocabSize to modelInfo response
gary-boon +1: Claude Opus 4.5Frontend needs vocab size to display dynamically instead of hardcoded "51,200 tokens". Added vocabSize to the modelInfo object in the research attention analysis response. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>499afba -
Update .env.spark.example: TORCH_DTYPE now auto-detected
gary-boon +1: Claude Opus 4.5TORCH_DTYPE is automatically set from model config: - codegen-350m → fp16 - devstral-small → bf16 Only uncomment to override the default. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>543454f -
Add recommended_dtype to model configs
gary-boon +1: Claude Opus 4.5Each model now specifies its recommended dtype: - codegen-350m: fp16 - code-llama-7b: fp16 - devstral-small: bf16 (required for Mistral models) Model loader now uses recommended_dtype from config when TORCH_DTYPE env var is not explicitly set. This ensures Devstral automatically loads with bf16 without requiring manual configuration. Priority: TORCH_DTYPE env > model config > device default 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>62525b2 -
Phase 2: Add Devstral backend support
gary-boon +1: Claude Opus 4.5Add support for Mistral-based models (Devstral) with: - MistralAdapter in model_adapter.py for Mistral architecture - devstral-small config (40 layers, 32 Q heads, 8 KV heads, 131K vocab) - Percentage-based layer classification (works for any layer count) - Environment variable support: - DEFAULT_MODEL: which model to load (default: codegen-350m) - TORCH_DTYPE: bf16/fp16/fp32 (default: auto based on device) - MAX_CONTEXT: context length limit (default: 8192) - BATCH_SIZE: batch size (default: 1) - GET /models endpoint: list available models with hardware availability - GET /models/current endpoint: return currently loaded model info Devstral requires TORCH_DTYPE=bf16 when deployed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>9080f28 -
Update plan: Phase 1 paused due to GB10 GPU support
gary-boon +1: Claude Opus 4.5Document the blocker with DGX Spark's GB10 GPU (sm_121 compute capability) not being supported by current PyTorch releases. - Mark Phase 0 and 0.5 as complete - Mark Phase 1 as paused (not blocked permanently) - Document what we tried and what we learned - Add clear restart instructions for when PyTorch 2.9.x adds sm_121 - List infrastructure already in place on Spark The Spark deployment makes no sense on CPU when Mac Studio and HuggingFace Spaces are available. Wait for official GPU support. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>e694533 -
Add DEVICE env var to force CPU mode on DGX Spark
gary-boon +1: Claude Opus 4.5GB10 GPU (sm_121 compute capability) is not yet supported by PyTorch/NGC containers. This adds a DEVICE environment variable override to force CPU mode until GPU support is available. - Add os import to model_service.py - Check DEVICE env var before auto-detecting device - Support DEVICE=cpu or DEVICE=cuda to override detection - Default .env.spark.example to DEVICE=cpu for Spark 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>5f122aa -
Use NGC PyTorch 24.08 for Python 3.10 compatibility
gary-boon +1: Claude Opus 4.524.11 uses Python 3.12 which breaks older packages. 24.08 has Python 3.10 which works with our requirements.txt 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>a2875a2 -
Use NVIDIA NGC PyTorch container for GB10 support
gary-boon +1: Claude Opus 4.5NGC containers are optimized for latest NVIDIA hardware including GB10. PyTorch comes pre-installed with proper CUDA/GPU support. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>a4cfbff -
Try PyTorch nightly for GB10/sm_121 GPU support
gary-boon +1: Claude Opus 4.5The NVIDIA GB10 (Grace Blackwell) requires sm_121 compute capability which isn't in stable PyTorch yet. Nightly may have it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>a009a49 -
Make zarr/numcodecs imports optional for ARM64 compatibility
gary-boon +1: Claude Opus 4.5- Use try/except for zarr imports - Add ZARR_AVAILABLE flag - Skip storage operations gracefully when zarr not available - Core model inference still works without tensor caching 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>6435a75 -
Skip zarr/numcodecs in Spark build (ARM64 incompatible)
gary-boon +1: Claude Opus 4.5These are for tensor caching, not core inference. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>d129e37 -
Fix numcodecs ARM64 compatibility in Dockerfile.spark
gary-boon +1: Claude Opus 4.5- Skip torch from requirements (already installed with CUDA) - Upgrade numcodecs to >=0.13.0 (ARM64 compatible) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>772fc80 -
Fix Dockerfile.spark for CUDA 13.0 compatibility
gary-boon +1: Claude Opus 4.5- Use nvidia/cuda:12.4.0-runtime-ubuntu22.04 base image - Install PyTorch with CUDA 12.4 wheels (compatible with CUDA 13.0 driver) - Build Python dependencies properly for ARM64 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>a4927aa -
Fix Dockerfile.spark for ARM64 architecture (DGX Spark)
gary-boon +1: Claude Opus 4.5- Use NVIDIA NGC pytorch image (nvcr.io/nvidia/pytorch:24.01-py3) - Supports ARM64/aarch64 with CUDA 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>9d00d33 -
Add GPU-enabled Dockerfile for Spark
gary-boon +1: Claude Opus 4.5- Create docker/Dockerfile.spark using pytorch/pytorch CUDA base image - Update compose.spark.yml to use Spark-specific Dockerfile - Fixes CUDA not available issue on DGX Spark 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>9377cd8 -
Fix Dockerfile: add build-essential for numcodecs compilation
gary-boon +1: Claude Opus 4.5numcodecs requires C compiler during pip install. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>3b5c3ac -
Phase 1: DGX Spark infrastructure
gary-boon +1: Claude Opus 4.5- Add .env.spark.example template for Spark configuration - Add docker/compose.spark.yml with GPU support and model cache mount - Add runs/.gitkeep for runtime outputs - Update .gitignore for .env.spark and runs/* - Add /ready endpoint (503 until model loaded, then 200) - Add /debug/device endpoint for GPU verification (no secrets) Paths follow DGX Spark deployment spec: - /srv/projects/visualisable-ai-backend - /srv/models-cache/huggingface (mounted) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>a2bd186
-
Add Devstral + DGX Spark implementation plan
gary-boon +1: Claude Opus 4.5Phased plan for: - Securing GPU HF Space - Adding Devstral model support - Frontend dynamic layer handling - DGX Spark deployment 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>ab4534a
-
Make QKV hook robust against shape mismatches
gary-boon +1: Claude- Add try-except around entire hook body - Validate tensor dimensions before reshape - Skip QKV capture silently if format doesn't match - Prevents crashes during forward pass with different model architectures 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>343dd57 -
Fix research attention endpoint model compatibility
gary-boon +1: ClaudeMake QKV hook registration more robust to support different model architectures (CodeGen vs GPT-2 style attention modules) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>f5ba954 -
Fix zarr/numcodecs version compatibility
gary-boon +1: ClaudePin zarr to 2.14.2 and numcodecs to 0.11.0 to resolve import error 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>9e9dc34 -
Add zarr to requirements.txt for storage module
gary-boon +1: Claude🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>f54e3f9 -
Add research attention analysis endpoint with real CodeGen tokenization
gary-boon +1: Claude- Implement /analyze/research/attention endpoint - Extract real token IDs from CodeGen tokenizer - Track attention weights across all 20 layers per generation step - Return top-k token alternatives with probabilities - Store per-step attention data for token-by-token analysis 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>8f63685
-
Add research attention analysis endpoints with Q/K/V extraction
gary-boon +1: Claude- Add /analyze/research/attention endpoint with layer-by-layer attention data - Implement PyTorch hooks for Q/K/V matrix extraction from qkv_proj layer - Add token-by-token generation with layersDataByStep for tracing - Add top-k token alternatives with probabilities (logprobs) - Add tokenizer utilities for vocabulary analysis - Add exploration scripts for vocabulary inspection - Return all 16 attention heads sorted by importance - Fix tensor dimension handling and NaN sanitization 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>37ed739
-
Fix ablation study for Code Llama compatibility
gary-boon +1: ClaudeFixed AttributeError in ablation generation when using Code Llama: - CodeGen uses: n_layer, n_head - Llama/Code Llama uses: num_hidden_layers, num_attention_heads Changes: - Added config attribute compatibility variables at start of ablation method - Replaced hardcoded config.n_layer and config.n_head with compatibility variables - Now handles both CodeGen and Llama model architectures This fixes the 500 error in Ablation Study when using Code Llama 7B. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>cd300ee -
Fix model info endpoint for Code Llama compatibility
gary-boon +1: ClaudeFixed AttributeError when accessing model config attributes: - CodeGen uses: n_layer, n_head, n_embd, n_positions - Llama/Code Llama uses: num_hidden_layers, num_attention_heads, hidden_size, max_position_embeddings Changes: - Use getattr() with fallbacks to handle both config schemas - Return dynamic model name from manager.model_name instead of hardcoded value - Handle different activation function and layer norm attribute names This fixes the 500 Internal Server Error on GPU backend when loading Code Llama 7B. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>7dd568f
-
Add Code Llama 7B support with hardware-aware filtering and ICL timeout fixes
gary-boon +1: Claude- Added multi-architecture support to ICL components (attention extractor, service, induction detector) - Implemented hardware-aware model filtering for CPU/GPU spaces - Fixed Code Llama tokenizer padding token configuration - Updated model config with accurate Code Llama 7B specifications - Added model adapter pattern for seamless architecture switching 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>ed40a9a
-
Add Claude Code configuration
gary-boon +1: Claude🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>03971da
-
Fix pyarrow compatibility issue with datasets library
gary-boon +1: Claude- Update datasets to 2.16.1 and pyarrow to 14.0.2 for compatibility - Add graceful error handling when dataset can't load - Return 503 error instead of crashing when dataset unavailable - Maintain research integrity - no mock data fallback 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>1680fda -
Fix syntax error in swe_bench_service.py
gary-boon +1: Claude- Remove commented-out mock data function with nested triple quotes - Triple quotes inside comments were causing Python syntax errors - Clean removal ensures no mock data and fixes deployment 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>9dbec03 -
Remove all mock data from SWE-bench - real data only
gary-boon +1: Claude- Remove _load_mock_tasks function completely - Raise exceptions when dataset unavailable instead of falling back - Ensure research integrity by rejecting mock data - Real SWE-bench dataset required for PhD research 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>c0d95bf -
Add GitHub URLs and improve mock data for SWE-bench
gary-boon +1: Claude- Generate 300 mock tasks with better difficulty distribution - Include GitHub issue/PR URLs in task data - Mark mock data with _is_mock flag for validation - Improve error messages and logging 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>22c69fa -
Fix SWE-bench service to gracefully handle dataset loading failures
gary-boon +1: Claude- Add fallback to mock data when HuggingFace dataset loading fails - Improve error handling for deployment environments - Add detailed mock problem statements for better testing - Fix initialization errors on HuggingFace Spaces 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>ae9e159 -
Fix SWE-bench service to return full problem statements
gary-boon +1: Claude- Return complete problem_statement instead of truncating at 500 chars - Enables frontend to display full multi-line issue descriptions - Fixes issue where problems were showing as one-liners 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>1d23728
-
Add SWE-bench integration and improve backend routing
gary-boon +1: Claude- Integrate SWE-bench dataset with 300 real-world GitHub issues - Add comprehensive SWE-bench evaluator UI with task selection and solution generation - Implement dynamic backend routing for CPU/GPU based on user settings - Move protected pages to (protected) folder structure for authentication - Add syntax highlighting for code displays using react-syntax-highlighter - Create confidence visualization components for transparency metrics - Fix navigation duplication issues and improve layout consistency - Add backend indicator showing which backend (Local/CPU/GPU) is being used - Implement special routing for SWE-bench to always use local backend - Add debugging and logging for backend selection - Improve error handling and user feedback 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>4444ae2