Spaces:
Paused
Paused
Commit History
-
Integrate mistral-common for correct Devstral tokenization
gary-boon +1: Claude Opus 4.5Root cause: Devstral's Tekken tokenizer is incompatible with HuggingFace's standard tokenization. When [INST]/[/INST] are formatted as text and tokenized with HF, the model receives corrupted tokens -> garbage output. Solution: - Add mistral-common>=1.5.0 dependency - Create MistralTokenizerWrapper using MistralTokenizer.from_hf_hub() - Use encode_chat_completion() for correct Tekken token encoding - Relax numpy/pydantic version constraints for compatibility The mistral-common library produces correct token sequences by encoding chat messages directly to token IDs, bypassing text-based formatting. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>ed06dcb -
Remove mistral_common to fix dependency conflict
gary-boon +1: Claude Opus 4.5- Remove mistral_common from requirements (conflicts with numpy/pydantic) - Simplify prompt_formatter to use manual Mistral format - Format: [INST] {system}\n\n{user} [/INST] (no <s>, tokenizer adds BOS) - Keep recommended_temperature=0.15 for Devstral 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>3d9d9ee -
Use mistral_common for proper Devstral prompt formatting
gary-boon +1: Claude Opus 4.5- Add mistral_common>=1.5.0 dependency - Update prompt_formatter to use MistralTokenizer for Devstral - Add recommended_temperature to model configs (0.15 for Devstral) - Remove hardcoded <s> from manual format (tokenizer adds BOS) The mistral_common library provides the correct chat template encoding that Devstral expects, which should fix the garbage token output. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>3e80769 -
Add system prompt support for instruction-tuned models
gary-boon +1: Claude Opus 4.5- Add prompt_style and system_prompt fields to ModelConfig - Create prompt_formatter.py service for unified prompt handling - Update research endpoint to use formatter with proper system prompts - Devstral now receives proper system + user message format This fixes the garbage token output from Devstral by properly formatting prompts for instruction-tuned models. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>2860768 -
fix: Simpler prompt format and temperature=0 for Devstral
gary-boon +1: Claude Opus 4.5- Remove newlines from instruction (they become <SPECIAL_*> tokens) - Add explicit <s> BOS token to manual format - Use temperature=0 for fully deterministic output - Clearer instruction: "Continue this Python code:" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>76020ee -
fix: Sanitize JSON response for NaN/Inf float values
gary-boon +1: Claude Opus 4.5Add sanitize_for_json() helper that recursively traverses response and replaces NaN/Infinity values with null. Fixes JSON serialization errors when attention/entropy calculations produce invalid floats. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>99f6209 -
fix: Check chat_template is set before using apply_chat_template
gary-boon +1: Claude Opus 4.5The tokenizer has the apply_chat_template method but no template configured. Now we check tokenizer.chat_template is not None before attempting to use it, falling back to manual [INST] format. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>474927d -
fix: Add chat template support for Devstral instruct model
gary-boon +1: Claude Opus 4.5- Add uses_chat_template field to model configs - Apply tokenizer.apply_chat_template() for instruct models - Fallback to manual [INST] format if needed - Lower temperature to 0.15 for instruct models (better code output) Fixes garbage token generation when using Devstral without proper instruction formatting. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>8d85da8
-
fix: Convert bfloat16 to float32 for numpy compatibility
gary-boon +1: Claude Opus 4.5NumPy doesn't support bfloat16 directly. Add .float() conversion before .numpy() calls in attention/activation extraction. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>cb6f39c -
fix: Use eager attention for output_attentions support
gary-boon +1: Claude Opus 4.5Devstral defaults to SDPA (flash attention) which doesn't support output_attentions=True. Force eager attention to enable capturing attention weights for visualization. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>5333b21 -
Add vocabSize to modelInfo response
gary-boon +1: Claude Opus 4.5Frontend needs vocab size to display dynamically instead of hardcoded "51,200 tokens". Added vocabSize to the modelInfo object in the research attention analysis response. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>499afba -
Add recommended_dtype to model configs
gary-boon +1: Claude Opus 4.5Each model now specifies its recommended dtype: - codegen-350m: fp16 - code-llama-7b: fp16 - devstral-small: bf16 (required for Mistral models) Model loader now uses recommended_dtype from config when TORCH_DTYPE env var is not explicitly set. This ensures Devstral automatically loads with bf16 without requiring manual configuration. Priority: TORCH_DTYPE env > model config > device default 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>62525b2 -
Phase 2: Add Devstral backend support
gary-boon +1: Claude Opus 4.5Add support for Mistral-based models (Devstral) with: - MistralAdapter in model_adapter.py for Mistral architecture - devstral-small config (40 layers, 32 Q heads, 8 KV heads, 131K vocab) - Percentage-based layer classification (works for any layer count) - Environment variable support: - DEFAULT_MODEL: which model to load (default: codegen-350m) - TORCH_DTYPE: bf16/fp16/fp32 (default: auto based on device) - MAX_CONTEXT: context length limit (default: 8192) - BATCH_SIZE: batch size (default: 1) - GET /models endpoint: list available models with hardware availability - GET /models/current endpoint: return currently loaded model info Devstral requires TORCH_DTYPE=bf16 when deployed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>9080f28 -
Add DEVICE env var to force CPU mode on DGX Spark
gary-boon +1: Claude Opus 4.5GB10 GPU (sm_121 compute capability) is not yet supported by PyTorch/NGC containers. This adds a DEVICE environment variable override to force CPU mode until GPU support is available. - Add os import to model_service.py - Check DEVICE env var before auto-detecting device - Support DEVICE=cpu or DEVICE=cuda to override detection - Default .env.spark.example to DEVICE=cpu for Spark 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>5f122aa -
Make zarr/numcodecs imports optional for ARM64 compatibility
gary-boon +1: Claude Opus 4.5- Use try/except for zarr imports - Add ZARR_AVAILABLE flag - Skip storage operations gracefully when zarr not available - Core model inference still works without tensor caching 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>6435a75 -
Phase 1: DGX Spark infrastructure
gary-boon +1: Claude Opus 4.5- Add .env.spark.example template for Spark configuration - Add docker/compose.spark.yml with GPU support and model cache mount - Add runs/.gitkeep for runtime outputs - Update .gitignore for .env.spark and runs/* - Add /ready endpoint (503 until model loaded, then 200) - Add /debug/device endpoint for GPU verification (no secrets) Paths follow DGX Spark deployment spec: - /srv/projects/visualisable-ai-backend - /srv/models-cache/huggingface (mounted) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>a2bd186
-
Make QKV hook robust against shape mismatches
gary-boon +1: Claude- Add try-except around entire hook body - Validate tensor dimensions before reshape - Skip QKV capture silently if format doesn't match - Prevents crashes during forward pass with different model architectures 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>343dd57 -
Fix research attention endpoint model compatibility
gary-boon +1: ClaudeMake QKV hook registration more robust to support different model architectures (CodeGen vs GPT-2 style attention modules) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>f5ba954 -
Add research attention analysis endpoint with real CodeGen tokenization
gary-boon +1: Claude- Implement /analyze/research/attention endpoint - Extract real token IDs from CodeGen tokenizer - Track attention weights across all 20 layers per generation step - Return top-k token alternatives with probabilities - Store per-step attention data for token-by-token analysis 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>8f63685
-
Add research attention analysis endpoints with Q/K/V extraction
gary-boon +1: Claude- Add /analyze/research/attention endpoint with layer-by-layer attention data - Implement PyTorch hooks for Q/K/V matrix extraction from qkv_proj layer - Add token-by-token generation with layersDataByStep for tracing - Add top-k token alternatives with probabilities (logprobs) - Add tokenizer utilities for vocabulary analysis - Add exploration scripts for vocabulary inspection - Return all 16 attention heads sorted by importance - Fix tensor dimension handling and NaN sanitization 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>37ed739
-
Fix ablation study for Code Llama compatibility
gary-boon +1: ClaudeFixed AttributeError in ablation generation when using Code Llama: - CodeGen uses: n_layer, n_head - Llama/Code Llama uses: num_hidden_layers, num_attention_heads Changes: - Added config attribute compatibility variables at start of ablation method - Replaced hardcoded config.n_layer and config.n_head with compatibility variables - Now handles both CodeGen and Llama model architectures This fixes the 500 error in Ablation Study when using Code Llama 7B. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>cd300ee -
Fix model info endpoint for Code Llama compatibility
gary-boon +1: ClaudeFixed AttributeError when accessing model config attributes: - CodeGen uses: n_layer, n_head, n_embd, n_positions - Llama/Code Llama uses: num_hidden_layers, num_attention_heads, hidden_size, max_position_embeddings Changes: - Use getattr() with fallbacks to handle both config schemas - Return dynamic model name from manager.model_name instead of hardcoded value - Handle different activation function and layer norm attribute names This fixes the 500 Internal Server Error on GPU backend when loading Code Llama 7B. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>7dd568f
-
Add Code Llama 7B support with hardware-aware filtering and ICL timeout fixes
gary-boon +1: Claude- Added multi-architecture support to ICL components (attention extractor, service, induction detector) - Implemented hardware-aware model filtering for CPU/GPU spaces - Fixed Code Llama tokenizer padding token configuration - Updated model config with accurate Code Llama 7B specifications - Added model adapter pattern for seamless architecture switching 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>ed40a9a
-
Fix pyarrow compatibility issue with datasets library
gary-boon +1: Claude- Update datasets to 2.16.1 and pyarrow to 14.0.2 for compatibility - Add graceful error handling when dataset can't load - Return 503 error instead of crashing when dataset unavailable - Maintain research integrity - no mock data fallback 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>1680fda -
Fix syntax error in swe_bench_service.py
gary-boon +1: Claude- Remove commented-out mock data function with nested triple quotes - Triple quotes inside comments were causing Python syntax errors - Clean removal ensures no mock data and fixes deployment 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>9dbec03 -
Remove all mock data from SWE-bench - real data only
gary-boon +1: Claude- Remove _load_mock_tasks function completely - Raise exceptions when dataset unavailable instead of falling back - Ensure research integrity by rejecting mock data - Real SWE-bench dataset required for PhD research 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>c0d95bf -
Add GitHub URLs and improve mock data for SWE-bench
gary-boon +1: Claude- Generate 300 mock tasks with better difficulty distribution - Include GitHub issue/PR URLs in task data - Mark mock data with _is_mock flag for validation - Improve error messages and logging 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>22c69fa -
Fix SWE-bench service to gracefully handle dataset loading failures
gary-boon +1: Claude- Add fallback to mock data when HuggingFace dataset loading fails - Improve error handling for deployment environments - Add detailed mock problem statements for better testing - Fix initialization errors on HuggingFace Spaces 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>ae9e159 -
Fix SWE-bench service to return full problem statements
gary-boon +1: Claude- Return complete problem_statement instead of truncating at 500 chars - Enables frontend to display full multi-line issue descriptions - Fixes issue where problems were showing as one-liners 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>1d23728
-
Add SWE-bench integration and improve backend routing
gary-boon +1: Claude- Integrate SWE-bench dataset with 300 real-world GitHub issues - Add comprehensive SWE-bench evaluator UI with task selection and solution generation - Implement dynamic backend routing for CPU/GPU based on user settings - Move protected pages to (protected) folder structure for authentication - Add syntax highlighting for code displays using react-syntax-highlighter - Create confidence visualization components for transparency metrics - Fix navigation duplication issues and improve layout consistency - Add backend indicator showing which backend (Local/CPU/GPU) is being used - Implement special routing for SWE-bench to always use local backend - Add debugging and logging for backend selection - Improve error handling and user feedback 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>4444ae2
-
Add layer_stride parameter for PromptDiff optimization
gary-boon +1: Claude- Add layer_stride parameter to control which layers are captured - Default to 1 (all layers) for AttentionExplorer - PromptDiff can use layer_stride=2 for every other layer - Reduces matrix count from 20 to 10 for better visualization fit 🤖 Generated with Claude Code (https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>5aed1a9
-
Capture complete attention patterns after generation
gary-boon- Modified attention capture to occur after all tokens are generated - Added tokens field to TraceData model for token display - Increased sampling threshold from 20 to 100 tokens - Capture all layers instead of sampling every Nth layer - Include full token list (prompt + generated) in attention traces - This ensures complete attention matrices that match token count992dc8c
-
Fix: Use scaling approach instead of skipping layers
gary-boon +1: Claude- Changed strategy: scale down layer output by 99.9% instead of skipping - This maintains exact format compatibility - Avoids tuple/tensor mismatch issues entirely - Layer still runs but contributes only 0.1% to output This simpler approach should work reliably with any transformer version since we're not trying to bypass the normal data flow. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>3c774b5 -
Fix: Refine layer hook output format handling
gary-boon +1: Claude- Simplified logic to match exact output structure - Ensure compatibility with layer_norm expectations - Handle all tuple/tensor cases properly Testing different approach to prevent layer_norm type errors. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>4b03268 -
Fix: Handle single-element tuple outputs in layer hook
gary-boon +1: Claude- Fixed "tuple index out of range" when output has only one element - Properly checks tuple length before accessing output[1:] - Handles all output format cases correctly The previous fix assumed output tuples always had multiple elements, but some layers return single-element tuples. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>9e42df9
-
Fix: Correct layer hook output format for layer_norm compatibility
gary-boon +1: Claude- Fixed "layer_norm() argument 'input' must be Tensor, not tuple" error - Hook now properly handles different input/output formats - Maintains compatibility with subsequent layers expecting tensors The previous fix was returning tuples when tensors were expected, breaking layer_norm operations after skipped layers. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>070f9b8 -
Fix: Prevent hook persistence after ablation errors
gary-boon +1: Claude- Fixed tuple index error in layer hook when input format varies - Added try-finally block to ensure hooks are ALWAYS removed - Fixed indentation in generation loop - Prevents 500 errors from persisting after failed ablation The issue was that when ablation hooks failed, they weren't removed, causing ALL subsequent generation calls to fail until Space restart. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>3ee2b4b
-
Simplify workflow: Remove automated HuggingFace deployment
gary-boon- Renamed workflow to 'Security Check' since deployment is manual - Removed deploy-to-huggingface job - Keeps security scanning with Snyk, Bandit, and Safety - Manual deployment to HuggingFace continues as beforee6aff69
-
feat: Add pipeline analyzer and QKV extractor for transformer visualization
gary-boon +1: ClaudeBackend Components: - TransformerPipelineAnalyzer: Captures all intermediate transformer states - Real tokenization with actual token IDs - Embedding extraction with position encodings - Attention weight extraction from QKV projections - FFN activation statistics (mean, std, sparsity, active neurons) - Output projection with top-5 predictions - Multi-token generation support with proper context updating - QKVAttentionExtractor: Specialized attention weight extraction - Supports CodeGen qkv_proj architecture - Handles GPT-2 c_attn style projections - Computes real attention scores with causal masking - Returns full attention patterns for visualization API Enhancements: - /analyze/pipeline endpoint with multi-token support - /analyze/attention endpoint for detailed attention analysis - Configurable generation parameters (temperature, top_k, top_p) - Backward compatible with single-token requests Data Integrity: - All data extracted directly from model forward pass …767a3fd
-
Add backend support for ICL emergence analysis
gary-boon +1: Claude- Implement ICL attention extractor with PyTorch hooks - Add induction head detector for pattern recognition - Create context efficiency analyzer for optimal example usage - Update model service with ICL emergence endpoints - Support real-time attention weight extraction during generation - Enable token-by-token generation for attention capture 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>920a98d
-
Add ablation support to model service with comprehensive testing
gary-boon +1: Claude- Implement ablation hooks for attention, FFN, and layer disabling - Fix string-to-int conversion for frontend compatibility - Add repetition-aware perplexity calculation - Include detailed logging for ablation debugging - Add comprehensive unit tests for ablation functionality - Fix temperature=0 handling for deterministic generation Tests confirm: - Attention ablation increases entropy from 0.44 to 1.82 - FFN ablation has strongest effect (5.32 mean difference) - All ablation patterns produce appropriately degraded outputs 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>bb8a292
-
Update backend URL to new HuggingFace organization
gary-boon- Changed from garyboon-visualisable-ai-backend.hf.space - To: visualisable-ai-api.hf.space - Updated all API routes, WebSocket URLs, and environment files - Cleaner, more professional URL structure53dcecd
-
Add CORS support for Vercel production domain
gary-boon- Allow WebSocket and API connections from visualisable-ai.vercel.app - Support all Vercel preview deployments with wildcard3c5bd74 -
Add API key authentication
gary-boon- Integrated auth.py into model_service.py endpoints - All endpoints now require X-API-Key header when API_KEY env var is set - Created setup instructions for HuggingFace Spaces configuration96a6300 -
Fix backend structure - remove duplicates
gary-boon2c0fd9b