Fix Online Softmax: move subplot titles inside chart area (y=0.95)
a0y0346
7720e5f
Fix Online Softmax charts: increase height (350px), margins, and y-axis range for labels
a0y0346
73045e3
Fix Memory Hierarchy: move labels further right (x=0.65), shorten arrow
a0y0346
83079d4
Fix Memory Hierarchy diagram: move transfer labels to right of arrow
a0y0346
1307f62
Fix GQA scaling chart x-axis: use log scale to space tick labels properly
a0y0346
287497c
Fix GQA tab chart overlaps
a0y0346
783749d
- Memory comparison chart: increased height, added y-axis headroom for bar labels, repositioned savings annotation with arrow
- Scaling chart: increased height and bottom margin, moved legend below x-axis properly, formatted x-axis ticks as 1K/2K/etc
Add separate KV Cache dtype selector (FP16/BF16/FP8/INT8)
a0y0346
509d7b6
- KV cache precision can now be configured independently from weight precision
- Realistic representation: model weights and KV cache have different quantization options
- FP8/INT8 KV cache halves memory vs FP16, enabling 2x longer contexts
- Updated insight panel to show KV cache memory savings
- Added more batch sizes (64, 128) to tradeoff chart
Add .gitignore, remove pycache from tracking
a0y0346
6492c04
Fix legend overlap in context scaling chart, increase batch size limit to 256
a0y0346
3de9ded
Fix text overlap in Memory Budget charts
a0y0346
aabd982
Add Memory Budget tab with real model and GPU detection
a0y0346
80187ae
- Add src/memory_budget.py for memory analysis
- Extract parameter count from model.num_parameters()
- Get KV cache config from model.config
- Detect actual GPU memory via torch.cuda
- Memory breakdown chart, scaling chart, batch tradeoff chart
Add GQA/MQA tab with real model configuration
a0y0346
a9b1e87
- Add src/gqa.py module for GQA analysis using real model.config
- Extract num_heads, num_kv_heads, head_dim from model config
- Compare KV cache memory for MHA vs GQA vs MQA
- Visual head sharing diagram for attention types
- Memory scaling chart across sequence lengths
Refactor benchmarks to use real model.config values
a0y0346
af9b854
- Add get_real_model_config() to extract config from model.config
- Refactor run_prefill_benchmark to use F.scaled_dot_product_attention
- Refactor run_decode_benchmark with proper KV cache and GQA support
- Update create_kv_cache_chart to use model.config (no constants)
- All config values now come from actual loaded models
Fix NoneType comparison error in prefill/decode charts
a0y0346
c30936f
- Add safe_get helper to handle None values from failed benchmarks
- Enhance attention pattern chart with dynamic sizing and ops display
fix: Add fallback SDPA benchmark when attention layer fails
a0y0346
685194e
- Improved benchmark_attention_layer error handling with logging
- Added fallback to F.scaled_dot_product_attention using real model
dimensions when attention layer forward pass fails
- This ensures benchmarks still work with model's actual head
configuration even if layer-level benchmarking has issues
feat: Use real HuggingFace model attention layers for benchmarks
a0y0346
374d38b
- Add attention_utils.py with functions to extract and benchmark
real attention layers from loaded HF models
- Refactor benchmark.py to load actual models and run attention
layer forward passes instead of raw SDPA with random tensors
- Refactor prefill_decode.py to use real model attention for both
prefill and decode phase comparisons
- Update app.py to pass model names to benchmark functions
This ensures all GPU benchmarks use real HuggingFace model
attention layers (SmolLM2-360M, Qwen2.5-0.5B, Llama-3.2-1B)
rather than synthetic random tensors.
Use scatter markers for attention pattern visualization
a0y0346
47751f7
- Replace heatmap with scatter squares for more reliable rendering
- Generates explicit coordinate pairs for filled cells
- Clear lower triangular pattern for prefill
- Growing rows for decode pattern
Fix attention pattern heatmaps with standard colorscales
a0y0346
fb9f765
- Use Plotly's built-in Blues/Greens colorscales
- Add explicit zmin/zmax for proper color mapping
- Remove autorange=reversed that was causing issues
Fix Prefill vs Decode chart visibility issues
a0y0346
30d461e
- Attention Patterns: Better colorscale with contrast, reversed y-axis
- Timing chart: Move text labels inside bars, add headroom for labels
- Increased chart heights and margins for better readability
Implement Prefill vs Decode tab (Phase 4)
a0y0346
473fcf2
- Add src/prefill_decode.py with phase comparison benchmarks
- Prefill: Full N×N attention simulation with FlashAttention vs Math
- Decode: 1×N attention per token with KV cache simulation
- Attention pattern visualization showing prefill vs decode patterns
- KV cache growth chart showing memory usage over generation
- Detailed insight text explaining compute-bound vs memory-bound phases
Use compute capability + SM count for better GPU spec estimation
a0y0346
5642188
Instead of guessing based only on memory size, now uses:
- torch.cuda.get_device_properties() for SM count
- torch.cuda.get_device_capability() for architecture
- Calculates estimated TFLOPS from: SMs × FLOPs/SM/cycle × clock
- Estimates bandwidth from memory size × architecture factor
Add H200 GPU support and improve roofline chart visibility
a0y0346
e2de6cd
- Add H200, L40S, L4, RTX 4090 to GPU detection
- Use dynamic memory detection for unknown GPUs
- Add visible labels with backgrounds on roofline chart
- Show GPU specs in chart title
- Use annotations with arrows instead of inline text
Auto-detect GPU instead of dropdown selection
a0y0346
d5ef20e
- Add detect_gpu() function to identify actual GPU hardware
- Remove GPU Model dropdown (was misleading on Zero GPU)
- Display detected GPU name and specs when benchmark runs
- Use detected GPU specs for roofline chart
- Supports A10G, A100, H100, T4, V100 with fallback for unknown GPUs
Make Roofline chart dynamic with measured benchmark results
a0y0346
3fec038
- Add calculate_attention_flops() to compute FLOPs for SDPA
- Add calculate_memory_traffic() to compute IO bytes for each backend
- Add calculate_roofline_metrics() to derive arithmetic intensity and achieved TFLOPS
- Update create_roofline_chart() to plot measured vs theoretical values
- Update get_roofline_insight() to show measured performance data
- Wire run_single_benchmark to update roofline chart with actual measurements
- Store benchmark metrics in Gradio State for GPU dropdown changes
Add Phase 3: Benchmark tab with GPU performance measurement
a0y0346
05810bb
- Create src/benchmark.py with GPU benchmark functions
- Backend comparison (math vs flash vs mem_efficient) with real timing
- Scaling curve chart (time/memory vs sequence length)
- Roofline chart showing arithmetic intensity analysis
- Wire up Tab 2 UI with benchmark controls and visualizations
