fix: Add fallback SDPA benchmark when attention layer fails
a0y0346
685194e
- Improved benchmark_attention_layer error handling with logging
- Added fallback to F.scaled_dot_product_attention using real model
dimensions when attention layer forward pass fails
- This ensures benchmarks still work with model's actual head
configuration even if layer-level benchmarking has issues
feat: Use real HuggingFace model attention layers for benchmarks
a0y0346
374d38b
- Add attention_utils.py with functions to extract and benchmark
real attention layers from loaded HF models
- Refactor benchmark.py to load actual models and run attention
layer forward passes instead of raw SDPA with random tensors
- Refactor prefill_decode.py to use real model attention for both
prefill and decode phase comparisons
- Update app.py to pass model names to benchmark functions
This ensures all GPU benchmarks use real HuggingFace model
attention layers (SmolLM2-360M, Qwen2.5-0.5B, Llama-3.2-1B)
rather than synthetic random tensors.
Use compute capability + SM count for better GPU spec estimation
a0y0346
5642188
Instead of guessing based only on memory size, now uses:
- torch.cuda.get_device_properties() for SM count
- torch.cuda.get_device_capability() for architecture
- Calculates estimated TFLOPS from: SMs × FLOPs/SM/cycle × clock
- Estimates bandwidth from memory size × architecture factor
Add H200 GPU support and improve roofline chart visibility
a0y0346
e2de6cd
- Add H200, L40S, L4, RTX 4090 to GPU detection
- Use dynamic memory detection for unknown GPUs
- Add visible labels with backgrounds on roofline chart
- Show GPU specs in chart title
- Use annotations with arrows instead of inline text
Auto-detect GPU instead of dropdown selection
a0y0346
d5ef20e
- Add detect_gpu() function to identify actual GPU hardware
- Remove GPU Model dropdown (was misleading on Zero GPU)
- Display detected GPU name and specs when benchmark runs
- Use detected GPU specs for roofline chart
- Supports A10G, A100, H100, T4, V100 with fallback for unknown GPUs
Make Roofline chart dynamic with measured benchmark results
a0y0346
3fec038
- Add calculate_attention_flops() to compute FLOPs for SDPA
- Add calculate_memory_traffic() to compute IO bytes for each backend
- Add calculate_roofline_metrics() to derive arithmetic intensity and achieved TFLOPS
- Update create_roofline_chart() to plot measured vs theoretical values
- Update get_roofline_insight() to show measured performance data
- Wire run_single_benchmark to update roofline chart with actual measurements
- Store benchmark metrics in Gradio State for GPU dropdown changes
Add Phase 3: Benchmark tab with GPU performance measurement
a0y0346
05810bb
- Create src/benchmark.py with GPU benchmark functions
- Backend comparison (math vs flash vs mem_efficient) with real timing
- Scaling curve chart (time/memory vs sequence length)
- Roofline chart showing arithmetic intensity analysis
- Wire up Tab 2 UI with benchmark controls and visualizations
