Configuration Parsing Warning:Invalid JSON for config file config.json
SOA MODULE — GPT2 Benchmark Result
Status: ✅ PASS
Dataset: GLUE SST-2 (Binary)
Architecture: gpt2
Module: soa_module
This repository hosts empirical benchmark results and factorized checkpoints for soa_module applied to gpt2 on GLUE SST-2 (Binary).
Trained weights (weights.pt) are included in this repository.
Benchmark Performance Metrics
| Metric | Value |
|---|---|
| Model Family | gpt2 |
| Module / Method | soa_module |
| Dataset | GLUE SST-2 (Binary) |
| Validation Status | PASS |
| Validation Accuracy | 66.74% |
| Pass Threshold (3*SE Margin) | 31.63% |
| F1 Score (Macro) | 0.6327 |
| F1 Score (Weighted) | 0.7456 |
| Precision | 0.7513 |
| Recall | 0.662 |
| Validation Loss | 0.5718 |
| Compression Factor | 1.1201x |
| Peak GPU VRAM | 509.14 MB |
| Inference Throughput | 6485.02 tokens/sec |
| Evaluation Time | 46.82 s |
| Parameters (Before) | 124441344 |
| Parameters (After) | 111096576 |
| Compressed Layers | 0 |
| LoRA Trainable Params | 811008 |
| Benchmark Timestamp | 2026-09-15T23:47:13.664132 |
Architecture & Module Specification
The soa_module module is evaluated under the Hoosha AI NeuralOps SVD & Linear Attention Suite, benchmarking subquadratic attention and rank-adaptive tensor factorization against full quadratic baselines.
Framework & Verification
Developed and benchmarked using the SVD Linear Attention Framework — Universal GPU Benchmark Suite.
- Repository: tahamajs/SVD_linear_Attention
- Organization: Hooshaai
- Downloads last month
- 1