--- language: - en license: mit license_link: https://github.com/tahamajs/NeuralOps/blob/main/LICENSE library_name: neuralops base_model: google-bert/bert-base-uncased datasets: - nyu-mll/glue metrics: - accuracy tags: - neuralops - bert - model-compression - compressor-zoo - lora-healing - glue-sst2 - quantization - smoothquant pipeline_tag: text-classification model-index: - name: bert-comp-smoothquant-sst2 results: - task: type: text-classification name: Sentiment Classification dataset: type: nyu-mll/glue name: GLUE SST-2 config: sst2 split: validation metrics: - type: accuracy value: 91.74 name: GLUE SST-2 Validation Accuracy --- # BERT-Base (smoothquant) — NeuralOps Compressed & LoRA-Healed Model Zoo [![SST-2 Acc](https://img.shields.io/badge/GLUE_SST--2_Acc-91.74%25-brightgreen)](#-multi-rank-pareto-evaluation-k256-128-64-32) [![Teacher Baseline](https://img.shields.io/badge/Teacher_Baseline-91.63%25-blue)](#-multi-rank-pareto-evaluation-k256-128-64-32) [![Compression Ratio](https://img.shields.io/badge/Compression-1.00x_(0.0%25_Saved)-purple)](#-compression--architecture-profile) [![Showcase Rank](https://img.shields.io/badge/Showcase_Rank-k=32-teal)](#-multi-rank-pareto-evaluation-k256-128-64-32) [![Weights](https://img.shields.io/badge/Weights-291.0_MB-indigo)](#-packaged-artifacts) [![License](https://img.shields.io/badge/License-MIT-green)](https://github.com/tahamajs/NeuralOps/blob/main/LICENSE) This repository provides a physically compressed and LoRA-healed checkpoint of **`google-bert/bert-base-uncased`** fine-tuned on **GLUE SST-2**, compressed using the **`smoothquant`** (`SmoothQuantCompressor`) engine from the [NeuralOps Library](https://github.com/tahamajs/NeuralOps) at Pareto-selected rank **`k=32`** (`1.00x` compression ratio, `-0.0%` parameter reduction). --- ## Compression & Architecture Profile > **SmoothQuantCompressor**: Unified BaseCompressor wrapper for SmoothQuant W8A8 with compressed-tensors support. | Property | Specification | |---|---| | **Registry Key (`COMPRESSOR_REGISTRY`)** | `smoothquant` | | **NeuralOps Class & Family** | `SmoothQuantCompressor` (`quantization`) | | **Source Implementation** | [`neuralops/compression/quantization/smoothquant.py`](https://github.com/tahamajs/NeuralOps/blob/main/neuralops/compression/quantization/smoothquant.py) | | **Base Architecture** | [`google-bert/bert-base-uncased`](https://huggingface.co/google-bert/bert-base-uncased) (`109,483,778` parameters, 12 layers, $D=768$) | | **Showcase Rank (`k`)** | **`32`** (`LoRA Recovery Applied: False`) | | **Compressed Parameter Count** | **`109,483,778`** (`-0.00%` reduction, **`1.000x`** physical compression ratio) | | **Teacher Baseline SST-2 Acc** | `91.63%` | | **Showcase Final SST-2 Acc** | **`91.74%`** (Zero-Shot: `91.74%`, Parity Delta: `+0.11%`) | --- ## Multi-Rank Pareto Evaluation (`k=256, 128, 64, 32`) Evaluated physically on **NVIDIA GeForce RTX 3090 (24GB)** on the GLUE SST-2 validation split (`872` samples). Whenever zero-shot compression drops accuracy by `> 2.0%` alongside `> 1.10x` reduction, NeuralOps applies targeted `r=8` LoRA healing (`50` steps, `lr=2e-4`, 45s budget): | Rank (`k`) | Physical Ratio | Compressed Params (Reduction) | Zero-Shot Acc | Healed Acc | Parity vs Teacher (`91.63%`) | |:---:|:---:|---:|---:|---:|---:| | **`256`** | `1.00x` | `109,483,778` (`-0.0%`) | `91.74%` | **`91.74%`** | `+0.11%` | | **`128`** | `1.00x` | `109,483,778` (`-0.0%`) | `91.74%` | **`91.74%`** | `+0.11%` | | **`64`** | `1.00x` | `109,483,778` (`-0.0%`) | `91.74%` | **`91.74%`** | `+0.11%` | | **`32` ⭐** | `1.00x` | `109,483,778` (`-0.0%`) | `91.74%` | **`91.74%`** | `+0.11%` | *(⭐ indicates the serialized checkpoint packaged in this Hugging Face repository.)* --- ## Packaged Artifacts - **`pytorch_model.bin`**: Full serialized compressed BERT state dictionary (with low-rank factors / quantization buffers / LoRA healing adapters baked in). - **`config.json`**: Model configuration tagged with `compression_method: "smoothquant"` and `compression_rank: 32`. - **`compression_summary.json`**: Complete machine-readable telemetry across all evaluated ranks `[256, 128, 64, 32]`. - **Tokenizer Assets**: Standard `bert-base-uncased` vocabulary (`vocab.txt`, `tokenizer.json`, `tokenizer_config.json`). --- ## Quick Start (Reconstruction & Inference) ```python import torch from huggingface_hub import hf_hub_download from transformers import AutoTokenizer, BertForSequenceClassification from neuralops.core.compression_pipeline import compress_model repo_id = "tahamajs/bert-comp-smoothquant-sst2" tokenizer = AutoTokenizer.from_pretrained(repo_id) # Reconstruct compressed NeuralOps topology and load serialized compressed + healed weights model = BertForSequenceClassification.from_pretrained("google-bert/bert-base-uncased", num_labels=2) model = compress_model(model, {"compressors": [{"method": "smoothquant", "rank": 32}]}) ckpt_path = hf_hub_download(repo_id=repo_id, filename="pytorch_model.bin") state_dict = torch.load(ckpt_path, map_location="cpu") model.load_state_dict(state_dict, strict=False) model.eval() inputs = tokenizer("NeuralOps achieves strong compression without sacrificing downstream accuracy.", return_tensors="pt") with torch.no_grad(): logits = model(**inputs).logits pred = logits.argmax(dim=-1).item() print("Predicted SST-2 Sentiment (0=Negative, 1=Positive):", pred) ``` --- ## Quick Start (Universal NeuralOps Hub API) ```python import torch from neuralops.hub import load_surgeried_model # Load model and tokenizer/processor in one line directly from Hugging Face Hub model, tokenizer = load_surgeried_model("tahamajs/bert-comp-smoothquant-sst2") model.eval() print(f"Successfully loaded tahamajs/bert-comp-smoothquant-sst2 with NeuralOps modular surgical weights!") ``` ## Citation ```bibtex @software{neuralops2026compressorzoo, title = {NeuralOps: Multi-Rank Compressed BERT-SST2 Zoo with Pareto LoRA Recovery}, author = {Majlesi, Taha}, year = {2026}, url = {https://github.com/tahamajs/NeuralOps} } ```