Hooshaai/bert-comp-come-compressor-sst2

BERT-Base (come_compressor) — NeuralOps Compressed & LoRA-Healed Model Zoo

This repository provides a physically compressed and LoRA-healed checkpoint of google-bert/bert-base-uncased fine-tuned on GLUE SST-2, compressed using the come_compressor (CoMeCompressor) engine from the NeuralOps Library at Pareto-selected rank k=32 (1.12x compression ratio, -10.9% parameter reduction).

---

Compression & Architecture Profile

CoMeCompressor: CoMe Compressor: Layer shedding with learnable low-rank interpolation adapters (bridges) between non-consecutive retained layers. Config parameters (via kwargs or comp_config): rank : int, optional — SVD-style rank. If provided and bridge_rank/sparsity are not explicitly set, they are derived from rank via safe mappings. target_layers : int, optional — target layer count after shedding. sparsity : float, default 0.10

Property	Specification
Registry Key (COMPRESSOR_REGISTRY)	come_compressor
NeuralOps Class & Family	CoMeCompressor (low_rank)
Source Implementation	neuralops/compression/low_rank/come_compressor.py
Base Architecture	google-bert/bert-base-uncased (109,483,778 parameters, 12 layers, $D=768$)
Showcase Rank (k)	32 (LoRA Recovery Applied: True)
Compressed Parameter Count	97,545,988 (-10.90% reduction, 1.122x physical compression ratio)
Teacher Baseline SST-2 Acc	91.63%
Showcase Final SST-2 Acc	90.14% (Zero-Shot: 84.63%, Parity Delta: -1.49%)

---

Multi-Rank Pareto Evaluation (k=256, 128, 64, 32)

Evaluated physically on NVIDIA GeForce RTX 3090 (24GB) on the GLUE SST-2 validation split (872 samples). Whenever zero-shot compression drops accuracy by > 2.0% alongside > 1.10x reduction, NeuralOps applies targeted r=8 LoRA healing (50 steps, lr=2e-4, 45s budget):

Rank (k)	Physical Ratio	Compressed Params (Reduction)	Zero-Shot Acc	Healed Acc	Parity vs Teacher (91.63%)
256	1.04x	104,928,003 (-4.2%)	91.40%	91.40%	-0.23%
128	1.12x	97,619,716 (-10.8%)	84.52%	89.56% (LoRA)	-2.07%
64	1.12x	97,570,564 (-10.9%)	84.52%	89.33% (LoRA)	-2.30%
32 ⭐	1.12x	97,545,988 (-10.9%)	84.63%	90.14% (LoRA)	-1.49%

(⭐ indicates the serialized checkpoint packaged in this Hugging Face repository.)

---

Packaged Artifacts

• pytorch_model.bin: Full serialized compressed BERT state dictionary (with low-rank factors / quantization buffers / LoRA healing adapters baked in).
• config.json: Model configuration tagged with compression_method: "come_compressor" and compression_rank: 32.
• compression_summary.json: Complete machine-readable telemetry across all evaluated ranks [256, 128, 64, 32].
• Tokenizer Assets: Standard bert-base-uncased vocabulary (vocab.txt, tokenizer.json, tokenizer_config.json).

---

Quick Start (Reconstruction & Inference)

import torch
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer, BertForSequenceClassification
from neuralops.core.compression_pipeline import compress_model

repo_id = "tahamajs/bert-comp-come-compressor-sst2"
tokenizer = AutoTokenizer.from_pretrained(repo_id)

# Reconstruct compressed NeuralOps topology and load serialized compressed + healed weights
model = BertForSequenceClassification.from_pretrained("google-bert/bert-base-uncased", num_labels=2)
model = compress_model(model, {"compressors": [{"method": "come_compressor", "rank": 32}]})

ckpt_path = hf_hub_download(repo_id=repo_id, filename="pytorch_model.bin")
state_dict = torch.load(ckpt_path, map_location="cpu")
model.load_state_dict(state_dict, strict=False)
model.eval()

inputs = tokenizer("NeuralOps achieves strong compression without sacrificing downstream accuracy.", return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
    pred = logits.argmax(dim=-1).item()
print("Predicted SST-2 Sentiment (0=Negative, 1=Positive):", pred)

---

Quick Start (Universal NeuralOps Hub API)

import torch
from neuralops.hub import load_surgeried_model

# Load model and tokenizer/processor in one line directly from Hugging Face Hub
model, tokenizer = load_surgeried_model("tahamajs/bert-comp-come-compressor-sst2")
model.eval()
print(f"Successfully loaded tahamajs/bert-comp-come-compressor-sst2 with NeuralOps modular surgical weights!")

Citation

@software{neuralops2026compressorzoo,
  title   = {NeuralOps: Multi-Rank Compressed BERT-SST2 Zoo with Pareto LoRA Recovery},
  author  = {Majlesi, Taha},
  year    = {2026},
  url     = {https://github.com/tahamajs/NeuralOps}
}