Grammar-V7D: Fast, Accurate English Grammar Correction

Grammar-V7D is a production-grade English Grammar Error Correction (GEC) model fine-tuned from T5-base (220M parameters). Built for speed and safety on CPU, it delivers clean corrections without over-editing correct text.

Key Features

🚀 Fast on CPU: ~95-104ms per sentence with ONNX Runtime (int8)
🛡️ Safe & conservative: Only 2.8% of clean text touched (35x safer than LanguageTool)
🎯 Fluency-focused: Achieves 62.73 GLEU on JFLEG, prioritizing natural, readable corrections
📦 Production-ready: Exact ONNX graphs used in live serving included
🔍 Transparent evaluation: All benchmarks measured with official scorers, including failures

Quick Start

Using Transformers

from transformers import T5ForConditionalGeneration, T5Tokenizer

tokenizer = T5Tokenizer.from_pretrained("mydeskmate/grammar-v7d")
model = T5ForConditionalGeneration.from_pretrained("mydeskmate/grammar-v7d")

text = "He go to school yesterday and dont know what to do."
input_text = "gec: " + text
inputs = tokenizer(input_text, return_tensors="pt", max_length=512, truncation=True)
outputs = model.generate(**inputs, max_length=512, num_beams=4)
corrected = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(corrected)

Using the Live Demo

Try it instantly at tools.mydeskmate.in/grammar.html

Performance Benchmarks

Pre-registered Promotion Results (GPU, beams=4)

All gates passed as per pre-registered criteria (tested before promotion):

Metric	Score	Threshold	Status
JFLEG GLEU (Official)	62.73	≥ 62.30	✅ PASS
CoNLL-2014 F0.5 (m2scorer)	49.04	> 47.35 (vs v5)	✅ PASS
BEA-2019-dev F0.5 (ERRANT)	35.41	> 33.90 (vs v5)	✅ PASS
Do-No-Harm (Clean text)	2.80% touched	< 5%	✅ PASS

Live Serving Performance (ONNX Runtime int8, CPU)

Measured through production engine with exact serving graphs:

Metric	Score	Target	Status
JFLEG GLEU (Official)	62.61	≥ 60	✅ PASS
CoNLL-2014 F0.5 (Official m2scorer)	49.34	≥ 65	⚠️ Open Target
BEA-2019-dev F0.5 (Official ERRANT)	34.61	≥ 70	⚠️ Open Target
Do-No-Harm (500 clean sentences)	4.20% touched	< 5%	✅ PASS
Over-correction (2000 sentences)	5.05% touched	< 5%	⚠️ Published as measured (+0.05pp)
Latency (Live HTTP, prod)	95ms / 102ms (median/mean)	< 200ms	✅ PASS
Latency (ORT in-process)	162ms / 188ms (median/mean)	Reference	✅ PASS

Head-to-Head Comparison

All systems evaluated in the same lab using identical official scorers for fair comparison:

System	JFLEG GLEU ↑	CoNLL-2014 F0.5 ↑	BEA-2019-dev F0.5 ↑	Do-No-Harm ↓
Grammar-V7D (Ours)	62.73	49.04	35.41	2.80%
LanguageTool 6.6	47.57	25.60	20.35	98.80%
GECToR (bea-dkt+conll, kc=.3)	53.94	56.94	49.52	10.40%

We outperform LanguageTool on every metric while being 35x safer. GECToR achieves higher scores on narrow academic benchmarks but over-corrects 10.4% of clean text (failing our safety gate) and scores 8.7 GLEU lower on fluency.

Note: GECToR's published numbers may differ due to different evaluation pipelines/thresholds. We publish all numbers measured on the same yardstick.

Model Details

Architecture: T5ForConditionalGeneration (T5-base)
Parameters: ~220M
Language: English
Task: Grammatical Error Correction
Training: Continued from grammar-v7b lineage with ~12% dose of exam-style calibration data
Precision: FP32 weights included; optimized int8 ONNX graphs available for CPU inference
Context Length: 512 tokens
License: Other (research/educational use; check source dataset licenses for commercial use)

Training Data

Trained on research corpora including JFLEG, CoEdit, and WikiText-103 subsets, with carefully calibrated synthetic and exam-style data.

Files

config.json, generation_config.json - Model configuration
tokenizer.json, tokenizer_config.json - Tokenizer
model.safetensors - Model weights

Limitations

Academic benchmarks: CoNLL-2014 (49.34) and BEA-2019 (34.61) remain below some larger models. Focus is on fluency and safety over maximizing exam scores.
Context: Limited to 512 tokens
Language: English only
Conservative by design: May miss some valid corrections to prioritize not breaking correct text

Citation

If you use this model in research, please cite appropriately. For more details or to report issues, visit our live demo.

mydeskmate
/

grammar-v7d