| --- |
| language: |
| - en |
| license: mit |
| library_name: onnxruntime |
| tags: |
| - text-generation |
| - token-classification |
| - grammar-correction |
| - autocomplete |
| - typo-correction |
| - character-aware |
| - edge-ai |
| - onnx |
| - int8 |
| - rope |
| - grouped-query-attention |
| - swiglu |
| - rmsnorm |
| - offline-nlp |
| pipeline_tag: text-generation |
| inference: false |
| widget: |
| - text: "Could you please confirm if the proposed implementation timeline aligns with your" |
| example_title: "Client Inquiry Completion" |
| - text: "We decoupled the event processing pipeline to achieve sub-" |
| example_title: "Engineering Autocomplete" |
| - text: "We dont expect any downtime during the blue-green deployment phase." |
| example_title: "Grammar & Typo Correction" |
| --- |
| |
| <p align="center"> |
| <img src="typoformer_full_logo.png" alt="TypoFormer Banner" width="620"/> |
| </p> |
|
|
| <h1 align="center">TypoFormer (Stage 2 INT8)</h1> |
|
|
| <p align="center"> |
| <strong>High-Precision Character-Aware Hybrid Transformer for Edge Autocomplete & Non-Autoregressive Grammar Correction</strong><br/> |
| <sub>Created & Authored by <strong><a href="https://github.com/fahiiim">Md. Fahim Sarker Mridul</a></strong></sub> |
| </p> |
|
|
| <p align="center"> |
| <a href="https://github.com/fahiiim/TyPoTaKer"><img src="https://img.shields.io/badge/GitHub-fahiiim%2FTyPoTaKer-181717?style=for-the-badge&logo=github&logoColor=white" alt="GitHub Repository"/></a> |
| <img src="https://img.shields.io/badge/Author-Md._Fahim_Sarker_Mridul-1E293B?style=for-the-badge&logo=github&logoColor=white" alt="Author"/> |
| <img src="https://img.shields.io/badge/Model-TypoFormer--v2.4-76B900?style=for-the-badge&logo=nvidia&logoColor=white" alt="Model Version"/> |
| <img src="https://img.shields.io/badge/Parameters-93.88M_INT8-4F8A0E?style=for-the-badge&logo=cpu&logoColor=white" alt="Parameters"/> |
| <img src="https://img.shields.io/badge/Latency-Sub--10ms_CPU-2B5A0C?style=for-the-badge&logo=speedtest&logoColor=white" alt="Latency"/> |
| <img src="https://img.shields.io/badge/Privacy-100%25_AirGapped_Offline-1A3B05?style=for-the-badge&logo=shield&logoColor=white" alt="Privacy"/> |
| <img src="https://img.shields.io/badge/License-MIT-blue?style=for-the-badge" alt="License"/> |
| </p> |
|
|
| <p align="center"> |
| <a href="#model-architecture"><strong>Architecture</strong></a> | |
| <a href="#key-highlights"><strong>Highlights</strong></a> | |
| <a href="#architectural-comparison-typoformer-vs-bert-vs-cloud-llms"><strong>Comparison</strong></a> | |
| <a href="#benchmark-and-evaluation-metrics"><strong>Benchmarks</strong></a> | |
| <a href="#repository-contents"><strong>Files</strong></a> | |
| <a href="#quickstart-and-usage"><strong>Quickstart</strong></a> | |
| <a href="#windows-desktop-keystroke-companion-1-line-install"><strong>Windows Desktop</strong></a> | |
| <a href="#citation-and-author-attribution"><strong>Citation</strong></a> |
| </p> |
|
|
| --- |
|
|
| ## Overview |
|
|
| **TypoFormer** is a purpose-engineered, character-aware neural backbone designed from the ground up to solve two conflicting natural language problems inside a single unified **94M parameter** trunk: |
|
|
| 1. **Causal Next-Word Autocomplete**: Low-latency autoregressive token prediction under strict sub-15ms local keystroke budgets without cloud connectivity. |
| 2. **Non-Autoregressive Grammar & Typo Correction (GEC)**: High-precision edit-tag sequence labeling that repairs punctuation slips, confused vocabulary, subject-verb agreement, and severe typos without generative hallucination. |
|
|
| By fusing **Byte-Level BPE subwords** with **Multi-Scale Character 1D-Convolutions** through a learned gating highway, TypoFormer eliminates out-of-vocabulary token fractures and delivers enterprise-grade real-time writing assistance completely offline on standard desktop CPUs. |
|
|
| --- |
|
|
| ## Model Architecture |
|
|
| <p align="center"> |
| <img src="typoformer_architecture.png" alt="TypoFormer Full Internal Neural Architecture Diagram" width="620"/> |
| </p> |
| <p align="center"> |
| <em>Figure 1: TypoFormer Full Internal Neural Architecture: Fusing byte-level subwords with multi-scale character convolutions, Pre-RMSNorm grouped-query attention (GQA), rotary embeddings (RoPE), SwiGLU feed-forward networks, and multi-task dual heads for offline autocomplete and grammar error correction (Visualized in NVIDIA Emerald Green <code>#76B900</code>).</em><br/> |
| <a href="https://huggingface.co/fahiiim/TypoFormer/resolve/main/typoformer_architecture.png"><strong>[View High-Resolution 300 DPI Architecture Diagram (PNG)]</strong></a> |
| </p> |
|
|
| ### 1. Dual-Channel Gated Input Highway |
| Standard tokenizers fail when words are corrupted: mistyping `"definitely"` as `"definatly"` converts a single semantic token into multiple rare subword fragments. TypoFormer prevents this tokenization distortion through a hybrid input representation: |
| * **Byte-Level BPE**: Words are split into standard BPE tokens from a compact 16,000-vocabulary dictionary ($x_{\text{tok}} \to \mathbf{E}_{\text{tok}} \in \mathbb{R}^{d_{\text{model}}}$). |
| * **Multi-Scale Character-CNN**: Concurrently, the exact UTF-8 character bytes of each token ($x_{\text{char}}$) are fed into parallel 1D convolutional banks with kernel sizes $k \in \{2, 3, 5, 8\}$: |
| $$\mathbf{C}_k = \mathrm{MaxPool}\left(\mathrm{GELU}\left(\mathrm{Conv1D}_k(\text{Bytes})\right)\right)$$ |
| * **Learned Gating Highway**: |
| $$\mathbf{h}_0 = \mathbf{E}_{\text{tok}} + \alpha \cdot \mathbf{W}_{\text{proj}}\left([\mathbf{C}_2 \,\|\, \mathbf{C}_3 \,\|\, \mathbf{C}_5 \,\|\, \mathbf{C}_8]\right)$$ |
| where $\alpha$ is a learnable scalar gate initialized to ensure robust character-level fallback when unknown or miscoded subwords appear. |
| |
| ### 2. Transformer Trunk (10 Blocks, $d=768$, $d_{\text{ffn}}=2048$) |
| * **Attention Mechanism**: Grouped-Query Attention (**GQA**) with 12 Query heads and 4 Key-Value heads (3:1 ratio), cutting KV-cache memory traffic by 66% during sequence generation. |
| * **Rotary Position Embeddings (RoPE)**: Frequency-based relative query-key rotation for superior relative token distance awareness across cursor movements without absolute position limits. |
| * **Feed-Forward Backbone**: **SwiGLU** non-linearity: |
| $$\text{SwiGLU}(x) = W_2\left(\text{SiLU}(W_1 x) \odot W_3 x\right)$$ |
| * **Pre-Normalization**: **RMSNorm** applied prior to each sub-layer for stable gradient propagation and reduced memory overhead. |
| |
| ### 3. Multi-Task Dual Output Heads |
| * **Task 1: Causal LM Autocomplete Head**: Linear projection ($d_{\text{model}} \to 16{,}000$, weight-tied with token embeddings) yielding next-token probability distribution $P(w_{t+1} \mid w_{\le t})$. |
| * **Task 2: Keystroke GEC Edit Head**: Linear tagging projection ($d_{\text{model}} \to 6{,}148$) mapping each token position to atomic edits: |
| * `KEEP` (0): Token is grammatically and lexically correct. |
| * `DELETE` (1): Redundant or erroneous token removal. |
| * `CASE_CAP` (2) / `CASE_LOWER` (3): Casing adjustments. |
| * `REPLACE` (4 .. 4099): Atomic swap with one of 4,096 high-frequency correction tokens. |
| * `APPEND` (4100 .. 6147): Atomic right-insertion with one of 2,048 suffix/continuation tokens. |
| |
| --- |
| |
| ## Key Highlights |
| |
| * **Ultra-Low Keystroke Latency**: Mean end-to-end CPU inference of **9.97 ms** (P95: **12.12 ms**), fitting comfortably inside the 16.6 ms 60 FPS display frame deadline. |
| * **Typo Immunity**: Gated character-level convolutions preserve word roots even under severe phonetic and fat-finger keyboard slips. |
| * **100% Air-Gapped and Offline**: Zero network calls, zero telemetry, zero risk of leaking confidential documents, code, or private emails. |
| * **Guaranteed Content Fidelity**: Non-autoregressive tagging guarantees zero generative hallucination and achieves **100.0% clean negative sentence preservation**. |
| * **Compact Edge Footprint**: Symmetric INT8 quantization packs the complete 94M dual-task architecture into just **130.5 MB** ONNX graphs. |
| |
| --- |
| |
| ## Architectural Comparison: TypoFormer vs. BERT vs. Cloud LLMs |
| |
| | Architectural Dimension | Classical BERT-Base (110M) | Cloud Generative LLMs (7B - 70B) | TypoFormer (94M INT8) | TypoFormer Advantage | |
| | :--- | :--- | :--- | :--- | :--- | |
| | **Generative Autocomplete** | **Incapable** (Bidirectional MLM only) | **Yes** (High quality autoregression) | **Native Causal Mode** (Tied causal head) | **Unified Trunk**: Single model serves both autocomplete & GEC. | |
| | **Typo & OOV Robustness** | **Brittle** (`teh` to `['te', '##h']`) | **Fair** (Relies on large BPE dictionary) | **Gated Multi-Scale Char-CNN** ($k=2,3,5,8$) | **Typo Immunity**: Never blinded by unknown character corruptions. | |
| | **Position Encoding** | **Absolute Learned** (Max 512 tokens) | **RoPE** / ALiBi | **Rotary Position Embeddings (RoPE)** | **Sequence Extrapolation**: Relative distance aware across cursor hops. | |
| | **Attention Mechanism** | **Multi-Head (MHA)** (12 Q, 12 K, 12 V) | **GQA / MQA** | **Grouped Query Attention (GQA)** (12 Q, 4 KV) | **3x Memory Reduction**: Cuts KV-cache bandwidth by 66%. | |
| | **FFN Non-Linearity** | **GeLU + LayerNorm** (Additive bias) | **SwiGLU + RMSNorm** | **SwiGLU + Pre-RMSNorm** | **LLaMA/Mistral-Grade Efficiency**: Higher expressivity per parameter. | |
| | **GEC Formulation** | **Seq2Seq / MLM** (Slow, iterative) | **Generative Prompting** (Prone to hallucinations) | **Constrained Atomic Edit-Tagging** (6,148 tags) | **Zero Hallucination**: 100% preservation of correct text. | |
| | **End-to-End Latency** | **150ms - 400ms** on CPU | **300ms - 2,500ms** via Cloud API | **< 10ms** on CPU (INT8 ONNX) | **Real-Time Keystroke Budget**: Executes within 16ms display frame. | |
| | **Network & Privacy** | Local or Cloud | **Mandatory Cloud API** (Data leaves host) | **100% Offline Air-Gapped** | **Complete Privacy**: Zero telemetry, password field suppression. | |
| |
| --- |
| |
| ## Benchmark and Evaluation Metrics |
| |
| Live CPU benchmarks measured on standard Intel/AMD consumer hardware ([scripts/evaluate_gec_live.py](https://github.com/fahiiim/TyPoTaKer)): |
| |
| | Metric | Measured Value | Benchmark Description | |
| | :--- | :--- | :--- | |
| | **Mean CPU Latency** | **`9.97 ms`** | Per-sentence end-to-end inference | |
| | **P95 CPU Latency** | **`12.12 ms`** | 95th percentile latency (well below 16.6 ms display frame) | |
| | **P99 CPU Latency** | **`14.80 ms`** | 99th percentile worst-case latency | |
| | **GEC Stage 2 Validation Loss** | **`0.2597`** | Down from 4.1067 (93.7% loss reduction across training) | |
| | **LM Stage 2 Loss** | **`1.2054`** | Fine-tuned on curated engineering and client communications | |
| | **Clean Negative Preservation** | **`100.0%`** | Zero false-positive edits on grammatically correct input text | |
| | **GEC Model Footprint** | **`130.5 MB`** | Symmetric INT8 ONNX graph (`gec_stage2_int8.onnx`) | |
| | **LM Model Footprint** | **`138.1 MB`** | Symmetric INT8 ONNX graph (`lm_stage2_int8.onnx`) | |
| | **Active Working Set RAM** | **`< 250 MB`** | Total memory footprint during active inference | |
| |
| --- |
| |
| ## Repository Contents |
| |
| | File | Size | Format | Description | |
| | :--- | :--- | :--- | :--- | |
| | **`gec_stage2_int8.onnx`** | 130.5 MB | ONNX INT8 | INT8 Quantized graph for grammar, punctuation, and typo correction | |
| | **`lm_stage2_int8.onnx`** | 138.1 MB | ONNX INT8 | INT8 Quantized graph for causal next-word autocomplete | |
| | **`edit_maps.json`** | 228 KB | JSON | High-frequency confusion dictionary (4,096 replacements + 2,048 appends) | |
| | **`tokenizer.json`** | 1.11 MB | JSON | Byte-Level BPE tokenizer config (16,000 vocabulary) | |
| | **`vocab.json` / `merges.txt`** | 526 KB | Text/JSON | BPE vocabulary and merge table | |
| | **`config.json`** | 520 B | JSON | Architecture parameters & model metadata | |
| | **`infer.py`** | 6.5 KB | Python | Standalone, dependency-minimal Python inference engine | |
| | **`typoformer_architecture.png`**| 235 KB | PNG (300 DPI) | Full neural architecture diagram (NVIDIA emerald theme) | |
| | **`typoformer_full_logo.png`** | 188 KB | PNG | Official project branding logo | |
| | **`TypoFormer-win-x64.zip`** | 221.0 MB | ZIP | Standalone Windows distribution package | |
| |
| --- |
| |
| ## Quickstart and Usage |
| |
| ### 1. Minimal Installation |
| |
| ```bash |
| pip install onnxruntime tokenizers numpy |
| git clone https://huggingface.co/fahiiim/TypoFormer |
| cd TypoFormer |
| python infer.py |
| ``` |
| |
| ### 2. Standalone Python Example |
| |
| ```python |
| from infer import TypoFormer |
| |
| # Initialize the standalone inference engine (loads ONNX models locally) |
| model = TypoFormer(".") |
| |
| # 1. Autocomplete (Next-Word Generation) |
| prompt = "Could you please confirm if the proposed implementation timeline aligns with your" |
| suggestions = model.autocomplete(prompt, top_k=3) |
| print("Autocomplete suggestions:", suggestions) |
| # Output: [(' business', 0.9978), (' goals', 0.0002), (' company', 0.0001)] |
|
|
| # 2. Grammar, Punctuation & Typo Correction (GEC) |
| dirty_text = "We decoupled the event processing pipeline to achieve sub-" |
| suggestions = model.autocomplete(dirty_text, top_k=2) |
| print("Engineering completion:", suggestions) |
| # Output: [(' millisecond', 0.8841), (' second', 0.0712)] |
| |
| # 3. Clean Text Preservation (Zero Hallucination) |
| clean_text = "The quick brown fox jumps over the lazy dog." |
| result = model.correct_grammar(clean_text) |
| assert result == clean_text # 100% preservation |
| ``` |
| |
| --- |
| |
| ## Windows Desktop Keystroke Companion (1-Line Install) |
| |
| TypoFormer includes a native Windows x64 .NET 8 desktop daemon that runs silently in the system tray and renders a floating caret overlay across any active text editor (Word, Notepad, VS Code, Slack, Browser): |
| |
| Run in PowerShell: |
| ```powershell |
| irm https://raw.githubusercontent.com/fahiiim/TyPoTaKer/main/install.ps1 | iex |
| ``` |
| |
| * **Offline & Private**: Zero telemetry; no keystrokes ever leave your device. |
| * **Password Aware**: Automatically suppresses buffering on password and credential fields via Windows UI Automation. |
| * **Instant Action**: Press `[Tab]` to accept autocomplete suggestions or grammar corrections. |
| |
| --- |
| |
| ## Citation and Author Attribution |
| |
| ```bibtex |
| @software{typoformer2026, |
| author = {Md. Fahim Sarker Mridul}, |
| title = {TypoFormer: High-Precision Character-Aware Hybrid Transformer for Edge Autocomplete & Non-Autoregressive Grammar Correction}, |
| year = {2026}, |
| publisher = {Hugging Face}, |
| url = {https://huggingface.co/fahiiim/TypoFormer}, |
| howpublished = {\url{https://github.com/fahiiim/TyPoTaKer}} |
| } |
| ``` |
| |
| ## License |
| This model and its associated weights are distributed under the [MIT License](LICENSE). |
| Authored by **Md. Fahim Sarker Mridul**. |
| |
