File size: 3,967 Bytes
adbc6be c5827c5 adbc6be | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 | ---
language:
- en
license: apache-2.0
base_model: HuggingFaceTB/SmolLM2-360M-Instruct
tags:
- asr
- speech-formatting
- smollm2
- mlx
- formatting-engine
---
# SmolLM2-360M-Post-ASR-Engine
A compact, local fine-tune of **HuggingFaceTB/SmolLM2-360M-Instruct** designed to act as a lightweight post-ASR (Automatic Speech Recognition) formatting utility.
## Purpose & Scope
Raw ASR transcripts often lack punctuation, structure, capitalization, and contain disfluencies. This model transforms raw spoken transcripts into clean, structured text using specific task tokens.
### Supported Task Tokens
1. `<|task:format_email|>`: Formats raw spoken notes into a structured email with Subject line and body.
2. `<|task:bullet_points|>`: Extracts concise, action-oriented bullet points from meeting or standup dictations.
3. `<|task:clean_punctuate|>`: Restores punctuation, capitalization, numbers, and technical abbreviations.
4. `<|task:neutral_paraphrase|>`: Rewrites verbose or conversational transcripts into neutral, concise statements.
### Training Details
- **Base Architecture**: SmolLM2-360M-Instruct
- **Method**: Weight-Decomposed Low-Rank Adaptation (DoRA) with prompt loss masking on Apple Silicon (`mlx-lm`). Fused into standalone weights.
- **Style Constraints**: Banned conversational fluff ("delve", "in summary", "tapestry") and em dashes (`—`, `–`).
---
## Usage
### 1. With MLX-LM (Apple Silicon)
```python
from mlx_lm import load, generate
model_id = "vikramlingam/SmolLM2-360M-Post-ASR-Engine"
model, tokenizer = load(model_id)
prompt = """<|im_start|>system
You are a concise, neutral post-ASR formatting engine. Output clean formatting strictly following the task prefix token. Do not include introductory or concluding conversational filler, AI buzzwords, or em dashes.<|im_end|>
<|im_start|>user
<|task:format_email|> hey team quick heads up standup is moved to tomorrow at eleven am let me know if that works<|im_end|>
<|im_start|>assistant
"""
response = generate(model, tokenizer, prompt=prompt, max_tokens=256)
print(response)
```
### 2. With Hugging Face Transformers
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "vikramlingam/SmolLM2-360M-Post-ASR-Engine"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)
messages = [
{"role": "system", "content": "You are a concise, neutral post-ASR formatting engine. Output clean formatting strictly following the task prefix token. Do not include introductory or concluding conversational filler, AI buzzwords, or em dashes."},
{"role": "user", "content": "<|task:bullet_points|> alex will finish the auth service by wednesday and maria will write load tests on staging"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
### 3. With llama.cpp / GGUF (Q4_K_M)
```python
from llama_cpp import Llama
# Download SmolLM2-360M-Post-ASR-Q4_K_M.gguf from this repo
llm = Llama.from_pretrained(
repo_id="vikramlingam/SmolLM2-360M-Post-ASR-Engine",
filename="SmolLM2-360M-Post-ASR-Q4_K_M.gguf",
n_ctx=1024,
n_gpu_layers=-1,
verbose=False,
)
prompt = """<|im_start|>system
You are a concise, neutral post-ASR formatting engine. Output clean formatting strictly following the task prefix token. Do not include introductory or concluding conversational filler, AI buzzwords, or em dashes.<|im_end|>
<|im_start|>user
<|task:format_email|> hey mark we need to shift standup to eleven am please check slide four<|im_end|>
<|im_start|>assistant
"""
output = llm(prompt, max_tokens=256, stop=["<|im_end|>"], temperature=0.0)
print(output["choices"][0]["text"].strip())
```
|