Instructions to use kalcy0n/rewriter with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use kalcy0n/rewriter with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="kalcy0n/rewriter") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("kalcy0n/rewriter") model = AutoModelForCausalLM.from_pretrained("kalcy0n/rewriter", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use kalcy0n/rewriter with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "kalcy0n/rewriter" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kalcy0n/rewriter", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/kalcy0n/rewriter
- SGLang
How to use kalcy0n/rewriter with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "kalcy0n/rewriter" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kalcy0n/rewriter", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "kalcy0n/rewriter" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kalcy0n/rewriter", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use kalcy0n/rewriter with Docker Model Runner:
docker model run hf.co/kalcy0n/rewriter
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("kalcy0n/rewriter")
model = AutoModelForCausalLM.from_pretrained("kalcy0n/rewriter", device_map="auto")
messages = [
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))Source-grounded passage humanizer
This checkpoint uses Qwen3-4B-Instruct-2507 with Unsloth's NF4 quantization. The writer takes one prose passage, decides whether it needs a repair, and selects the span to edit. A source-derived brief records the genre, purpose, takeaway, and relationships the edit should preserve. The writer proposes a replacement within that span and compares the result against the original for changes in meaning.
The published weights are the unadapted model from upstream revision 7744afa8566e264af1a92a806d8d9aae00cc7c78. The editing behavior is provided by the prompts, schemas and span renderer in writer.py. base-lock.json identifies the exact files for restoring the weights and tokenizer. Quantized checkpoint ยท Original Qwen model
Use
From a complete downloaded snapshot, use Python 3.11+ with PyTorch, Transformers, Accelerate, bitsandbytes, lm-format-enforcer and a CUDA GPU supporting native BF16. Supply UTF-8 JSON with an original field on standard input.
printf '%s\n' '{"original":"The two copies was legible."}' | python writer.py run
KEEP retains a passage the planner judged sound. UNCHANGED records a proposal that reproduced the source verbatim. REJECTED records a candidate that failed the meaning check. REVIEW returns both original and candidate for independent review. The full experiment also applies surface and logical-operator checks, bidirectional entailment and an independent repair judge before delivering an edit. This portable entry point stops at the candidate stage. Truncated or malformed generations raise an error.
For a review service, the standard-library messages command exposes the exact messages for each stage. plan and brief take the original passage. replace also takes a validated EDIT plan and an optional brief. semantic takes the original and the candidate. Schema-bearing stages return their JSON schema with the prompt.
printf '%s\n' '{"original":"The two copies was legible."}' | python writer.py messages plan
Measured limitations
On nine synthetic development passages, the guarded pipeline delivered the original passage for all nine. It preserved all three deliberately structured passages and completed zero of five required repairs. Missed edits included actor attribution, ordering an exception before a general instruction, and subjectโverb agreement. No distinct delivered rewrites were available to measure author style. Larger or different passages require separate evaluation.
The checkpoint's upstream Apache 2.0 license is included in LICENSE.
- Downloads last month
- 132
Model tree for kalcy0n/rewriter
Base model
Qwen/Qwen3-4B-Instruct-2507
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="kalcy0n/rewriter") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)