Instructions to use tmickleydoyle/ghost-typer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tmickleydoyle/ghost-typer with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tmickleydoyle/ghost-typer")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tmickleydoyle/ghost-typer")
model = AutoModelForCausalLM.from_pretrained("tmickleydoyle/ghost-typer", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tmickleydoyle/ghost-typer with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tmickleydoyle/ghost-typer"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tmickleydoyle/ghost-typer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/tmickleydoyle/ghost-typer

SGLang

How to use tmickleydoyle/ghost-typer with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tmickleydoyle/ghost-typer" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tmickleydoyle/ghost-typer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tmickleydoyle/ghost-typer" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tmickleydoyle/ghost-typer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use tmickleydoyle/ghost-typer with Docker Model Runner:
```
docker model run hf.co/tmickleydoyle/ghost-typer
```

ghost-typer / data /SOURCES.md

tmickleydoyle

ghost-typer: float32 transformers model, compiler and docs

c160abb about 4 hours ago

preview code

Raw History Blame Contribute Delete

3.42 kB

Data sources

prompts.txt, replies.txt and wildchat.txt are produced by scripts/fetch_coding_prompts.py. Every line is English prose typed by people, taken from the datasets below. LLM-generated messages are excluded: the OASST synthetic flag, and WildChat's assistant turns, which are never used. The script keeps only prose: it removes code blocks, code-looking lines and markdown, keeps up to 2 sentences per prompt and 5 per reply, and keeps sentences of 3–24 words.

File	Dataset	What is used	License	Attribution
`prompts.txt`	OpenAssistant/oasst2	English user messages (all turns, reviewed, not deleted, not synthetic) from conversations whose opening prompt is about code	Apache-2.0	Köpf et al., OpenAssistant Conversations: Democratizing Large Language Model Alignment, 2023
`prompts.txt`	google-research-datasets/mbpp	task descriptions (`text`), all splits	CC-BY-4.0	Austin et al., Program Synthesis with Large Language Models, 2021
`prompts.txt`	neulab/conala	curated intents (`intent`), paired train + test	MIT	Yin et al., Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow, MSR 2018
`replies.txt`	OpenAssistant/oasst2	English assistant messages, written by human volunteers, from the same conversations	Apache-2.0	as above
`wildchat.txt`	allenai/WildChat-1M	English user turns from conversations whose first user turn is about code. Conversations flagged toxic and turns with redacted PII are skipped.	ODC-BY	Zhao et al., WildChat: 1M ChatGPT Interaction Logs in the Wild, ICLR 2024

Counts from the last fetch:

Source	Sentences
OASST user turns	4,169
MBPP	938
CoNaLa	2,820
OASST assistant turns	13,183
WildChat user turns	127,490

After removing duplicates and credential-like sentences: 7,132 lines in prompts.txt, 12,840 in replies.txt and 105,373 in wildchat.txt.

Credentials: some users paste API keys and tokens into prompts. Before filtering, the WildChat text contained 3 OpenAI keys, 4 Google API keys and 2 Telegram bot tokens. Every sentence matching a known key format, or containing a 32+ character token that mixes letters and digits, is dropped. That removed 239 lines, mostly tracebacks and file paths. test_model.py checks that no data line or vocabulary entry matches.

WildChat caveat: the user turns are what people typed into ChatGPT. Some users paste text (including earlier model output) into their messages, so "typed by people" is not perfectly clean. Only patterns that repeat across many prompts survive pruning, and a suggestion is shown only above a confidence threshold, which limits how much any single pasted message can influence what ghost-typer suggests.

The weights contain only n-gram statistics derived from these lines, plus the hand-written phrases in ../phrases/. No dataset was used to train anything. 10% of prompts.txt (chosen by content hash) is held out when measuring precision and keystrokes saved (scripts/sweep.py, scripts/benchmark.py).