Text Generation
Transformers
Safetensors
English
philolm
philosophy
base-model
trained-from-scratch
small-language-model
n-gram-embedding
muon
webassembly
custom_code
Instructions to use gustavecortal/PhiloLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use gustavecortal/PhiloLM with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="gustavecortal/PhiloLM", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("gustavecortal/PhiloLM", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use gustavecortal/PhiloLM with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "gustavecortal/PhiloLM" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gustavecortal/PhiloLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/gustavecortal/PhiloLM
- SGLang
How to use gustavecortal/PhiloLM with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "gustavecortal/PhiloLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gustavecortal/PhiloLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "gustavecortal/PhiloLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gustavecortal/PhiloLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use gustavecortal/PhiloLM with Docker Model Runner:
docker model run hf.co/gustavecortal/PhiloLM
Download generate.py from gustavecortal/PhiloLM: direct link, hf CLI and curl.
- Browser
- Download file 1.94 kB
-
https://huggingface.co/gustavecortal/PhiloLM/resolve/main/generate.py
- Command line
-
hf download hf://gustavecortal/PhiloLM/generate.py
-
curl -L -o generate.py https://huggingface.co/gustavecortal/PhiloLM/resolve/main/generate.py
1.94 kB
| """Continue a text with the model: python generate.py "The problem of personal identity" | |
| Needs torch, safetensors and tokenizers. Sampling as on gustavecortal.com/philosophy-lm/: presence penalty 2.0, | |
| then temperature 0.7, top-k 20 and top-p 0.8. A text is continued as a new document (the end-of-text token first), | |
| as in training.""" | |
| import argparse | |
| import json | |
| import torch | |
| from safetensors.torch import load_file | |
| from tokenizers import Tokenizer | |
| from decode import Decoder | |
| from model import Transformer | |
| p = argparse.ArgumentParser() | |
| p.add_argument("text") | |
| p.add_argument("--tokens", type=int, default=256) | |
| p.add_argument("--temperature", type=float, default=0.7) | |
| p.add_argument("--top-k", type=int, default=20) | |
| p.add_argument("--top-p", type=float, default=0.8) | |
| p.add_argument("--presence-penalty", type=float, default=2.0) | |
| p.add_argument("--seed", type=int, default=0) | |
| args = p.parse_args() | |
| config = json.load(open("config.json")) | |
| model = Transformer(config["model"], config["vocab_size"]) | |
| model.load_state_dict(load_file("model.safetensors")) | |
| model.eval() | |
| tokenizer = Tokenizer.from_file("tokenizer.json") | |
| eos = config["eos_id"] | |
| torch.manual_seed(args.seed) | |
| decoder, seen, out = Decoder(model), set(), [] | |
| with torch.no_grad(): | |
| logits = decoder.prefill([eos] + tokenizer.encode(args.text, add_special_tokens=False).ids) | |
| for _ in range(args.tokens): | |
| logits = logits.float().clone() | |
| logits[list(seen)] -= args.presence_penalty | |
| top = torch.topk(logits / args.temperature, args.top_k) | |
| probs = torch.softmax(top.values, -1) | |
| keep = (torch.cumsum(probs, -1) - probs) < args.top_p | |
| token = top.indices[keep][torch.multinomial(probs[keep] / probs[keep].sum(), 1)].item() | |
| if token == eos: | |
| break | |
| seen.add(token) | |
| out.append(token) | |
| logits = decoder.step(token) | |
| print(args.text + tokenizer.decode(out)) | |