How to use from
Docker Model Runner
docker model run hf.co/gustavecortal/PhiloLM
Quick Links

PhiloLM

PhiloLM is a language model with 112M parameters, trained from scratch on open philosophy data (PhiloText). A web demo runs it in your browser.

Architecture

PhiloLM architecture in two panels. Model: token IDs feed a token embedding and a bigram embedding, then four groups of four blocks, each with three local attention blocks (256-token window) and one global attention block (9 query heads, 3 key-value heads), a final RMSNorm and an LM head tied to the token embedding. Decoder block: the bigram vector, times a learned scale, is added to the first 128 of 576 dimensions of the residual stream, then RMSNorm and causal attention (QK norm, partial RoPE), then RMSNorm and a SwiGLU feed-forward network, each with a residual connection

Training

The model processed 1.45B tokens across four source-weighted passes over PhiloText's 450.8M training tokens. Training used document-masked attention, with windows growing from 1,024 to 2,048 tokens, Muon for hidden weight matrices, and AdamW for the remaining parameters. The final pass emphasizes reference sources and selected primary texts. The release blends final weights with a late-training exponential moving average. Training took 15.2 hours on one Intel Arc A770 (16 GB).

Examples

Continuations sampled as on the web page, cut after the first paragraph. The prompt is in bold.

A good life is a state of well-being that is characterized by the absence of suffering and the fulfillment of human potential. It may also be described as a positive experience, a state of happiness, or a state of being in a good mood.

Justice is a concept that has been used to describe the relationship between individuals, groups and institutions. It can be used to refer to what society considers fair or just, but also to apply it in more general ways such as when people are treated equally.

According to Kant, the moral law is the categorical imperative, or a law of nature. It is an idea that can be deduced from reason and not merely derived from experience. In this way it provides a basis for morality, which Kant calls the "Categorical Imperative".

Hume argued that causation is a matter of constant conjunction, where the term "cause" (or "substance") refers to all or most of the occurrences that we have observed. The notion of causation in Hume's philosophy has been interpreted as being about regularities and causal relations between events. In this view, there are no necessary connections among things. Instead, all our experience can be explained by reference to these patterns of observation.

Husserl's phenomenology begins with the question of what is common to all human experience. It asks: "What is it for a given thing to be present in the world?" (p. 1). In other words, how do we get from there to the objects that we encounter and their properties? What is it that makes an object appear as such?

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gustavecortal/PhiloLM")
model = AutoModelForCausalLM.from_pretrained("gustavecortal/PhiloLM", trust_remote_code=True)
inputs = tokenizer("The problem of personal identity", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(output[0], skip_special_tokens=True))

For faster CPU generation, generate_fast.py runs the web page's 8-bit WebAssembly engine, about three times faster than PyTorch:

pip install wasmtime numpy tokenizers
hf download gustavecortal/PhiloLM core.wasm core.bin tokenizer.json generate_fast.py --local-dir PhiloLM
cd PhiloLM && python generate_fast.py "The problem of personal identity"

Limitations

PhiloLM is a base checkpoint without instruction tuning. It can repeat itself, lose argumentative coherence, and invent citations or attributions. For factual use, verify claims against the original sources.

Weights and inference code are released under Apache-2.0.

Citation

@misc{cortal2026philolm,
  author = {Cortal, Gustave},
  title = {PhiloLM: a small language model trained on philosophy},
  year = {2026},
  url = {https://huggingface.co/gustavecortal/PhiloLM}
}
Downloads last month
222
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Dataset used to train gustavecortal/PhiloLM