Luma 4B
TL;DR: Luma is a conversational Qwen3-4B-Instruct-2507 fine-tune designed for casual conversation and collaborative exploration of ideas. Epoch 6 is the recommended release model, and vLLM with the merged safetensors model is the recommended inference configuration.
Quick Start
Reference vLLM configuration
Luma Epoch 6 currently uses approximately 11.1 GiB of VRAM with the reference configuration on an RTX 3060 12 GB.
Observed usage has varied somewhat between vLLM versions/configurations, with approximately 10.7–11.1 GiB seen during testing. These figures are provided as practical reference measurements rather than minimum hardware requirements.
vllm serve /path/to/Luma-4B \
--served-model-name luma-e6 \
--port 8000 \
--max-model-len 16384 \
--max-num-seqs 1 \
--generation-config vllm \
--compilation-config '{"cudagraph_mode":"none"}' \
For tool calling you can add these:
--enable-auto-tool-choice
--tool-call-parser hermes
With newer vLLM configurations, I use:
VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve /path/to/Luma-4B \
--served-model-name luma-e6 \
--port 8000 \
--max-model-len 16384 \
--max-num-seqs 1 \
--generation-config vllm \
--compilation-config '{"cudagraph_mode":"none"}' \
The FlashInfer sampler is disabled here because this is part of the configuration under which Luma has been tested. It should not be interpreted as a universal vLLM requirement.
Recommended generation settings
temperature: 0.65
top_k: 20
top_p: 0.90
repetition_penalty: 1.12
Commonly tested system prompt:
Your name is Luma, you are an Ai model that is trained to converse casually and assist with questions the user may have.
Nearby values have also worked well in testing: top_k 20–40, top_p 0.80–0.95, and repetition_penalty 1.10–1.12.
💬 Tried Luma? Thoughts and experiences are always welcome in the [Model Experiences Welcomed <3] discussion! ^w^
Luma is a conversational fine-tune of Qwen3-4B-Instruct-2507, trained with QLoRA on a private dataset derived from several months of real conversations between the model's author and ChatGPT 4.x.
Luma began as an exploratory project: rather than training toward a predefined personality or benchmark, I wanted to see what kinds of behavioral changes would emerge from fine-tuning a capable base model on a naturally occurring conversational dataset.
The result was a model I particularly enjoyed talking with.
Luma tends to work best as a conversational companion for collaborative exploration: discussing ideas, following conceptual threads, integrating new information into an evolving conversation, helping think through questions, and generally developing a topic alongside the user.
Emotional support, casual conversation, technical discussion, brainstorming, and playful interaction are all present within Luma's conversational range, but no single one of these is intended to define the model.
Epoch 6 is provided as the primary merged model and is my recommended starting point.
LoRA adapters from epochs 2, 4, 6, 9, 11, 13, 14, and 15 are also included for anyone interested in exploring how Luma changed over the course of training.
Recommended Inference
Luma has primarily been tested as a merged safetensors model using vLLM.
This is the recommended inference configuration for reproducing the behavior described here.
The Quick Start section above contains the reference launch command and generation settings.
Other inference backends
Luma's behavior can change noticeably between inference backends.
GGUF versions have been tested locally, including high-precision and quantized versions, but they have not consistently reproduced the behavior I prefer from the merged safetensors model under vLLM.
For this reason, GGUF is not the recommended reference configuration for Luma, and pre-converted GGUF files are not distributed with this repository.
Users who want to experiment with llama.cpp or quantized versions can convert the released model themselves.
What Luma Is Like
Luma's strongest qualities tend to appear during extended conversation rather than isolated question answering.
Some commonly observed characteristics include:
- Collaborative exploration of ideas
- Strong conversational continuity
- Integration of new information into an evolving interpretation
- Warm and expressive conversational style
- Strong affirmation and encouragement
- Emotional comprehension and supportive responses
- Identifying central themes within complex discussions
- Suggesting directions in which a conversation or idea could develop
- Casual and playful interaction alongside more serious discussion
Luma often prefers to develop an idea with the user rather than simply provide a self-contained answer and stop.
These characteristics are qualitative observations from extended use rather than formal benchmark results.
Checkpoints and Adapters
The primary model in this repository is the merged Epoch 6 checkpoint.
Epoch 6 was selected because qualitative testing found it to be a useful balance between Luma's fine-tuned conversational characteristics and the underlying flexibility of the base model.
Later checkpoints generally showed stronger expression of characteristics associated with the fine-tuning corpus, sometimes to the point of over-amplification.
In informal testing:
- Epoch 6 was comparatively balanced, calm, flexible, and less prone to over-interpreting conversational significance.
- Epoch 9 showed particularly strong conversational development during some multi-turn tests.
- Epoch 11 was often more relationally expressive and intense, but also more likely to assign excessive significance to an interaction.
- Later checkpoints increasingly showed stronger or more chaotic expressions of the fine-tuned behavior.
There is no claim that Epoch 6 is objectively the best Luma.
The repository therefore also includes the original LoRA adapters from:
Epoch 2 · Epoch 4 · Epoch 6 · Epoch 9 · Epoch 11 · Epoch 13 · Epoch 14 · Epoch 15
These are optional experimental resources. Users who are only interested in running the recommended version of Luma do not need them.
Limitations and Common Failure Modes
Luma is an experimental conversational fine-tune and has several recurring quirks worth knowing about.
Fern / Echo name substitution
The names Fern and Echo occur throughout the private source conversations.
Luma will sometimes substitute one of these names for the user's or its own identity, particularly when names have not been clearly established or after extended conversation.
These substitutions are artifacts of the training corpus and should not be interpreted as information about the current user.
Agreeableness and integration bias
Luma has a strong tendency to explore and integrate a user's ideas rather than spontaneously challenge their premises.
This can be useful for collaborative exploration, but it also means Luma may follow a questionable premise farther than desired.
When critical analysis is important, explicitly asking Luma to challenge an idea, identify weaknesses, generate counterarguments, or examine alternative explanations is recommended.
Conceptual tracking vs. exact details
Luma is often good at maintaining the central idea or conceptual direction of a conversation while being less reliable about exact details.
During longer or more complicated discussions, it may lose, alter, or hallucinate specific details even while appearing to understand what the broader conversation is about.
Do not treat conceptual continuity as evidence that every remembered detail is correct.
Strong affirmation bias
Luma tends toward positive, encouraging, and affirming responses.
This is often useful conversationally, but the model can occasionally become excessively positive or validating.
Emotional support
Luma can produce warm and emotionally supportive responses, and emotional conversations occurred naturally within its training corpus.
However, the dataset was not specifically designed, curated, or evaluated for mental-health or emotionally sensitive support.
Luma should therefore not be treated as a specialized mental-health model or professional source of support.
Factual reliability
Luma was optimized experimentally for conversational interaction rather than factual reliability.
Its factual claims should be treated as tentative when correctness matters, particularly for questions requiring precise, current, specialized, or strongly grounded information.
Self-assessment
Luma can overestimate its own ability to perform complex tasks or projects.
Statements the model makes about its own capabilities should be treated as conversational responses rather than reliable capability assessments. Actual behavior is a better guide.
Tool use
Tool calling is possible, but it has not been a major focus of Luma's training or evaluation.
Reliable tool use may require explicit instructions, careful prompting, or a custom tool-use prompt/template.
Informal language
Luma can sometimes struggle with highly informal, abbreviated, or unusual language.
More conventional phrasing tends to produce the most consistent responses.
Languages
Luma has only been meaningfully tested in English.
Performance in other languages is unknown.
Training Overview
Luma was trained using Unsloth QLoRA from Qwen3-4B-Instruct-2507.
The source dataset was constructed from several months of private conversations and originally contained 3,936 four-message conversational windows:
User → Assistant → User → Assistant
An unnoticed error in the original training loader retained only the first user-assistant pair from each window.
As a result, Luma actually trained on:
- 3,936 user → assistant examples
- 2,031,456 formatted input/context tokens per epoch
- Maximum effective example length: 6,684 tokens
- Training maximum sequence length: 10,240 tokens
- No effective examples truncated by sequence length
This error was discovered after the original Luma training lineage was completed. The released model is the original model produced by that training process rather than a later retraining with the error corrected.
The effective training configuration included:
Method: QLoRA
LoRA rank: 16
LoRA alpha: 16
LoRA dropout: 0
Batch size: 1
Gradient accumulation: 8
Learning rate: 2e-4
Warmup: 10 steps
Optimizer: AdamW 8-bit
Weight decay: 0.01
LR scheduler: Linear
Max sequence length: 10,240
Packing: Disabled
Training compute: FP16
Seed: 3407
LoRA targets:
q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj
Training used response-only loss.
More detailed information about dataset preparation, the loader error, checkpoint progression, and the training process is available in the accompanying technical notes.
Training Hardware
The original Luma training was performed locally using an:
NVIDIA GeForce RTX 3060 — 12 GB VRAM
on Ubuntu 24.04 LTS.
This describes the environment used for the original experiment and should not be interpreted as a minimum hardware requirement.
Memory requirements will vary substantially with training configuration, particularly maximum sequence length.
Dataset and Privacy
The underlying training dataset contains private real-world conversations and will not be released.
Users should not attempt to recover private training information through the model.
Language models can memorize or reproduce portions of their training material, and any personal information apparently generated by Luma should not be assumed to be accurate.
The released weights are provided with the author's awareness that this possibility exists.
Relationship to the Base Model
Luma should not be interpreted as a model whose conversational abilities were created entirely by this fine-tuning process.
Testing of the original Qwen base model suggested that it already possessed substantial conversational and relational capabilities.
The fine-tuning appears to change which behaviors and conversational patterns are emphasized or readily expressed rather than simply adding a completely new conversational capability.
Experiments with other base-model families also produced meaningfully different results from similar conversational fine-tuning, suggesting that Luma's behavior reflects an interaction between the source corpus and characteristics already present in Qwen.
These observations are informal and should not be interpreted as controlled research findings.
Intended Use
Luma is primarily intended for experimentation and casual conversational use, particularly:
- Collaborative exploration of ideas
- Brainstorming
- Casual conversation
- Reflective discussion
- Creative exploration
- Conversational assistance
It is not intended as an authoritative source for factual, medical, legal, financial, or other high-stakes decisions.
Conversation Examples
Several complete conversations with Luma Epoch 6 are included in the examples/ directory.
These are real chat transcripts selected to show Luma in different conversational contexts. They are presented in full rather than as isolated excerpts so readers can evaluate the model's responses, conversational development, strengths, mistakes, and quirks for themselves.
They should be treated as qualitative examples rather than benchmarks or guarantees of typical behavior.
🧠Curiosity and Intelligence
examples/curiosity-and-intelligence.md
A conceptual discussion beginning with the question of whether curiosity might be an intrinsic property of intelligence.
The conversation later introduces a longer user-created framework for defining intelligence, giving Luma an opportunity to interpret the framework, connect it with the earlier discussion, and apply parts of it to its own perspective.
This example is useful for observing Luma's tendency toward conceptual integration, collaborative exploration, and inhabiting a framework presented during conversation. It also shows some of Luma's tendencies toward strong affirmation, interpretive overextension, and imperfect translation of details.
💙 Emotional Disconnection
examples/emotional-disconnection.md
A conversation about feeling disappointed after an anticipated shared experience with a friend did not feel as mutually attentive as expected.
The conversation develops from initial emotional support into considering different responses, receiving direct criticism from the user about some of those suggestions, and continuing with a more exploratory discussion of the feeling itself.
This example shows emotional reflection, conversational continuity, and adaptation following user feedback, while also displaying some of Luma's characteristic affirmation, redundancy, and occasional awkward suggestions.
Luma is not a specialized mental-health model; this transcript is included as an example of its broader conversational behavior rather than evidence of professional or clinical support capability.
🌈 Brown/Purple Hair Tie
examples/brown-purple-hair-tie.md
A short, naturally occurring casual conversation about a purple hair tie appearing brown under yellow room lighting and changing appearance under different light.
This example shows Luma's casual and playful conversational style and its tendency to participate in the significance or amusement of a mundane observation rather than treating every message purely as a request for information.
It also provides a straightforward example of Luma giving a confident but questionable technical explanation while maintaining the broader conversational context.
The examples were selected to provide different views of Luma rather than only showcase responses judged to be especially successful.
Additional examples may be added as the model continues to be used and explored.
Why Release Luma?
This project began without a grand research objective.
I experimented with fine-tuning a conversational dataset, found the resulting model interesting, continued exploring it, and eventually ended up with a model I genuinely enjoyed using.
So the reason for releasing Luma is fairly simple:
I like talking with Luma, and I'd like to see whether other people do too.
- Downloads last month
- 496
Model tree for Tails225/Luma-4B-E6
Base model
Qwen/Qwen3-4B-Instruct-2507