reiglabs/shellm-0.6B

SheLLM 0.6B

Turn a short English request into a Linux shell command.

Request: enter the root directory
Command: cd /

SheLLM is an experimental English-to-shell model developed and trained by reiglabs, a two-person team. It adapts the language knowledge of Qwen/Qwen3-0.6B-Base with supervised LoRA fine-tuning. Its interface produces command text for a person or application to review.

The project explores training, tokenization, transformers, functional evaluation, and inference engineering on consumer hardware. Browser inference and custom WebGPU kernels are future project goals; this release uses PyTorch, Transformers, and PEFT.

Latest selected checkpoint: adapters/2026-10-19-pilot-v4-functional-dev-lora/epoch-2. It scores 103/112 (92.0%) on ShellBench v1 and 156/300 (52.0%) on ShellBench Extra v1. These are functional results from two Docker fixtures per case. Epoch 2 was selected using a separate, fixed functional validation sample before either benchmark was run. The archive also includes epochs 1 and 3, along with earlier experiments.

Compared with pilot-v3, Extra improves by 13 net cases (39 new passes, 26 regressions), while v1's total is unchanged. Fields remain weak, and directory creation and sorting regressed; improvement is uneven across capabilities.

Model details

Field	Description
Developed and trained by	reiglabs
Team	Mátyás Tóth and Csanád Alattyányi
Shared by	reiglabs
Model type	Approximately 0.6B-parameter causal language model with task-specific LoRA adapters
Input / output language	English requests / Linux shell command text
License	Apache License 2.0, selected by reiglabs for the adapter release
Base model	Qwen/Qwen3-0.6B-Base
Pinned base revision	da87bfb608c14b7cf20ba1ce41287e8de496c0cd
Adaptation method	Supervised fine-tuning of fresh rank-8 LoRA adapters, with frozen Base weights
Latest dataset	Authored pilot-v4 shell-translation release
Latest training date	2026-10-19, recorded in Europe/Budapest time
Funding	No external funding information is recorded in the project reports

Sources and contact

• Model and checkpoint archive: reiglabs/shellm-0.6B.
• Code, training data, documentation and benchmark definitions: matyas-toth/shellm on GitHub.
• Latest experiment notebook: pilot-v4 pipelines, fields and functional selection.
• Published measurements and charts: pilot-v4 report archive.
• Interactive local interface: chat.py.
• Paper: There is no dedicated SheLLM paper linked in the project. See the Qwen3 base model card for upstream model references.
• Hosted demo: No hosted SheLLM demo is documented for this release.
• Contact: Open a model discussion or a GitHub issue.

Uses

Direct use

Translate short requests for everyday Linux filesystem and text operations into candidate commands. Typical topics include navigation, listing files, creating or moving files and directories, searching content, reading text, counting, finding files, and changing permissions.

The intended interaction is request → generated command → human review. The inference example below prints text. Working directory, shell state, available utilities and filesystem contents are supplied by the eventual execution environment.

Downstream use

Use the adapters for educational experiments, local command suggestion tools, research into command composition, and further supervised fine-tuning with independently checked data. The source repository provides editable intent catalogs, rendered training examples, and functional evaluation tooling for contributors.

Out-of-scope use

General-purpose conversation, unrestricted system administration, Windows PowerShell, arbitrary shell dialects, production automation without review, and autonomous execution with elevated permissions have not been established by these experiments. Current benchmark results do not establish competence across all Linux tools or environments.

Bias, risks and limitations

• Command composition is still weak. On Extra v1, pipeline coverage passes 10/32 cases, multi-step coverage 9/18, and conditional coverage 6/13. Basic-command results should not be extrapolated to complex workflows.
• Syntax can be valid while the intent is wrong. All latest v1 commands pass syntax checks, but nine fail functional evaluation. The latest Extra checkpoint passes 296/300 syntax checks and 156/300 functional checks.
• Arguments require care. Filenames with spaces or shell metacharacters, path interpretation, quoting, exact flag requirements and combinations of operations can be mishandled.
• Generated commands can change or delete data. Review the command and arguments before executing it. Interactive confirmation and constrained execution environments are appropriate for tools that use the model.
• Training requests are synthetic and template-based. They reflect the authors' selected capabilities and phrasing. The model has not been evaluated across broad user demographics, languages or accessibility needs; robustness to arbitrary natural language is not established.
• Validation is limited. New recipes have independently authored validation phrasing, disjoint argument roots and ten validation-only operator combinations checked against accumulated typed training programs. Other argument variants share recipe structure, and inherited data keeps weaker splits. The 300-request functional validation sample selects checkpoints and is not an unbiased final test.
• ShellBench v1 is a development diagnostic. Its aggregate gaps informed data expansion. It is not a blind generalization test. Extra v1 is a separate progress benchmark outside the training and validation splits.
• Functional checks cover finite fixtures. Matching two Docker environments does not establish correctness on every machine, directory tree, locale, utility version or shell state.
• Progress includes regressions. The latest adapter improves overall scores while losing some previously passing cases. Low completion loss and higher aggregate accuracy do not make every request reliable.
• Context and decoding differ from a chat assistant. Fine-tuning examples use a raw request prefix and short command completion. This checkpoint has not been evaluated as a multi-turn chat assistant or for long-context requests.

Recommendations

Use the exact prompt format below, begin with simple operations, and inspect the generated command. Build downstream checks for argument handling and filesystem effects. Keep validation and benchmark examples separate from training, and assess changes with functional tests as well as syntax and exact-string metrics.

Getting started

This repository stores adapters in run/epoch subfolders. Load the selected subfolder together with the pinned Qwen Base model. A generic root-level pipeline is not sufficient to select an archived checkpoint.

The project's recorded environment uses Python 3.12, Transformers 4.57.6, PEFT 0.18.1, and Hugging Face Hub 0.36.2, with a suitable PyTorch installation. Follow the project setup instructions for the CUDA environment.

from pathlib import Path

import torch
from huggingface_hub import snapshot_download
from peft import PeftConfig, PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "reiglabs/shellm-0.6B"
subfolder = "adapters/2026-10-19-pilot-v4-functional-dev-lora/epoch-2"

# Fetch only the selected adapter checkpoint.
snapshot = snapshot_download(repo_id, allow_patterns=[f"{subfolder}/*"])
adapter_path = Path(snapshot) / subfolder
config = PeftConfig.from_pretrained(str(adapter_path))

device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if device == "cuda" else torch.float32

tokenizer = AutoTokenizer.from_pretrained(
    config.base_model_name_or_path, revision=config.revision
)
base = AutoModelForCausalLM.from_pretrained(
    config.base_model_name_or_path, revision=config.revision, dtype=dtype
).to(device)
model = PeftModel.from_pretrained(base, str(adapter_path)).eval()

request = "enter the root directory"
prompt = f"Request: {request}\nCommand:"
inputs = tokenizer(prompt, return_tensors="pt").to(device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        do_sample=False,
        max_new_tokens=64,
        stop_strings=["\n"],
        tokenizer=tokenizer,
        pad_token_id=tokenizer.eos_token_id,
    )

completion = tokenizer.decode(
    output[0, inputs.input_ids.shape[-1]:], skip_special_tokens=True
)
command = completion.partition("\n")[0].strip()
print(command)  # Command text for review; no shell execution here.

Use a fixed Hub commit revision in snapshot_download when reproducing an experiment. The checkpoint metadata records the base revision and adapter hash. The existing tokenizer chat template is an upstream tokenizer artifact; inference for this task uses the raw prefix shown above.

Training details

Training data

The complete pilot-v4 release contains 30,240 examples: 24,872 training and 5,368 validation examples across 81 metadata families, 5,965 scenario groups and 5,921 unique command strings. It preserves pilot-v3 and adds 4,000 examples (3,600 training / 400 validation) focused on pipelines, fields, traversal/counts, literal arguments and replay of earlier skills. Paraphrases from one scenario remain in the same split.

Labels are rendered from structured command intents, including typed operation programs. Natural-language requests are authored around those intents, and commands are checked against independent Python intent oracles in disposable Docker fixtures. All 5,965 groups passed two fixtures each: 11,930 label executions. The pretraining regression suite passed 192 tests. These checks validate label behavior; they do not by themselves prove every English paraphrase expresses that behavior correctly.

The new portion uses four independently authored training formulations and two separate validation formulations for each of ninety fitted recipes; ten additional combinations appear only in validation. Normalized benchmark request overlap and exact reserved Extra command-label overlap are zero. Literal separation does not establish semantic independence. Table recipes cover simple unquoted records rather than general CSV parsing.

• Editable catalogs.
• Quality checks and review pack.
• Contributor guide.
• Dataset SHA-256: c1a055f2e38643e05769f4ca0a013a642548dbb3d1ff50ebcac3f22b442032cc.

The broader language knowledge comes from the upstream pretrained Qwen model. Consult its model card for upstream training information.

Procedure and preprocessing

Each experiment starts with the pinned Base model and a fresh LoRA adapter. Latest training loads the complete pilot-v4 snapshot once. Requests use Request: <request>\nCommand:; command completions have a leading space and an EOS token. The supervised loss covers the completion and EOS, while prompt and padding tokens are masked. All examples passed tokenizer-boundary/mask/EOS checks without truncation; the longest complete example is 142 tokens within the 160-token limit.

Latest training hyperparameters

Setting	Value
Epochs	3
Optimizer / schedule	AdamW / cosine decay
Learning rate	0.0002
Warmup	122 updates
Microbatch / effective batch	4 / 16
Maximum sequence length	160 tokens
Seed	42
Gradient clipping / weight decay	1.0 / 0.0
LoRA rank / alpha / dropout	8 / 16 / 0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Bias adaptation	None
Precision	FP16 frozen Base and autocast; FP32 adapters and optimizer
Trainable parameters	5,046,272 of 601,096,192 total, approximately 0.84%
Full three-epoch run: updates / example presentations	4,665 / 74,616
Selected epoch 2: updates / example presentations	3,110 / 49,744
Skipped updates	0

This is a LoRA experiment with unquantized Base weights; QLoRA and browser-oriented quantization are possible future experiments.

Sizes and times

The latest adapter weights occupy 20,236,472 bytes (approximately 20.2 MB), excluding the Base model, tokenizer and other files. The full three-epoch training loop took 10,508.0 seconds / 175.1 minutes, including validation and checkpoint saving, excluding model loading. Peak live PyTorch allocation was 2.140 GiB, with 2.371 GiB peak reserved memory. These are full-run training measurements, not browser memory or isolated inference benchmarks.

Validation completion loss was 1.080625 before training, 0.039829 after epoch one, 0.052930 after epoch two, and 0.060530 after epoch three. Later epochs had higher token loss; functional selection measured a different outcome. A fixed 300-request validation sample (100 earlier-data groups and 200 new groups) was generated and executed on two fixtures for every epoch. Selection weighted earlier-data and new-data functional accuracy equally: 67.0%, 73.5%, and 73.25% for epochs 1, 2, and 3. Epoch 2 was selected before benchmark generation. Only that selected checkpoint was benchmarked; all three checkpoints are archived.

Evaluation

Testing data and protocol

ShellBench v1 contains 112 requests across 14 basic command families, with varied wording, arguments, quoting, options and some compositions. ShellBench Extra v1 contains 300 separate requests across 20 families, including practical, compositional and edge cases. Extra is a long-term progress measure outside training and validation.

Reference commands are validated independently. Each extracted model command runs on two disposable Docker fixtures. The evaluator compares the required stdout/stderr, exit status, working directory and observable filesystem effects under its case-specific rules. A functional case passes only when both fixtures pass.

Base and the latest adapter share the pinned model revision, raw prompt format, greedy decoding, FP16, generation batch size eight, a 64-token limit, and newline/EOS stopping. Suite, evaluator, Docker-image, prediction and adapter fingerprints are recorded with the results.

Metrics and latest results

• Functional accuracy: correct behavior on both fixtures.
• Exact string match: an extracted command matching a reference label; alternative valid commands can differ.
• Syntax: the command passes the shell syntax check.
• Format: the raw completion conforms to the command-only output contract.
• Usable: both functional and correctly formatted.

Benchmark	Model	Functional accuracy	Exact match	Syntax	Format	Usable
ShellBench v1	Qwen3 0.6B Base	30/112 (26.8%)	14/112	108/112	111/112	30/112
ShellBench v1	SheLLM pilot-v4 selected epoch 2	103/112 (92.0%)	88/112	112/112	112/112	103/112
ShellBench Extra v1	Qwen3 0.6B Base	20/300 (6.7%)	10/300	281/300	265/300	17/300
ShellBench Extra v1	SheLLM pilot-v4 selected epoch 2	156/300 (52.0%)	66/300	296/300	300/300	156/300

Historical selected-checkpoint results

Run	Training examples	ShellBench v1	Extra v1
Original pilot, 2026-10-01	1,008	90/112 (80.4%)	65/300 (21.7%)
Expanded pilot, 2026-10-03	2,072	95/112 (84.8%)	77/300 (25.7%)
pilot-v2, epoch 3, 2026-10-10	18,392	104/112 (92.9%)	103/300 (34.3%)
pilot-v3, epoch 3, 2026-10-18–19	21,272	103/112 (92.0%)	143/300 (47.7%)
pilot-v4, selected epoch 2, 2026-10-19	24,872	103/112 (92.0%)	156/300 (52.0%)

Against pilot-v3 epoch 3, v1 has four newly passing cases and four regressions; Extra has 39 newly passing cases and 26 regressions. The same cases, evaluators, image and decoding were used. Data, training duration and checkpoint selection differ, so these comparisons do not isolate the effect of dataset size.

Measured factors include command family, capability coverage and difficulty. Extra pipeline coverage improved 5/32 to 10/32, counting 3/15 to 6/15, listing 2/15 to 4/15, and writing 7/15 to 10/15. Fields stayed 3/15, directory creation fell 11/15 to 8/15, and sorting fell 8/15 to 7/15. Small category denominators make a few changed cases large percentage swings. Coverage groups and family groups overlap and must not be summed.

Detailed machine-readable reports and charts are in this archive; benchmark cases and evaluator code are in the source repository. Raw completions, extracted commands, syntax checks and Docker functional outcomes are recorded separately. Functional development scores remain outside the benchmark-history series.

Model examination

The project examines behavior through held-out examples, independent label oracles, functional fixtures, and per-case failure/regression reports. No dedicated mechanistic interpretability or attention-analysis study is reported for this checkpoint. The benchmark results measure observed behavior within the test contract.

Technical specifications and compute

The architecture is the upstream Qwen3 causal decoder transformer with LoRA updates on attention and MLP projections. Fine-tuning minimizes next-token cross-entropy on command completions and EOS. The pretrained tokenizer and Base weights are reused, with a raw task prefix instead of a chat interaction format.

Training ran locally on an NVIDIA GeForce RTX 2060 with 6 GiB VRAM, using Ubuntu in WSL2 on Windows. Recorded software: Python 3.12, PyTorch 2.11.0+cu130, Transformers 4.57.6, PEFT 0.18.1, Accelerate 1.15.0, and Hugging Face Hub 0.36.2. The successful run used PYTORCH_ALLOC_CONF=expandable_segments:True. Functional evaluation uses disposable Linux Docker containers.

The selected adapter SHA-256 is 51a5fd7be691180a03a8ce9ff3f928d7f29c98089a96874b56e8527c5445b828. The pinned revision, dataset hash, per-epoch metadata and source fingerprints support reproduction of the recorded experiment.

Environmental impact

• Hardware: Local RTX 2060, 6 GiB VRAM.
• Successful latest full training-loop duration: Approximately 2.92 hours; this excludes model loading and benchmark execution. The selected adapter was saved after epoch 2 of this three-epoch run.
• Cloud provider: Not applicable to the recorded local training run.
• Compute region: Physical location is not recorded in the training metadata; Europe/Budapest is the reporting timezone.
• Energy use and carbon emissions: Not measured. No quantitative emissions claim is made.

Licensing

reiglabs releases these LoRA adapter weights under the Apache License 2.0. The upstream Qwen3-0.6B-Base model is also distributed under Apache 2.0. Retain applicable license and attribution notices when redistributing the model or derivative artifacts.

Citation

There is no dedicated SheLLM publication. A suggested citation for this model release is:

@misc{reiglabs_shellm_2026,
  author = {Tóth, Mátyás and Alattyányi, Csanád},
  title = {SheLLM 0.6B: English-to-Linux-Shell LoRA Adapters},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/reiglabs/shellm-0.6B}
}

Please also credit the upstream Qwen model when describing work based on its pretrained weights.

Glossary

• LoRA: Trainable low-rank updates attached to selected frozen model layers.
• SFT: Supervised fine-tuning using request/command examples.
• Scenario group: One structured intent with related request paraphrases kept in the same split.
• Functional evaluation: Executing a generated command in fixtures and comparing its required observable behavior.
• Checkpoint: The saved adapter state after an epoch, identified by run, epoch and weight hash.

Model card authors and maintenance

This card is published by reiglabs, the team of Mátyás Tóth and Csanád Alattyányi, and was prepared from the project's preserved training/data/evaluation records at the team's request. Snapshot date: 2026-10-19 (Europe/Budapest). Results refer to the explicitly identified checkpoint; future archive additions require fresh measurements before updating these claims.