Instructions to use rotextechnologies/rotexmech-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use rotextechnologies/rotexmech-8B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf rotextechnologies/rotexmech-8B:Q4_K_M # Run inference directly in the terminal: llama cli -hf rotextechnologies/rotexmech-8B:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf rotextechnologies/rotexmech-8B:Q4_K_M # Run inference directly in the terminal: llama cli -hf rotextechnologies/rotexmech-8B:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf rotextechnologies/rotexmech-8B:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf rotextechnologies/rotexmech-8B:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf rotextechnologies/rotexmech-8B:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf rotextechnologies/rotexmech-8B:Q4_K_M
Use Docker
docker model run hf.co/rotextechnologies/rotexmech-8B:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use rotextechnologies/rotexmech-8B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "rotextechnologies/rotexmech-8B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "rotextechnologies/rotexmech-8B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/rotextechnologies/rotexmech-8B:Q4_K_M
- Ollama
How to use rotextechnologies/rotexmech-8B with Ollama:
ollama run hf.co/rotextechnologies/rotexmech-8B:Q4_K_M
- Unsloth Studio
How to use rotextechnologies/rotexmech-8B with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for rotextechnologies/rotexmech-8B to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for rotextechnologies/rotexmech-8B to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for rotextechnologies/rotexmech-8B to start chatting
- Pi
How to use rotextechnologies/rotexmech-8B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf rotextechnologies/rotexmech-8B:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "rotextechnologies/rotexmech-8B:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- OpenClaw new
How to use rotextechnologies/rotexmech-8B with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf rotextechnologies/rotexmech-8B:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "rotextechnologies/rotexmech-8B:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use rotextechnologies/rotexmech-8B with Docker Model Runner:
docker model run hf.co/rotextechnologies/rotexmech-8B:Q4_K_M
- Lemonade
How to use rotextechnologies/rotexmech-8B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull rotextechnologies/rotexmech-8B:Q4_K_M
Run and chat with the model
lemonade run user.rotexmech-8B-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use rotextechnologies/rotexmech-8B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf rotextechnologies/rotexmech-8B:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default rotextechnologies/rotexmech-8B:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- RotexMech-8B โ Public Preview GGUF Model
- 1. Executive Summary
- 2. Why This Model Exists
- 3. Target Audience
- 4. Model Details
- 5. What Is Included โ and What Is Not
- 6. Permitted & Prohibited Uses
- 7. Quick Start
- 8. Evaluation & Benchmarking
- 9. Known Limitations
- 10. Safety & Responsible Disclosure
- 11. Citation
- 12. Commercial Access
- 13. Repository Documents
- 14. Acknowledgements
โ ๏ธ PUBLIC PREVIEW โ NOT OPEN SOURCE. This is a controlled evaluation release. Commercial use, hosted inference, derivative model creation, fine-tuning, distillation, and redistribution are not permitted without a separate written commercial licence from Rotex Technologies Ltd. See license.md.
RotexMech-8B โ Public Preview GGUF Model
1. Executive Summary
RotexMech-8B is a public preview GGUF release developed by Rotex Technologies Ltd โ a UK-based digital manufacturing, engineering services specialising in CFD, FEA, and technical consultancy across the nuclear, aerospace, automotive, marine, wind, solar, energy storage, and oil & gas sectors.
RotexMech-8B is fine-tuned from the Qwen3-8B base model, with a training and evaluation focus on mechanical engineering domains โ structural integrity, materials and corrosion, thermal and fluid systems, dynamics and tribology, manufacturing and DFMA, reliability and lifecycle, rotating equipment diagnostics, and pressure systems integrity.
This release provides a Q4_K_M quantised model (RotexMech-8B-q4_k_m.gguf) intended to demonstrate the capability of our wider commercial model family, whilst preserving our proprietary fullโprecision weights, training pipeline, domainโspecific datasets, and deployment optimisations.
โ This model is suitable for:
- Local, offline evaluation and experimentation
- Technical benchmarking against publicly reported metrics
- Early-stage customer and partner feedback
- Investor and technical due diligence
- Edge and air-gapped inference testing
- Academic and industry technical review
โ This model is not licensed for:
- Commercial deployment or production use
- Hosted inference, SaaS, or API services
- Fineโtuning, distillation, or adapter training
- Derivative model creation of any kind
- Redistribution at scale or mirroring
- Use as a component in a competing product or service
For commercial access, evaluation under NDA, or partnership discussions, contact: ๐ง sales@rotextechnologies.com
2. Why This Model Exists
Most current models in this parameter class are either:
- too large for practical edge and offline deployment;
- too generic for specialist industrial and engineering use;
- costly to host and difficult to validate in regulated environments;
- trained on data of unknown provenance, creating unacceptable IP and compliance risk;
- unsuitable for controlled, auditable engineering workflows.
Rotex Technologies Ltd is building a model family focused on:
- Domain precision โ trained and evaluated on curated mechanical and industrial engineering literature and case material, spanning structural, thermal, materials, dynamics, manufacturing, reliability, and rotating/pressure equipment topics;
- Deployability โ designed for edge, air-gapped, and resource-constrained environments from the outset;
- Auditability โ built for workflows where traceability, provenance, and controlled outputs matter;
- Data security and privacy โ fully air-gapped development pipeline; all processes, from raw document ingestion to model build, are isolated from the internet, ensuring complete confidentiality of datasets and training corpora;
- Commercial clarity โ a transparent licensing model that respects both open evaluation and proprietary IP.
This public preview is a release in our production model pipeline โ a controlled, quantised snapshot sufficient for meaningful technical evaluation without compromising our proprietary training assets, architecture, or development IP.
3. Target Audience
This preview is intended for:
| Audience | Purpose |
|---|---|
| Technical evaluators | Assess model capability, quality, and behaviour |
| Engineering teams | Test domain-specific performance on representative mechanical engineering tasks |
| Enterprise innovation groups | Evaluate fit within controlled internal workflows |
| Investors & due diligence teams | Conduct independent technical assessment |
| Research partners | Explore collaboration potential |
| Early design partners | Inform commercial roadmap and integration requirements |
It is not intended for, and not licensed for, production deployment of any kind.
4. Model Details
| Field | Detail |
|---|---|
| Model name | RotexMech-8B |
| File name | RotexMech-8B-q4_k_m.gguf |
| Base model | Qwen3-8B |
| Release type | Public preview (evaluation only) |
| File format | GGUF |
| Quantisation | Q4_K_M |
| Primary language | English |
| Domain focus | Mechanical engineering โ structural integrity, materials, thermal/fluids, dynamics, manufacturing/DFMA, reliability, rotating equipment, pressure systems |
| Release date | 3 August 2026 |
| Pipeline position | Public preview release; higher-precision variants (BF16, Q8_0) available to qualified commercial customers |
| Licence | Model Public Preview Licence v1.0 |
| Commercial use | Not permitted without written licence from Rotex Technologies Ltd |
| Developer | Rotex Technologies Ltd, United Kingdom |
5. What Is Included โ and What Is Not
โ Public in this release:
- Q4_K_M GGUF model file
- Inference instructions for llama.cpp and compatible runtimes
- Evaluation prompts and example outputs
- Safety, limitation, and known-issue notes
- Public benchmark summary
- Full licence terms and acceptable use policy
โ Not included in this release:
- Higher-precision model variants (BF16)
- Higher-quality quantisations (Q8_0)
- Training, fineโtuning, or alignment datasets
- Fineโtuning recipes, hyperparameters, or training code
- Internal evaluation harness and proprietary benchmarks
- Commercial deployment configuration and system prompts
- Domainโspecific commercial variants
- Any component that would enable derivative model creation
6. Permitted & Prohibited Uses
โ Permitted (subject to licence terms):
- Local, nonโcommercial evaluation and experimentation
- Technical benchmarking and comparison
- Academic and industry review and publication of fair results
- Internal proofโofโconcept assessment
- Investor and partner due diligence
โ Prohibited (without separate written commercial licence):
- Production deployment of any kind
- Commercial use, revenue generation, or cost reduction
- SaaS, API, or hosted inference services
- Fineโtuning, distillation, merging, or adapter creation
- Training or improving any other model (competing or otherwise)
- Redistribution, mirroring, or repackaging at scale
- Any use violating the Acceptable Use Policy
Full terms: license.md
7. Quick Start
Prerequisites
- llama.cpp (built with GPU acceleration recommended)
- Or any GGUF-compatible runtime (Ollama, LM Studio, etc.)
Basic inference with llama.cpp
# Download the model file from this repository
# Then run:
./llama-cli \
-m RotexMech-8B-q4_k_m.gguf \
-p "Explain the key technical considerations for fatigue analysis in pressure vessel design." \
-n 512 \
--temp 0.6 \
--ctx-size 2048
Interactive mode
./llama-cli \
-m RotexMech-8B-q4_k_m.gguf \
--interactive \
--temp 0.6 \
--ctx-size 2048
With Ollama
Create a Modelfile:
FROM ./RotexMech-8B-q4_k_m.gguf
Then:
ollama create rotexmech-8b-preview -f Modelfile
ollama run rotexmech-8b-preview
Suggested evaluation prompts
"Summarise the key failure modes of a welded joint in cooling water pipes under cyclic thermal loading."
"Outline a root-cause investigation approach for bearing overheating in a centrifugal pump."
"Explain the difference between hydrostatic and pneumatic pressure testing for a vessel returning to service after a repair, and the key precautions for each."
8. Evaluation & Benchmarking
We encourage independent benchmarking and technical review. When publishing results:
- โ Attribute the model as: "RotexMech-8B by Rotex Technologies Ltd, used under the Model Public Preview Licence v1.0."
- โ Be transparent about your evaluation methodology, prompt format, and sampling parameters.
- โ Publish both strengths and limitations.
- โ Do not imply endorsement, partnership, or certification by Rotex Technologies Ltd.
- โ Do not disclose any information that would enable reverse engineering or derivative model creation.
- โ Do not use model outputs to train or improve competing models.
8.1 MMLU-Pro benchmark
MMLU-Pro is a public, reasoning-focused extension of MMLU spanning 14 academic and technical domains. The figures below combine self-reported RotexMech-8B results with published third-party figures from qwen.ai and TIGER-AI-Lab, using the "Overall", "Engineering", and "Math" subsets.
Figure 1 โ MMLU-Pro benchmark: Math, Engineering, and Overall subset scores.
| Model | Overall | Engineering | Math | Data source |
|---|---|---|---|---|
| Qwen3-8B-Instruct | 0.683 | 0.567 | 0.676 | qwen.ai |
| Qwen2.5-14B | 0.637 | 0.495 | 0.679 | TIGER-AI-Lab |
| Qwen3-8B (base) | 0.594 | 0.484 | 0.602 | qwen.ai |
| RotexMech-8B | 0.595 | 0.500 | 0.745 | Self-reported |
| Qwen2.5-7B | 0.552 | 0.416 | 0.523 | qwen.ai |
| Llama-3.1-8B-Instruct | 0.443 | 0.297 | 0.438 | TIGER-AI-Lab |
8.2 In-house mechanical engineering benchmark
We also assess models against an in-house mechanical engineering benchmarking tool: 100 mechanical engineering questions spanning eight topic areas (structural/fatigue/fracture, materials/corrosion/joining, thermal/fluids/sealing, dynamics/vibration/tribology, manufacturing/DFMA/quality, reliability/safety/lifecycle, rotating equipment diagnostics, and pressure systems integrity), scored by a judge model out of 100.
Figure 2 โ Overall mean judged score (100 in-house mechanical engineering questions). Source: in-house mechanical engineering benchmarking tool.
| Model | Mean score / 100 | Non-zero coverage | Rank |
|---|---|---|---|
| Qwen2.5-14B | 58.2 | 88% | 1 |
| Llama-3.1-8B | 47.0 | 73% | 2 |
| RotexMech-8B | 43.1 | 56% | 3 |
| Qwen2.5-7B | 40.2 | 59% | 4 |
| Qwen3-8B-Instruct | 31.5 | 40% | 5 |
| Qwen3-8B (base) | 28.3 | 35% | 6 |
Figure 3 โ Capability profile by mechanical engineering topic. Source: in-house mechanical engineering benchmarking tool. Read as a capability shape comparison rather than a standalone ranking.
We welcome feedback on evaluation methodology, benchmark design, and domain coverage at: ๐ง info@rotextechnologies.com
9. Known Limitations
As a public preview Q4_K_M quantised model, RotexMech-8B has the following known limitations:
- Quantisation degradation โ Q4_K_M quantisation intentionally reduces fidelity compared to our fullโprecision commercial models. Expect some degradation on nuanced technical reasoning and numerical precision.
- No production safety guarantees โ This model has not undergone production safety alignment, redโteaming, or adversarial robustness testing at commercial grade.
- Benchmark coverage โ On the in-house mechanical engineering benchmark, RotexMech-8B returned a lower non-zero response coverage (56%) than several comparison models, and trails Qwen2.5-14B and Llama-3.1-8B on rotating equipment diagnostics. Results should be read topic-by-topic rather than from the overall score alone.
- Domain coverage โ The model's knowledge reflects its training distribution. It may perform less well on topics outside mechanical and industrial engineering domains.
- Hallucination โ Like all large language models, RotexMech-8B may generate plausible but factually incorrect output. Never rely on model output for safetyโcritical decisions without independent verification.
- No regulatory approval โ This model has not been assessed or approved by any regulatory body for use in safetyโcritical or regulated applications.
- Language โ Optimised for English. Performance in other languages is untested and may be unreliable.
This model is an evaluation artefact. It is not a productionโready, safetyโassured product.
10. Safety & Responsible Disclosure
We take safety concerns seriously. If you discover:
- Generation of hazardous, dangerous, or illegal content
- Instructions that could cause physical harm
- Bypass of safety guardrails
- Biased, discriminatory, or harmful outputs
- Vulnerabilities that could be exploited
- Copyright or IP concerns in model outputs
Please report responsibly to: ๐ง info@rotextechnologies.com
We commit to acknowledging reports within 5 working days and providing a substantive response within 20 working days. We request a 90โday embargo before public disclosure of vulnerabilities to allow for assessment and mitigation.
Do not use this model to generate, distribute, or facilitate content that would violate applicable laws in the United Kingdom, European Union, United States, People's Republic of China, or any other relevant jurisdiction.
11. Citation
If you reference RotexMech-8B in academic or technical publications:
@software{rotex_rotexmech8b_2026,
author = {{Rotex Technologies Ltd}},
title = {RotexMech-8B โ Public Preview GGUF Model},
year = {2026},
publisher = {Rotex Technologies Ltd},
url = {https://huggingface.co/rotextechnologies/rotexmech-8B},
note = {Model Public Preview Licence v1.0. Q4_K_M quantisation. Fine-tuned from Qwen3-8B.}
}
Suggested in-text attribution: "RotexMech-8B by Rotex Technologies Ltd, used under the Model Public Preview Licence v1.0."
12. Commercial Access
This public preview is not licensed for commercial use. Our commercial offering includes:
- Higher-precision (BF16) and higher-quality quantised (Q8_0) model variants
- Domainโspecific fineโtuned variants
- Commercial deployment licence
- SLAs, support, and integration assistance
- Access to proprietary evaluation tooling
- Custom development and bespoke model adaptation
For commercial enquiries, partnership discussions, or evaluation under NDA: ๐ง sales@rotextechnologies.com ๐ www.rotextechnologies.com
See COMMERCIAL_USE.md for more detail.
13. Repository Documents
| Document | Description |
|---|---|
| license.md | Full legal terms โ Model Public Preview Licence v1.0 |
| ACCEPTABLE_USE_POLICY.md | Acceptable use policy and prohibited use categories |
| COMMERCIAL_USE.md | Commercial licensing pathway and FAQ |
14. Acknowledgements
RotexMech-8B is fine-tuned from the Qwen3-8B base model. Benchmark comparisons in Section 8.1 draw on published and self-reported data from qwen.ai and TIGER-AI-Lab (MMLU-Pro leaderboard). We gratefully acknowledge these organisations for their open publication of model architectures and benchmark results, which support transparent, independent evaluation across the field. Inclusion of third-party figures does not imply endorsement of RotexMech-8B by qwen.ai or TIGER-AI-Lab.
ยฉ Rotex Technologies Ltd. All rights reserved. Rotex Technologies Ltd is a company registered in England and Wales.
- Downloads last month
- -
4-bit


