Text Generation
GGUF
English
Portuguese
mark
mark-ai
edge-kernel
synthetic-microbenchmarks
experimental
non-conversational
conversational
Instructions to use miceworks/Mark-Engineer-302M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use miceworks/Mark-Engineer-302M with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf miceworks/Mark-Engineer-302M:IQ4_NL # Run inference directly in the terminal: llama cli -hf miceworks/Mark-Engineer-302M:IQ4_NL
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf miceworks/Mark-Engineer-302M:IQ4_NL # Run inference directly in the terminal: llama cli -hf miceworks/Mark-Engineer-302M:IQ4_NL
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf miceworks/Mark-Engineer-302M:IQ4_NL # Run inference directly in the terminal: ./build/bin/llama cli -hf miceworks/Mark-Engineer-302M:IQ4_NL
Use Docker
docker model run hf.co/miceworks/Mark-Engineer-302M:IQ4_NL
- Llama new
- LM Studio
- Jan
- vLLM
How to use miceworks/Mark-Engineer-302M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "miceworks/Mark-Engineer-302M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "miceworks/Mark-Engineer-302M", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/miceworks/Mark-Engineer-302M:IQ4_NL
- Ollama
How to use miceworks/Mark-Engineer-302M with Ollama:
ollama run hf.co/miceworks/Mark-Engineer-302M:IQ4_NL
- Unsloth Desktop
- Docker Model Runner
How to use miceworks/Mark-Engineer-302M with Docker Model Runner:
docker model run hf.co/miceworks/Mark-Engineer-302M:IQ4_NL
- Lemonade
How to use miceworks/Mark-Engineer-302M with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull miceworks/Mark-Engineer-302M:IQ4_NL
Run and chat with the model
lemonade run user.Mark-Engineer-302M-IQ4_NL
List all available models
lemonade list
- Atomic Chat
docs: update license to GNU Affero GPL v3.0 (AGPL-3.0) and add full benchmark suite
Browse files
README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
language:
|
| 3 |
- en
|
| 4 |
- pt
|
| 5 |
-
license:
|
| 6 |
tags:
|
| 7 |
- mark-ai
|
| 8 |
- edge-kernel
|
|
@@ -21,14 +21,12 @@ Parameter Count: 302,019,848 Physical | ~110M Active per Token
|
|
| 21 |
Status: Experimental Edge Checkpoint / Synthetic Micro-Testbed
|
| 22 |
Target Hardware: Edge Devices, Embedded Linux, Low-Power Micro-Nodes (< 500 MB VRAM)
|
| 23 |
|
| 24 |
-
Strict Constraint: Zero emojis across all documentation and logs.
|
| 25 |
-
|
| 26 |
---
|
| 27 |
|
| 28 |
## Technical Disclaimer & Operational Limitations
|
| 29 |
|
| 30 |
### Non-Conversational Architecture
|
| 31 |
-
Mark Engineer 300M does NOT possess conversational fluency or natural dialogue capabilities. It was not trained as an assistant and cannot maintain coherent multi-turn chit-chat or explain concepts in prose.
|
| 32 |
|
| 33 |
### High Hallucination Probability in Free-Form Generation
|
| 34 |
Due to its ultralight parameter budget (302M physical parameters), open-ended generation, reasoning prompts, or conversational queries will produce disjointed output, repetitive loops, or severe factual hallucinations.
|
|
@@ -36,7 +34,7 @@ Due to its ultralight parameter budget (302M physical parameters), open-ended ge
|
|
| 36 |
### Purpose and Valid Use Cases
|
| 37 |
This checkpoint serves exclusively as:
|
| 38 |
1. An experimental micro-testbed to validate the forward tensor graph of Multi-Head Latent Attention (MLA) and DeepSeekMoE routing on severely constrained hardware.
|
| 39 |
-
2. A synthetic syntax validator for micro-kernel snippets and rigid template completion.
|
| 40 |
3. Edge deployment benchmarking on low-power devices with under 500 MB RAM/VRAM.
|
| 41 |
|
| 42 |
If your task requires conversational tech lead dialogue, system architecture reasoning, multi-file code synthesis, or zero-hallucination analysis, use **Mark Engineer 1.5B** or **MarkVL-2.2B**.
|
|
@@ -56,18 +54,23 @@ If your task requires conversational tech lead dialogue, system architecture rea
|
|
| 56 |
| **Quantization Format** | IQ4_NL (4-bit non-linear lookup) / Universal GGUF |
|
| 57 |
| **On-Disk Checkpoint Size** | 162.99 MB (GGUF: 225.21 MB) |
|
| 58 |
| **Runtime VRAM Footprint** | ~350 MB on NVIDIA GeForce RTX 2060 Super |
|
|
|
|
| 59 |
| **Context Length** | 4,096 Tokens |
|
| 60 |
|
| 61 |
---
|
| 62 |
|
| 63 |
## 2. Benchmark Results on Synthetic Test Suites
|
| 64 |
|
| 65 |
-
|
| 66 |
|
| 67 |
-
|
| 68 |
-
-
|
| 69 |
-
|
| 70 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
|
| 72 |
---
|
| 73 |
|
|
@@ -75,20 +78,18 @@ Mark Engineer 300M passes narrow, deterministic synthetic unit tests while faili
|
|
| 75 |
|
| 76 |
### 3.1 Ollama
|
| 77 |
```bash
|
| 78 |
-
|
| 79 |
-
ollama create mark-engineer-302m -f Modelfile
|
| 80 |
-
ollama run mark-engineer-302m "/* C Ringbuffer */"
|
| 81 |
```
|
| 82 |
|
| 83 |
-
### 3.2
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
```
|
| 88 |
-
2. Move to `~/.lmstudio/models/Mark-AI/mark-engineer-302m/mark_engineer_302m.gguf`.
|
| 89 |
-
3. Load the model. Allocate ~350 MB VRAM.
|
| 90 |
|
| 91 |
-
### 3.3
|
| 92 |
```bash
|
| 93 |
-
|
|
|
|
|
|
|
|
|
|
| 94 |
```
|
|
|
|
| 2 |
language:
|
| 3 |
- en
|
| 4 |
- pt
|
| 5 |
+
license: agpl-3.0
|
| 6 |
tags:
|
| 7 |
- mark-ai
|
| 8 |
- edge-kernel
|
|
|
|
| 21 |
Status: Experimental Edge Checkpoint / Synthetic Micro-Testbed
|
| 22 |
Target Hardware: Edge Devices, Embedded Linux, Low-Power Micro-Nodes (< 500 MB VRAM)
|
| 23 |
|
|
|
|
|
|
|
| 24 |
---
|
| 25 |
|
| 26 |
## Technical Disclaimer & Operational Limitations
|
| 27 |
|
| 28 |
### Non-Conversational Architecture
|
| 29 |
+
Mark Engineer 300M does NOT possess conversational fluency or natural dialogue capabilities. It was not trained as an assistant and cannot maintain coherent multi-turn chit-chat or explain concepts in prose ("não sabe falar").
|
| 30 |
|
| 31 |
### High Hallucination Probability in Free-Form Generation
|
| 32 |
Due to its ultralight parameter budget (302M physical parameters), open-ended generation, reasoning prompts, or conversational queries will produce disjointed output, repetitive loops, or severe factual hallucinations.
|
|
|
|
| 34 |
### Purpose and Valid Use Cases
|
| 35 |
This checkpoint serves exclusively as:
|
| 36 |
1. An experimental micro-testbed to validate the forward tensor graph of Multi-Head Latent Attention (MLA) and DeepSeekMoE routing on severely constrained hardware.
|
| 37 |
+
2. A synthetic syntax validator for micro-kernel snippets, endian transforms, and rigid template completion.
|
| 38 |
3. Edge deployment benchmarking on low-power devices with under 500 MB RAM/VRAM.
|
| 39 |
|
| 40 |
If your task requires conversational tech lead dialogue, system architecture reasoning, multi-file code synthesis, or zero-hallucination analysis, use **Mark Engineer 1.5B** or **MarkVL-2.2B**.
|
|
|
|
| 54 |
| **Quantization Format** | IQ4_NL (4-bit non-linear lookup) / Universal GGUF |
|
| 55 |
| **On-Disk Checkpoint Size** | 162.99 MB (GGUF: 225.21 MB) |
|
| 56 |
| **Runtime VRAM Footprint** | ~350 MB on NVIDIA GeForce RTX 2060 Super |
|
| 57 |
+
| **Inference Throughput** | 284 tokens/second on RTX 2060S |
|
| 58 |
| **Context Length** | 4,096 Tokens |
|
| 59 |
|
| 60 |
---
|
| 61 |
|
| 62 |
## 2. Benchmark Results on Synthetic Test Suites
|
| 63 |
|
| 64 |
+
Evaluated across rigid synthetic kernels vs conversational probes:
|
| 65 |
|
| 66 |
+
| Test Suite / Target Capability | Result | Pass Rate | Evaluation Behavior |
|
| 67 |
+
| :--- | :---: | :---: | :--- |
|
| 68 |
+
| **Rigid C Packed Structs & Bitmasks** | **PASS** | **96.0%** (48/50) | Accurate deterministic bit alignment. |
|
| 69 |
+
| **Endianness Swap (`ntohl`/`bswap32`)** | **PASS** | **98.0%** (49/50) | Exact register mapping and bit shifts. |
|
| 70 |
+
| **Fixed-Size Ringbuffer Boundary Logic** | **PASS** | **92.0%** (46/50) | Bounded queue index wrapping correctly. |
|
| 71 |
+
| **CRC32 Lookup Table Kernel Synthesis** | **PASS** | **94.0%** (47/50) | Synthesizes unrolled polynomial tables. |
|
| 72 |
+
| **Natural Language Conversational Dialogue** | **FAIL** | **12.0%** (6/50) | Disjointed syntax; loops or halts abruptly. |
|
| 73 |
+
| **Open-Domain Technical Explanations** | **FAIL** | **16.0%** (8/50) | High hallucination rate on non-kernel concepts. |
|
| 74 |
|
| 75 |
---
|
| 76 |
|
|
|
|
| 78 |
|
| 79 |
### 3.1 Ollama
|
| 80 |
```bash
|
| 81 |
+
ollama run mark-engineer-302m:latest "/* C Ringbuffer */"
|
|
|
|
|
|
|
| 82 |
```
|
| 83 |
|
| 84 |
+
### 3.2 Hugging Face / GGUF
|
| 85 |
+
```bash
|
| 86 |
+
huggingface-cli download miceworks/Mark-Engineer-302M mark_engineer_302m_iq4_nl.gguf --local-dir ./weights
|
| 87 |
+
```
|
|
|
|
|
|
|
|
|
|
| 88 |
|
| 89 |
+
### 3.3 llama.cpp CLI
|
| 90 |
```bash
|
| 91 |
+
./llama-cli \
|
| 92 |
+
-m ./weights/mark_engineer_302m_iq4_nl.gguf \
|
| 93 |
+
-p "/* C Packed Struct */\nstruct __attribute__((packed)) ethernet_hdr {" \
|
| 94 |
+
-n 128
|
| 95 |
```
|