docs: publish official Mark AI model card and benchmarks

2691045 verified 1 day ago

8.53 kB

---
language:
- en
- pt
license: agpl-3.0
tags:
- mark-ai
- systems-engineering
- linux-kernel
- rust
- cpp
- c
- reverse-engineering
- robotics
- electrical-engineering
- mla
- moe
- mtp
- gguf
- ollama
pipeline_tag: text-generation
---
---
language:
- en
- pt
license: agpl-3.0
tags:
- mark-ai
- systems-engineering
- linux-kernel
- rust
- cpp
- c
- reverse-engineering
- robotics
- electrical-engineering
- mla
- moe
- mtp
- gguf
- ollama
pipeline_tag: text-generation
inference: false
---
# Mark-Engineer-V2-8B: High-Density Cognitive Systems Architecture
Mark-Engineer-V2-8B is a specialized 8.01-billion-parameter Mixture-of-Experts (MoE) foundation model designed for low-level systems engineering, operating system kernels, distributed consensus, firmware, compiler toolchains, cybersecurity (AppSec P0/Red Team), and physical computing.
Engineered with Multi-Head Latent Attention (MLA), 24 routed experts plus 1 shared expert (Top-2 routing, ~1.63B active compute per token), and Multi-Token Prediction (MTP, Depth 1).
> **IMPORTANT DISCLOSURE: TRAINED BUT UNTESTED (TREINADO MAS NÃO TESTADO)**  
> Mark-Engineer-V2-8B has completed its pre-training curriculum (convergence loss: 1.9047). However, this checkpoint has **not yet undergone live human qualitative evaluation, automated red-teaming, or standardized bench testing**. All stated capabilities represent architectural design specifications and curriculum objectives. Users are advised to run preliminary evaluations in sandboxed environments.
---
## Architectural Specifications
| Parameter | Value | Description |
| :--- | :--- | :--- |
| **Total Physical Parameters** | **8,005,905,408 (~8.01B)** | Total neural weights across dense and expert layers |
| **Active Parameters per Token** | **~1,626,500,000 (~1.63B)** | Compute footprint per token during forward pass |
| **Layers** | **32** | 2 Initial Dense Blocks + 30 DeepSeekMoE Blocks |
| **Hidden Dimension ($d_{model}$)** | **2,560** | Base model embedding dimension |
| **Attention Mechanism** | **MLA (Multi-Head Latent Attention)** | Decoupled RoPE with low-rank KV & Q compression |
| **Query Compression ($q_{lora}$)** | **1,024** | Compresses query states into latent representation |
| **Key-Value Compression ($kv_{lora}$)** | **512** | Compresses KV states into 512-dim latent vector |
| **RoPE Decoupled Dimension** | **32** | Positional embedding applied only to decoupled key/query |
| **MoE Routing Topology** | **1 Shared + 24 Routed Experts** | Top-2 routed experts + 1 universal shared expert |
| **Expert Dimension ($d_{expert}$)** | **1,254** | Intermediate dimension per expert block |
| **Multi-Token Prediction (MTP)** | **Depth 1 ($t+1, t+2$)** | Parallel dual-head prediction for fast speculative decoding |
| **Vocabulary Size** | **49,152 tokens** | Byte-Pair Encoding (BPE) tuned for code, math, and technical prose |
| **Context Window** | **8,192 tokens** | Native rotary positional context span |
---
## Expert Domain Specialization Matrix
The 24 routed experts are partitioned across specialized computational domains:
| Expert ID | Domain Descriptor | Focus Area |
| :---: | :--- | :--- |
| **E00** | `linux_kernel_core_mm` | Linux kernel memory management, page tables, slab/slub allocators, eBPF |
| **E01** | `embedded_systems_firmware_rtos` | Bare-metal C, FreeRTOS, Zephyr, ARM Cortex-M/R, memory-mapped I/O |
| **E02** | `electrical_engineering_circuits_pcb` | Circuit theory, power converters, analog/digital filter design, KiCad schematics |
| **E03** | `mechanical_engineering_physics_cad` | Statics, dynamics, stress analysis, thermodynamics, mechanical tolerances |
| **E04** | `robotics_kinematics_dynamics_ros2` | Forward/inverse kinematics, Denavit-Hartenberg, PID controllers, ROS 2, odometry |
| **E05** | `hardware_fpga_verilog_riscv` | Digital logic, RTL design, Verilog/VHDL, RISC-V ISA extensions |
| **E06** | `c_cpp_systems_memory_models` | C99/C11, C++20/23, RAII, concurrency atomics, lock-free rings, memory orderings |
| **E07** | `rust_systems_concurrency` | Unsafe Rust, pointer provenance, Tokio runtime, crossbeam channels, borrow checker |
| **E08** | `go_distributed_backends` | Goroutines, Go scheduler internals, sync pools, low-latency microservices |
| **E09** | `asm_binary_reverse_engineering` | x86_64, ARM64, RISC-V disassembly, ELF/PE parsing, Ghidra decompilation |
| **E10** | `cybersec_red_team_pwn` | Heap exploitation, tcache poisoning, ROP/JOP chains, ASLR/DEP mitigations |
| **E11** | `cybersec_appsec_crypto` | Constant-time cryptography, AES-GCM, TLS 1.3 protocol verification, threat models |
| **E12** | `software_architecture_full_system` | C4 blueprints, Domain-Driven Design (DDD), clean architecture, event sourcing |
| **E13** | `distributed_systems_consensus` | Raft, Multi-Paxos, Kafka partitions, 2PC/Saga distributed transactions |
| **E14** | `database_internals_storage` | LSM-trees, B+Trees, Write-Ahead Logs (WAL), MVCC engines, query planners |
| **E15** | `networking_protocols_kernel` | TCP/IP stack tuning, QUIC, HTTP/3, AF_XDP, zero-copy packet processing |
| **E16** | `compilers_interpreters_jit` | LLVM IR passes, AST transformations, register allocation, SSA form |
| **E17** | `enterprise_polyglot_tier1` | Modern Python, Java/Kotlin, C#/.NET, TypeScript, PostgreSQL/MySQL |
| **E18** | `cloud_native_devops_infra` | Kubernetes operators, Docker/OCI runtimes, Terraform, NixOS, Prometheus |
| **E19** | `vague_prompt_intent_disambiguation` | Implied context deduction, sensible engineering defaults, interactive clarification |
| **E20** | `epistemic_calibration_research` | Uncertainty assessment, zero-hallucination verification, fact-checking citations |
| **E21** | `top20_polyglot_linguistics` | Technical vocabulary and syntax across top 20 global languages (EN, PT, ES, etc.) |
| **E22** | `high_eq_senior_tech_lead_persona` | Constructive, articulate, and patient technical communication persona |
| **E23** | `formal_verification_debugging` | TLA+ specifications, Model Checking, Valgrind, AddressSanitizer, Linux perf profiling |

---

## Training Convergence

The model was trained on the consolidated Mark Ultra-Master Curriculum with multi-objective optimization:

$$\mathcal{L}_{\text{Total}} = \mathcal{L}_{\text{NTP}} + 0.3 \cdot \mathcal{L}_{\text{MTP}} + 10^{-4} \cdot \mathcal{L}_{\text{Router\_Z}}$$
- **Initial Loss:** 11.4500
- **Final Pre-training Loss:** **1.9047**
  - Next-Token Prediction (NTP) Loss: **1.2952**
  - Knowledge Distillation (KD) Loss: **0.4190**
  - Multi-Token Prediction (MTP) Loss: **0.1905**
  - Router Z-Loss: Stable ($< 1.2 \times 10^{-4}$)
- **Learning Rate Schedule:** Cosine decay with linear warmup ($1.2 \times 10^{-4} \rightarrow 1.0 \times 10^{-5}$)
---
## Quickstart: Plug and Play Deployment
### 1. Ollama (Recommended)
1. Download the quantized model file `mark_engineer_v2_8b_iq4_nl.gguf` from the Releases section.
2. Create a `Modelfile`:

```dockerfile
FROM ./mark_engineer_v2_8b_iq4_nl.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
<think>
"""
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.2
PARAMETER top_p 0.95
SYSTEM """Voce e o Mark Engineer V2 8B, um modelo de sistemas e engenharia de software de altissima capacidade."""
```

3. Register and run the model:

```bash
ollama create mark-engineer-v2-8b -f Modelfile
ollama run mark-engineer-v2-8b
```

---

### 2. LM Studio / llama.cpp

Load `mark_engineer_v2_8b_iq4_nl.gguf` directly into LM Studio or llama.cpp:

```bash
./llama-cli -m mark_engineer_v2_8b_iq4_nl.gguf -p "Implement a lock-free ring buffer in Rust using atomic pointers." -n 1024 --temp 0.2
```

---

### 3. PyTorch / Transformers

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
    "miceworks/Mark-Engineer-V2-8B",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("miceworks/Mark-Engineer-V2-8B")
inputs = tokenizer("<|im_start|>user\nExplain the slab allocator in the Linux kernel.<|im_end|>\n<|im_start|>assistant\n", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

## License

GNU Affero General Public License v3.0 (AGPL-3.0). Free for open-source research and community deployment.

	---
	language:
	- en
	- pt
	license: agpl-3.0
	tags:
	- mark-ai
	- systems-engineering
	- linux-kernel
	- rust
	- cpp
	- c
	- reverse-engineering
	- robotics
	- electrical-engineering
	- mla
	- moe
	- mtp
	- gguf
	- ollama
	pipeline_tag: text-generation
	---

	---
	language:
	- en
	- pt
	license: agpl-3.0
	tags:
	- mark-ai
	- systems-engineering
	- linux-kernel
	- rust
	- cpp
	- c
	- reverse-engineering
	- robotics
	- electrical-engineering
	- mla
	- moe
	- mtp
	- gguf
	- ollama
	pipeline_tag: text-generation
	inference: false
	---

	# Mark-Engineer-V2-8B: High-Density Cognitive Systems Architecture

	Mark-Engineer-V2-8B is a specialized 8.01-billion-parameter Mixture-of-Experts (MoE) foundation model designed for low-level systems engineering, operating system kernels, distributed consensus, firmware, compiler toolchains, cybersecurity (AppSec P0/Red Team), and physical computing.

	Engineered with Multi-Head Latent Attention (MLA), 24 routed experts plus 1 shared expert (Top-2 routing, ~1.63B active compute per token), and Multi-Token Prediction (MTP, Depth 1).

	> IMPORTANT DISCLOSURE: TRAINED BUT UNTESTED (TREINADO MAS NÃO TESTADO)
	> Mark-Engineer-V2-8B has completed its pre-training curriculum (convergence loss: 1.9047). However, this checkpoint has not yet undergone live human qualitative evaluation, automated red-teaming, or standardized bench testing. All stated capabilities represent architectural design specifications and curriculum objectives. Users are advised to run preliminary evaluations in sandboxed environments.

	---

	## Architectural Specifications

	\| Parameter \| Value \| Description \|
	\| :--- \| :--- \| :--- \|
	\| Total Physical Parameters \| 8,005,905,408 (~8.01B) \| Total neural weights across dense and expert layers \|
	\| Active Parameters per Token \| ~1,626,500,000 (~1.63B) \| Compute footprint per token during forward pass \|
	\| Layers \| 32 \| 2 Initial Dense Blocks + 30 DeepSeekMoE Blocks \|
	\| Hidden Dimension ($d_{model}$) \| 2,560 \| Base model embedding dimension \|
	\| Attention Mechanism \| MLA (Multi-Head Latent Attention) \| Decoupled RoPE with low-rank KV & Q compression \|
	\| Query Compression ($q_{lora}$) \| 1,024 \| Compresses query states into latent representation \|
	\| Key-Value Compression ($kv_{lora}$) \| 512 \| Compresses KV states into 512-dim latent vector \|
	\| RoPE Decoupled Dimension \| 32 \| Positional embedding applied only to decoupled key/query \|
	\| MoE Routing Topology \| 1 Shared + 24 Routed Experts \| Top-2 routed experts + 1 universal shared expert \|
	\| Expert Dimension ($d_{expert}$) \| 1,254 \| Intermediate dimension per expert block \|
	\| Multi-Token Prediction (MTP) \| Depth 1 ($t+1, t+2$) \| Parallel dual-head prediction for fast speculative decoding \|
	\| Vocabulary Size \| 49,152 tokens \| Byte-Pair Encoding (BPE) tuned for code, math, and technical prose \|
	\| Context Window \| 8,192 tokens \| Native rotary positional context span \|

	---

	## Expert Domain Specialization Matrix

	The 24 routed experts are partitioned across specialized computational domains:

	\| Expert ID \| Domain Descriptor \| Focus Area \|
	\| :---: \| :--- \| :--- \|
	\| E00 \| `linux_kernel_core_mm` \| Linux kernel memory management, page tables, slab/slub allocators, eBPF \|
	\| E01 \| `embedded_systems_firmware_rtos` \| Bare-metal C, FreeRTOS, Zephyr, ARM Cortex-M/R, memory-mapped I/O \|
	\| E02 \| `electrical_engineering_circuits_pcb` \| Circuit theory, power converters, analog/digital filter design, KiCad schematics \|
	\| E03 \| `mechanical_engineering_physics_cad` \| Statics, dynamics, stress analysis, thermodynamics, mechanical tolerances \|
	\| E04 \| `robotics_kinematics_dynamics_ros2` \| Forward/inverse kinematics, Denavit-Hartenberg, PID controllers, ROS 2, odometry \|
	\| E05 \| `hardware_fpga_verilog_riscv` \| Digital logic, RTL design, Verilog/VHDL, RISC-V ISA extensions \|
	\| E06 \| `c_cpp_systems_memory_models` \| C99/C11, C++20/23, RAII, concurrency atomics, lock-free rings, memory orderings \|
	\| E07 \| `rust_systems_concurrency` \| Unsafe Rust, pointer provenance, Tokio runtime, crossbeam channels, borrow checker \|
	\| E08 \| `go_distributed_backends` \| Goroutines, Go scheduler internals, sync pools, low-latency microservices \|
	\| E09 \| `asm_binary_reverse_engineering` \| x86_64, ARM64, RISC-V disassembly, ELF/PE parsing, Ghidra decompilation \|
	\| E10 \| `cybersec_red_team_pwn` \| Heap exploitation, tcache poisoning, ROP/JOP chains, ASLR/DEP mitigations \|
	\| E11 \| `cybersec_appsec_crypto` \| Constant-time cryptography, AES-GCM, TLS 1.3 protocol verification, threat models \|
	\| E12 \| `software_architecture_full_system` \| C4 blueprints, Domain-Driven Design (DDD), clean architecture, event sourcing \|
	\| E13 \| `distributed_systems_consensus` \| Raft, Multi-Paxos, Kafka partitions, 2PC/Saga distributed transactions \|
	\| E14 \| `database_internals_storage` \| LSM-trees, B+Trees, Write-Ahead Logs (WAL), MVCC engines, query planners \|
	\| E15 \| `networking_protocols_kernel` \| TCP/IP stack tuning, QUIC, HTTP/3, AF_XDP, zero-copy packet processing \|
	\| E16 \| `compilers_interpreters_jit` \| LLVM IR passes, AST transformations, register allocation, SSA form \|
	\| E17 \| `enterprise_polyglot_tier1` \| Modern Python, Java/Kotlin, C#/.NET, TypeScript, PostgreSQL/MySQL \|
	\| E18 \| `cloud_native_devops_infra` \| Kubernetes operators, Docker/OCI runtimes, Terraform, NixOS, Prometheus \|
	\| E19 \| `vague_prompt_intent_disambiguation` \| Implied context deduction, sensible engineering defaults, interactive clarification \|
	\| E20 \| `epistemic_calibration_research` \| Uncertainty assessment, zero-hallucination verification, fact-checking citations \|
	\| E21 \| `top20_polyglot_linguistics` \| Technical vocabulary and syntax across top 20 global languages (EN, PT, ES, etc.) \|
	\| E22 \| `high_eq_senior_tech_lead_persona` \| Constructive, articulate, and patient technical communication persona \|
	\| E23 \| `formal_verification_debugging` \| TLA+ specifications, Model Checking, Valgrind, AddressSanitizer, Linux perf profiling \|

	---

	## Training Convergence

	The model was trained on the consolidated Mark Ultra-Master Curriculum with multi-objective optimization:

	$$\mathcal{L}_{\text{Total}} = \mathcal{L}_{\text{NTP}} + 0.3 \cdot \mathcal{L}_{\text{MTP}} + 10^{-4} \cdot \mathcal{L}_{\text{Router\_Z}}$$

	- Initial Loss: 11.4500
	- Final Pre-training Loss: 1.9047
	- Next-Token Prediction (NTP) Loss: 1.2952
	- Knowledge Distillation (KD) Loss: 0.4190
	- Multi-Token Prediction (MTP) Loss: 0.1905
	- Router Z-Loss: Stable ($< 1.2 \times 10^{-4}$)
	- Learning Rate Schedule: Cosine decay with linear warmup ($1.2 \times 10^{-4} \rightarrow 1.0 \times 10^{-5}$)

	---

	## Quickstart: Plug and Play Deployment

	### 1. Ollama (Recommended)

	1. Download the quantized model file `mark_engineer_v2_8b_iq4_nl.gguf` from the Releases section.
	2. Create a `Modelfile`:

	```dockerfile
	FROM ./mark_engineer_v2_8b_iq4_nl.gguf

	TEMPLATE """<\|im_start\|>system
	{{ .System }}<\|im_end\|>
	<\|im_start\|>user
	{{ .Prompt }}<\|im_end\|>
	<\|im_start\|>assistant
	<think>
	"""

	PARAMETER stop "<\|im_end\|>"
	PARAMETER stop "<\|endoftext\|>"
	PARAMETER temperature 0.2
	PARAMETER top_p 0.95

	SYSTEM """Voce e o Mark Engineer V2 8B, um modelo de sistemas e engenharia de software de altissima capacidade."""
	```

	3. Register and run the model:

	```bash
	ollama create mark-engineer-v2-8b -f Modelfile
	ollama run mark-engineer-v2-8b
	```

	---

	### 2. LM Studio / llama.cpp

	Load `mark_engineer_v2_8b_iq4_nl.gguf` directly into LM Studio or llama.cpp:

	```bash
	./llama-cli -m mark_engineer_v2_8b_iq4_nl.gguf -p "Implement a lock-free ring buffer in Rust using atomic pointers." -n 1024 --temp 0.2
	```

	---

	### 3. PyTorch / Transformers

	```python
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model = AutoModelForCausalLM.from_pretrained(
	"miceworks/Mark-Engineer-V2-8B",
	trust_remote_code=True,
	torch_dtype=torch.bfloat16,
	device_map="auto"
	)
	tokenizer = AutoTokenizer.from_pretrained("miceworks/Mark-Engineer-V2-8B")

	inputs = tokenizer("<\|im_start\|>user\nExplain the slab allocator in the Linux kernel.<\|im_end\|>\n<\|im_start\|>assistant\n", return_tensors="pt").to("cuda")
	outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2)
	print(tokenizer.decode(outputs[0], skip_special_tokens=True))
	```

	---

	## License

	GNU Affero General Public License v3.0 (AGPL-3.0). Free for open-source research and community deployment.