language: - en - pt license: agpl-3.0 tags: - mark-ai - systems-engineering - linux-kernel - rust - cpp - c - reverse-engineering - robotics - electrical-engineering - mla - moe - mtp - gguf - ollama pipeline_tag: text-generation inference: false

Mark-Engineer-V2-8B: High-Density Cognitive Systems Architecture

Mark-Engineer-V2-8B is a specialized 8.01-billion-parameter Mixture-of-Experts (MoE) foundation model designed for low-level systems engineering, operating system kernels, distributed consensus, firmware, compiler toolchains, cybersecurity (AppSec P0/Red Team), and physical computing.

Engineered with Multi-Head Latent Attention (MLA), 24 routed experts plus 1 shared expert (Top-2 routing, ~1.63B active compute per token), and Multi-Token Prediction (MTP, Depth 1).

IMPORTANT DISCLOSURE: TRAINED BUT UNTESTED (TREINADO MAS NÃO TESTADO)
Mark-Engineer-V2-8B has completed its pre-training curriculum (convergence loss: 1.9047). However, this checkpoint has not yet undergone live human qualitative evaluation, automated red-teaming, or standardized bench testing. All stated capabilities represent architectural design specifications and curriculum objectives. Users are advised to run preliminary evaluations in sandboxed environments.

Architectural Specifications

Parameter	Value	Description
Total Physical Parameters	8,005,905,408 (~8.01B)	Total neural weights across dense and expert layers
Active Parameters per Token	~~1,626,500,000 (~~1.63B)	Compute footprint per token during forward pass
Layers	32	2 Initial Dense Blocks + 30 DeepSeekMoE Blocks
Hidden Dimension ($d_{model}$)	2,560	Base model embedding dimension
Attention Mechanism	MLA (Multi-Head Latent Attention)	Decoupled RoPE with low-rank KV & Q compression
Query Compression ($q_{lora}$)	1,024	Compresses query states into latent representation
Key-Value Compression ($kv_{lora}$)	512	Compresses KV states into 512-dim latent vector
RoPE Decoupled Dimension	32	Positional embedding applied only to decoupled key/query
MoE Routing Topology	1 Shared + 24 Routed Experts	Top-2 routed experts + 1 universal shared expert
Expert Dimension ($d_{expert}$)	1,254	Intermediate dimension per expert block
Multi-Token Prediction (MTP)	Depth 1 ($t+1, t+2$)	Parallel dual-head prediction for fast speculative decoding
Vocabulary Size	49,152 tokens	Byte-Pair Encoding (BPE) tuned for code, math, and technical prose
Context Window	8,192 tokens	Native rotary positional context span

Expert Domain Specialization Matrix

The 24 routed experts are partitioned across specialized computational domains:

Expert ID	Domain Descriptor	Focus Area
E00	`linux_kernel_core_mm`	Linux kernel memory management, page tables, slab/slub allocators, eBPF
E01	`embedded_systems_firmware_rtos`	Bare-metal C, FreeRTOS, Zephyr, ARM Cortex-M/R, memory-mapped I/O
E02	`electrical_engineering_circuits_pcb`	Circuit theory, power converters, analog/digital filter design, KiCad schematics
E03	`mechanical_engineering_physics_cad`	Statics, dynamics, stress analysis, thermodynamics, mechanical tolerances
E04	`robotics_kinematics_dynamics_ros2`	Forward/inverse kinematics, Denavit-Hartenberg, PID controllers, ROS 2, odometry
E05	`hardware_fpga_verilog_riscv`	Digital logic, RTL design, Verilog/VHDL, RISC-V ISA extensions
E06	`c_cpp_systems_memory_models`	C99/C11, C++20/23, RAII, concurrency atomics, lock-free rings, memory orderings
E07	`rust_systems_concurrency`	Unsafe Rust, pointer provenance, Tokio runtime, crossbeam channels, borrow checker
E08	`go_distributed_backends`	Goroutines, Go scheduler internals, sync pools, low-latency microservices
E09	`asm_binary_reverse_engineering`	x86_64, ARM64, RISC-V disassembly, ELF/PE parsing, Ghidra decompilation
E10	`cybersec_red_team_pwn`	Heap exploitation, tcache poisoning, ROP/JOP chains, ASLR/DEP mitigations
E11	`cybersec_appsec_crypto`	Constant-time cryptography, AES-GCM, TLS 1.3 protocol verification, threat models
E12	`software_architecture_full_system`	C4 blueprints, Domain-Driven Design (DDD), clean architecture, event sourcing
E13	`distributed_systems_consensus`	Raft, Multi-Paxos, Kafka partitions, 2PC/Saga distributed transactions
E14	`database_internals_storage`	LSM-trees, B+Trees, Write-Ahead Logs (WAL), MVCC engines, query planners
E15	`networking_protocols_kernel`	TCP/IP stack tuning, QUIC, HTTP/3, AF_XDP, zero-copy packet processing
E16	`compilers_interpreters_jit`	LLVM IR passes, AST transformations, register allocation, SSA form
E17	`enterprise_polyglot_tier1`	Modern Python, Java/Kotlin, C#/.NET, TypeScript, PostgreSQL/MySQL
E18	`cloud_native_devops_infra`	Kubernetes operators, Docker/OCI runtimes, Terraform, NixOS, Prometheus
E19	`vague_prompt_intent_disambiguation`	Implied context deduction, sensible engineering defaults, interactive clarification
E20	`epistemic_calibration_research`	Uncertainty assessment, zero-hallucination verification, fact-checking citations
E21	`top20_polyglot_linguistics`	Technical vocabulary and syntax across top 20 global languages (EN, PT, ES, etc.)
E22	`high_eq_senior_tech_lead_persona`	Constructive, articulate, and patient technical communication persona
E23	`formal_verification_debugging`	TLA+ specifications, Model Checking, Valgrind, AddressSanitizer, Linux perf profiling

Training Convergence

The model was trained on the consolidated Mark Ultra-Master Curriculum with multi-objective optimization:

$\mathcal{L}_{\text{Total}} = \mathcal{L}_{\text{NTP}} + 0.3 \cdot \mathcal{L}_{\text{MTP}} + 10^{-4} \cdot \mathcal{L}_{\text{Router\_Z}}$

Initial Loss: 11.4500
Final Pre-training Loss: 1.9047
- Next-Token Prediction (NTP) Loss: 1.2952
- Knowledge Distillation (KD) Loss: 0.4190
- Multi-Token Prediction (MTP) Loss: 0.1905
- Router Z-Loss: Stable ($< 1.2 \times 10^{-4}$)
Learning Rate Schedule: Cosine decay with linear warmup ($1.2 \times 10^{-4} \rightarrow 1.0 \times 10^{-5}$)

Quickstart: Plug and Play Deployment

1. Ollama (Recommended)

Download the quantized model file mark_engineer_v2_8b_iq4_nl.gguf from the Releases section.
Create a Modelfile:

FROM ./mark_engineer_v2_8b_iq4_nl.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
<think>
"""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.2
PARAMETER top_p 0.95

SYSTEM """Voce e o Mark Engineer V2 8B, um modelo de sistemas e engenharia de software de altissima capacidade."""

ollama create mark-engineer-v2-8b -f Modelfile
ollama run mark-engineer-v2-8b

2. LM Studio / llama.cpp

Load mark_engineer_v2_8b_iq4_nl.gguf directly into LM Studio or llama.cpp:

./llama-cli -m mark_engineer_v2_8b_iq4_nl.gguf -p "Implement a lock-free ring buffer in Rust using atomic pointers." -n 1024 --temp 0.2

3. PyTorch / Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "miceworks/Mark-Engineer-V2-8B",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("miceworks/Mark-Engineer-V2-8B")

inputs = tokenizer("<|im_start|>user\nExplain the slab allocator in the Linux kernel.<|im_end|>\n<|im_start|>assistant\n", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

GNU Affero General Public License v3.0 (AGPL-3.0). Free for open-source research and community deployment.

Downloads last month: 60

GGUF

Model size

3B params

Architecture

llama

Hardware compatibility

4-bit