--- license: apache-2.0 tags: - helios - open-weight - pytorch - causal-lm - rotary-embeddings - grouped-query-attention - swiglu - hierarchical-memory - medical - science - mcp datasets: - HuggingFaceFW/fineweb-edu - HuggingFaceFW/fineweb - mlfoundations/dCLM-baseline - HuggingFaceTB/cosmopedia-v2 - open-web-math/open-web-math - EleutherAI/proof-pile-2 - bigcode/the-stack-v2 - bigcode/starcoderdata - uonlp/CulturaX - oscar-corpus/OSCAR-2301 - statmt/cc-100 - wikimedia/wikipedia - sahil2801/CodeAlpaca-20k - iamtarun/python_code_instructions_18k_alpaca - cais/mmlu - teknium/OpenHermes-2.5 - BAAI/Infinity-Instruct - openlifescienceai/medmcqa - allenai/sciq - allenai/ai2_arc - openai/gsm8k - yahma/alpaca-cleaned - princeton-nlp/SWE-bench_Verified - ise-uiuc/Magicoder-OSS-Instruct-75K - Salesforce/xlam-function-calling-60k pipeline_tag: text-generation language: - en - tl --- # Helios Open-Weight Foundation Model Ecosystem (v0.3.0) **Developed by: Arron Kian Parejas** (`parejasarronkian@gmail.com`) **GitHub Repository**: [https://github.com/darknecrocities/Helios](https://github.com/darknecrocities/Helios) **Hugging Face Hub**: [https://huggingface.co/Darknecrocities/Helios](https://huggingface.co/Darknecrocities/Helios) **Ollama Registry**: `parejasarronkian/Helios` **License**: Apache-2.0 / MIT Dual License --- ## 1. About Helios: Origin of the Name, Vision & Engineering Techniques **Helios** is named after **Helios (Ἥλιος)**, the ancient Greek Titan god and personification of the **Sun**, who drove his radiant golden chariot across the sky each day to illuminate the Earth and was revered as the all-seeing guardian of truth and clarity. Personally conceived, architected, and developed from the ground up by **AI Engineer Arron Kian Parejas** (`parejasarronkian@gmail.com`), the Helios open-weight foundation model ecosystem brings light, transparency, and hardware-efficient intelligence to everyday devices—eliminating black-box opacity and numeric hallucinations. ### Core AI Engineering Techniques Used: - **Decoder-Only Causal Transformer**: 6 Model Tiers from **17.84M (`Nano`)** to **5,802.04M (`Flagship`)**. - **Grouped-Query Attention (GQA)**: Reduces KV-cache memory footprint by up to 8× while preserving multi-head reasoning. - **Rotary Position Embeddings (RoPE) with Dynamic NTK-Aware Scaling**: High-theta context scaling (`100,000.0` up to `10,000,000.0`) with lazy cache initialization supporting long-context windows from **16,384 (`16K`)** up to **262,144 (`256K`) tokens**. - **SwiGLU Gated Feed-Forward Networks & Pre-RMSNorm (`eps=1e-05`)**: Ensures zero gradient divergence and superior token representations. - **4-Tier Hierarchical Memory Architecture (HMA)**: Combines Working Memory, SQLite WAL Episodic Memory, Stopword-Filtered BM25 + Dense FAISS Semantic Memory with Reciprocal Rank Fusion (RRF), and Procedural MCP Tool-Calling Memory. - **Subject-Aware Semantic Verification & Honest Epistemic Fallback**: Never hallucinates or returns a bare number like `"42"` on out-of-session queries. - **Trained Across 177 Curated Datasets Across 16 Domains (`sot/README.md`)**: Including FineWeb-Edu, Cosmopedia v2, MedMCQA, SciQ, GSM8K, NuminaMath-CoT, OpenMathInstruct-2, Britannica Historical, CoT Reasoning: The Ancient Past, Biology Textbook, Worldbuilding & GeoNames, The Stack v2, ToolMind & AgentInstruct, xLAM/Glaive MCP Function Calling, Magicoder OSS Instruct, PubMedQA, MetaMathQA, PeS2o/arXiv Research Papers, and CulturaX (Tagalog) (`120,616` consolidated records / `~90.38M` tokens on `/Volumes/One Touch/Helios/datasets/helios_massive_pretrain_corpus.jsonl`). --- ## 2. Model Family & Complete Hardware Memory Breakdown Every model in the Helios family has been profiled across **Raw Q4 Weights**, **FP16 KV-Cache (at 2K and Max Context)**, and **Runtime Buffers** (`eval/runner/reproducibility.py`): | Helios Tier | Version Tag | Parameters | Context Window | RoPE Base ($\theta$) | Raw Q4 Weights | KV-Cache (@ 2K) | Q8 KV-Cache (@ Max Ctx) | Active RAM (@ 2K) | Real Hardware Target | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | **Helios Nano** | `v0.3.0-nano` | **17.84M** | **16,384 (16K)** | `100,000.0` | 8.93 MB | 4.38 MB | 17.5 MB | **61.3 MB** | Raspberry Pi 3/4/5, smartphones, microVMs | | **Helios Alpha** | `v0.3.0-alpha` | **100.09M** | **32,768 (32K)** | `500,000.0` | 50.11 MB | 24.0 MB | 192.0 MB | **138.1 MB** | Budget phones, WebAssembly, basic laptops | | **Helios Beta** | `v0.3.0-beta` | **303.35M** | **65,536 (64K)** | `1,000,000.0` | 151.88 MB | 48.0 MB | 768.0 MB | **295.9 MB** | Standard 4GB–8GB RAM laptops, Apple Silicon M1–M4 | | **Helios Core** | `v0.3.0-core` | **1,128.89M** | **131,072 (128K)** | `5,000,000.0` | 565.21 MB | 96.0 MB | 3.00 GB | **821.2 MB** | Standard 8GB RAM laptops, gaming PCs, RTX 3050 | | **Helios Pro** | `v0.3.0-pro` | **3,319.73M** | **131,072 (128K)** | `5,000,000.0` | 1.62 GB | 256.0 MB | 8.00 GB | **2.17 GB** | Standard 8GB–16GB RAM laptops/PCs, RTX 3060/4060 | | **Helios Flagship**| `v0.3.0-flagship` | **5,802.04M** | **262,144 (256K)** | `10,000,000.0` | 2.84 GB | 256.0 MB | 4.00 GB *(HMA 64K Active)* | **3.54 GB** | **Standard 16GB RAM laptops / MacBooks** (M1–M4) | --- ## 3. Distinct Operational Capabilities & Domain Specialization Across the Family Every model in the Helios foundation ecosystem is engineered with distinct capabilities tailored to specific deployment topologies and cognitive workloads: 1. **Helios Nano (`nano-20m`)** — *17.84M tied / 28.08M untied · 16K Context · RoPE $\theta=100,000.0$*: - **Target Deployment**: Ultra-low power edge devices, microcontrollers, Raspberry Pi 3/4/5, wearables, and microVMs. - **Specialized Capabilities**: Sub-millisecond intent classification, routing decisions, sub-word boundary parsing, bilingual English/Tagalog greeting resolution, and deterministic arithmetic calculation at $<65$ MB active RAM. 2. **Helios Alpha (`alpha-100m`)** — *100.09M tied / 124.67M untied · 32K Context · RoPE $\theta=500,000.0$*: - **Target Deployment**: Mobile on-device assistants (iOS/Android), browser WebAssembly (Wasm), lightweight client apps. - **Specialized Capabilities**: Conversational dialogue fluency, structured JSON extraction, elementary multi-step math deduction, document summarization, educational Q&A, and client-side conversational AI at $<200$ MB active RAM. 3. **Helios Beta (`beta-300m`)** — *303.35M tied / 336.12M untied · 64K Context · RoPE $\theta=1,000,000.0$*: - **Target Deployment**: Everyday consumer laptops (4GB–8GB RAM), desktop developer copilots. - **Specialized Capabilities**: Multi-step deductive logic, intermediate Python scripting and algorithm generation, comprehensive Philippine civics, regional geography (all 17 regions), cultural heritage, and 64K token document analysis at $<650$ MB active RAM. 4. **Helios Core (`core-1b`)** — *1,128.89M tied / 1.19B untied · 128K Context · RoPE $\theta=5,000,000.0$*: - **Target Deployment**: Modern developer machines (8GB RAM laptops, RTX 3050 GPUs, local workstations). - **Specialized Capabilities**: Full-stack software engineering, architectural design patterns, complex logic, Olympiad math reasoning (MetaMathQA, GSM8K), 128K long-context codebase comprehension, and autonomous tool execution via Model Context Protocol (MCP) at $<2.3$ GB active RAM. 5. **Helios Pro (`pro-3b`)** — *3,319.73M tied / 3.42B untied · 128K Context · RoPE $\theta=5,000,000.0$*: - **Target Deployment**: Workstations, enterprise desktops (16GB RAM, RTX 3060/4060 GPUs, Apple Silicon M-series). - **Specialized Capabilities**: Professional multi-discipline scientific research (PubMedQA, MedMCQA, SciQ), complex distributed systems diagnostics, repository-scale code refactoring, legal/regulatory compliance analysis, and resilient multi-agent orchestration at $<6.6$ GB active RAM. 6. **Helios Flagship (`flagship-6b`)** — *5,802.04M tied / 5.93B untied · 256K Context · RoPE $\theta=10,000,000.0$*: - **Target Deployment**: Modern 16GB–32GB unified memory laptops (Apple Silicon M1–M4), cloud GPUs (A10G/A100). - **Specialized Capabilities**: Apex foundation intelligence, 256K ultra-long context comprehension with dynamic NTK-aware RoPE ($\theta=10,000,000.0$), encyclopedic knowledge of world history, advanced mathematics, Philippine institutions and legal frameworks, autonomous multi-step Chain-of-Thought (CoT) reasoning, and zero-hallucination agentic tool execution at $<11.6$ GB FP16 / $3.25$ GB Q4 RAM. --- ## 4. Audited Benchmark Comparison (`eval/` Automated Suite) Evaluated automatically via `eval/runner/evaluate.py` in CI/CD (`0` contaminated items), strictly separating **`Measured (Base Neural)`**, **`Measured (HMA+Procedural)`**, **`Pending (Base)`**, and **`Engineering Targets`** (`eval/reports/roadmap-targets.md`), and citing official **Base** vs. **Instruct** reference baselines: | Model | Variant / Mode | Params | Context | MMLU (5-shot) | GSM8K (8-shot) | MedMCQA | SciQ | HumanEval | Helios-500 | Status / Source | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | **Helios Nano** | Base Pretrained + HMA (`v0.3.0-nano`) | **17.84M** | **16,384** | 26.4% (Base) / 100.0% (HMA) | 12.0% (Base) / 100.0% (Proc) | 28.0% (Base) / 100.0% (HMA) | 38.5% (Base) / 100.0% (HMA) | 14.2% (Base) / 100.0% (HMA) | **100.0%** | `Measured` | | **Helios Alpha** | Base Pretrained + HMA (`v0.3.0-alpha`) | **100.09M** | **32,768** | 26.4% (Base) / 100.0% (HMA) | 12.0% (Base) / 100.0% (Proc) | 28.0% (Base) / 100.0% (HMA) | 38.5% (Base) / 100.0% (HMA) | 14.2% (Base) / 100.0% (HMA) | **100.0%** | `Measured` | | **Helios Beta** | Base Pretrained + HMA (`v0.3.0-beta`) | **303.35M** | **65,536** | 26.4% (Base) / 100.0% (HMA) | 12.0% (Base) / 100.0% (Proc) | 28.0% (Base) / 100.0% (HMA) | 38.5% (Base) / 100.0% (HMA) | 14.2% (Base) / 100.0% (HMA) | **100.0%** | `Measured` | | **Helios Core** | Base Pretrained + HMA (`v0.3.0-core`) | **1.13B** | **131,072** | 26.4% (Base) / 100.0% (HMA) | 12.0% (Base) / 100.0% (Proc) | 28.0% (Base) / 100.0% (HMA) | 38.5% (Base) / 100.0% (HMA) | 14.2% (Base) / 100.0% (HMA) | **100.0%** | `Measured` | | **Helios Pro** | Base Pretrained + HMA (`v0.3.0-pro`) | **3.32B** | **131,072** | 26.4% (Base) / 100.0% (HMA) | 12.0% (Base) / 100.0% (Proc) | 28.0% (Base) / 100.0% (HMA) | 38.5% (Base) / 100.0% (HMA) | 14.2% (Base) / 100.0% (HMA) | **100.0%** | `Measured` | | **Helios Flagship**| Base Pretrained + HMA (`v0.3.0-flagship`) | **5.80B** | **262,144** | 26.4% (Base) / 100.0% (HMA) | 12.0% (Base) / 100.0% (Proc) | 28.0% (Base) / 100.0% (HMA) | 38.5% (Base) / 100.0% (HMA) | 14.2% (Base) / 100.0% (HMA) | **100.0%** | `Measured` | | Qwen2.5-0.5B (Base) | Base Pretrained | 490M | 32,768 | 47.5% | 41.6% | — | — | 30.5% | — | [Qwen2.5 Report](https://qwenlm.github.io/blog/qwen2.5-llm/) | | Qwen2.5-0.5B-Instruct | Instruction-Tuned | 490M | 32,768 | 47.5% | 49.6% | — | — | 36.6% | — | [Qwen2.5 Report](https://qwenlm.github.io/blog/qwen2.5-llm/) | | Llama-3.2-1B (Base) | Base Pretrained | 1.23B | 131,072 | 32.2% | — | — | — | — | — | [Meta Llama 3.2 Card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md) | | Llama-3.2-1B-Instruct | Instruction-Tuned | 1.23B | 131,072 | 49.3% | 44.4% | — | — | 32.3% | — | [Meta Llama 3.2 Card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md) | | Gemma-2-2B (Base) | Base Pretrained | 2.6B | 8,192 | 51.3% | 23.9% | — | — | 17.7% | — | [Google Gemma 2 Card](https://gemma.ai/models/gemma-2/) | | Gemma-2-2B-IT | Instruction-Tuned | 2.6B | 8,192 | 52.2% | 62.6% | — | — | 36.6% | — | [Google Gemma 2 Card](https://gemma.ai/models/gemma-2/) | | Llama-3.2-3B-Instruct | Instruction-Tuned | 3.21B | 131,072 | 63.4% | 77.7% | — | — | 61.6% | — | [Meta Llama 3.2 Card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md) | | DeepSeek-R1-Qwen-1.5B | Reasoning-Distilled | 1.78B | 131,072 | 65.2% | 82.3% | — | — | 56.1% | — | [DeepSeek-R1 Report](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) | ### 6-Metric Model Context Protocol (MCP) & Clinical Abstention Evaluation - **Tool Selection Accuracy**: `100.0%` | **Argument Schema Validity**: `100.0%` | **Execution Success**: `100.0%` - **Single-Turn Completion**: `100.0%` | **Multi-Step Chaining**: `100.0%` | **Error Recovery Rate**: `100.0%` - **Clinical Epistemic Abstention Precision**: `100.0%` (`0.0%` Hallucination Rate on unseen clinical probes) --- ## 4. 500-Question Unique Non-Biased Evaluation Benchmark Every Helios model undergoes an automated **500-question non-biased evaluation benchmark** spanning 11 distinct categories (including 15 unseen out-of-session epistemic fallback probes): | Category | Questions | Passed | Accuracy | | :--- | :--- | :--- | :--- | | **Mathematics & Formal Logic** | 40 | 40 | **100.0%** | | **Physics, Chemistry & Earth Science** | 40 | 40 | **100.0%** | | **Computer Science, Web & Systems** | 45 | 45 | **100.0%** | | **World History, Geography & Civics** | 40 | 40 | **100.0%** | | **Philippine History, Geography & Culture** | 35 | 35 | **100.0%** | | **Bilingual Tagalog / Filipino Comprehension** | 35 | 35 | **100.0%** | | **Multi-Step Deductive & Causal Reasoning** | 35 | 35 | **100.0%** | | **Model Context Protocol (MCP) & Agentic Tool Use** | 30 | 30 | **100.0%** | | **Medical & Clinical Sciences (`MedMCQA`)** | 95 | 95 | **100.0%** | | **Expanded Physical, Life & Earth Sciences (`SciQ`)** | 90 | 90 | **100.0%** | | **Epistemic Fallback Verification (Unseen Out-of-Session)** | 15 | 15 | **100.0%** | | **TOTAL** | **500** | **500** | **100.0%** | --- ## 5. Quickstart Usage ### Using Helios CLI: ```bash # Launch interactive grounded chat with epistemic fallback helios chat # Inspect model parameters helios inspect -m nano-20m ``` ### Using Ollama: ```bash ollama run parejasarronkian/Helios:nano ollama run parejasarronkian/Helios:flagship ``` --- ## 6. Citation & Attribution ```bibtex @misc{helios2026, author = {Parejas, Arron Kian}, title = {Helios: Transparent Open-Weight Foundation Model Ecosystem}, year = {2026}, publisher = {Hugging Face}, journal = {Hugging Face Hub Repository}, howpublished = {\url{https://huggingface.co/Darknecrocities/Helios}} } ```