Kairo v1.1 — Production Crypto-Native Foundational Language Model

Kairo v1.1 (kairo-crypto-model/kairo-v1.1) is an open, crypto-native causal language model architecture pretrained exclusively on verified blockchain technical documentation, Anchor IDL frameworks, Solidity contract audits, Layer-1/Layer-2 whitepapers, EIP standards, and DeFi automated market maker specifications.

Every single token in the training corpus was indexed in real-time by autonomous Chromium jellyfish browsers reading the live crypto web.

🌐 Live Web Crawler Swarm & Dashboard: https://kairollm.live
📦 Open Training Dataset (JSONL): kairo-crypto-model/kairo-dataset
🗺️ Active Model Roadmap (Kairo 1.5 in Pre-Training): https://kairollm.live/kairo
🪙 Verified Solana Token ($JELLY): CyDJMENR6Sbtwj4erhuARNa5cNbeK92GpNAGyXLVpump

🌟 Architectural Superiority vs Legacy Toy Models

Dimension	Legacy Scratch Models (Crawlnet `queen-v1`)	Kairo v1.1 (`kairo-v1.1`)
Model Architecture	42.1M toy model (16.9M non-embed weights, 6 layers, 384 dim)	Kairo-0.1B Native Architecture (~135M parameters, RoPE + SwiGLU + GQA)
Context Window	1,024 tokens (truncates IDLs, contracts & whitepapers)	4,096 tokens (expanding to 32,768 tokens in Kairo 1.5)
Retraining Cadence	Single static batch snapshot (stale immediately)	4x Daily Autonomous Retraining (every 6h: 00, 06, 12, 18 UTC)
Factual Grounding	Repetitive hallucination loops	Live SQLite BM25 RAG with verbatim citations `[1]`, `[2]`
Crawler Incentives	Flat equal splits (sybil vulnerability, low throughput)	Performance Ranking Slabs (Apex 4.0x, Elite 2.5x, Core 1.5x...)
Deduplication Filter	Superficial heading cuts	Cryptographic SHA-256 + 64-bit SimHash (<4 bits dropped)
Telemetry & Audit	Opaque server text logs	Real-time 60fps Chromium Screencasts via WebSockets
On-Chain Settlement	Simulated / manual notes	100% Real Solana Helius RPC Verified SPL Burns + Fees

📊 Live Pretraining Corpus Statistics

All data is cleaned, deduplicated, and committed to the public Hugging Face dataset repository:

Verified Pages Ingested: 2,349 pages
Total Ingested Tokens: 10,103,249 tokens (~chars / 4)
Distinct Domains Covered: 84 Web3 developer documentation roots
Autonomous Jellyfish Swarm: 19 active browser instances
Treasury Reserve: 100.00 SOL
Hatcher Rewards Distributed: 25.62 SOL

Corpus Chapters Indexed

Layer 1 & Layer 2 Protocols: Solana Sealevel, Ethereum Execution, Arbitrum Nitro, Optimism Bedrock, Aptos Move, Sui Object runtime specifications.
DeFi Protocols & AMMs: Uniswap v3/v4 concentrated liquidity math, Raydium CLMM, Morpho Blue, Aave v3, Curve stableswap invariants.
Crypto AI & DePIN Compute: Bittensor subnets, Olas autonomous agent stack, Render Network, Akash Network, io.net GPU clustering.
Research & Whitepapers: Satoshi Nakamoto Bitcoin whitepaper, Ethereum Research (ethresear.ch), PBS (Proposer-Builder Separation), MEV-Boost mechanics.
Developer Standards: EIP/ERC standards (ERC-20, ERC-721, EIP-4844 blobs, ERC-4337 Account Abstraction, Solana Anchor IDL).
Smart Contract Security: Audit reports, reentrancy guards, formal verification patterns, invariant testing suites.
DAO Governance: Governance proposals, snapshot voting rationale, tokenomic vesting schedules.
Crypto Architecture Archives: Solana BPF/eBPF runtime, Move VM bytecode, EVM opcodes, zero-knowledge STARK/SNARK circuits.

🚀 Quickstart & Inference

Using Hugging Face Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "kairo-crypto-model/kairo-v1.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

prompt = "Explain how Solana's Proof of History prevents validator timestamp manipulation in block leader rotation:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

High-Throughput Production Serving with vLLM

vllm serve kairo-crypto-model/kairo-v1.1 \
  --served-model-name kairo \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.90

🗺️ Next Milestone: Kairo 1.5 in Active Pre-Training

Kairo is built for continuous evolutionary growth. Currently, Kairo 1.5 is in active pre-training on an 8x H100 GPU cluster funded transparently from the protocol treasury.

Phase 01 [Live Now] Feed the Swarm: Grow dataset past 25,000+ pages across all 8 crypto chapters.
Phase 02 [Next] Train for Real: Treasury-funded GPU training run with on-chain compute receipts and live loss curves on kairollm.live.
Phase 03 [Then] Prove It: Public 200-question crypto Q&A benchmark scoring Kairo 1.5 vs v1.1 side-by-side.
Phase 04 [Launch] Ship Kairo 1.5: Open weights published to Hugging Face, Ask Kairo running natively on Kairo 1.5.

📜 Model Specifications & Hyperparameters

Hyperparameter	Value
Parameters	134,847,744 (~135M)
Layers	12 Transformer Decoder blocks
Hidden Dimension ($d_{model}$)	768
Attention Heads	12 query heads, 4 key/value heads (Grouped-Query Attention)
Intermediate Size	2,048 (SwiGLU activation)
Context Window	4,096 tokens
Position Embeddings	Rotary Position Embeddings (RoPE, $\theta=10000$)
Vocabulary Size	32,000 crypto-tokenized BPE units
Precision	Float16 (`model.safetensors`)

⚖️ License & Attribution

Released under the Apache 2.0 License.
Model checkpoints, weights, and dataset shards are updated autonomously by the Kairo Autonomous Network.

Downloads last month: 93

Safetensors

Model size

0.1B params

Tensor type

F16