SLM-Agents / website /embeddings_server.html
spcv's picture
Design polish: welcome card layout & icons, assistant tables/links/typography, mobile responsiveness
61846f9 verified
Raw History Blame Contribute Delete
22.8 kB
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Ultra-fast local ONNX embedding generation providing 384-dimensional dense semantic vectors for RAG and semantic search.">
<link rel="canonical" href="https://www.slmagents.ai/embeddings_server.html">
<title>SLM Embeddings Server: High-Throughput Dense Vector Generator | SLMAgents</title>
<link rel="stylesheet" href="style.css?v=64.0">
<!-- OpenGraph / Social Meta Tags -->
<meta property="og:type" content="article">
<meta property="og:url" content="https://www.slmagents.ai/embeddings_server.html">
<meta property="og:title" content="SLM Embeddings Server: High-Throughput Dense Vector Generator | SLMAgents">
<meta property="og:description" content="Ultra-fast local ONNX embedding generation providing 384-dimensional dense semantic vectors for RAG and semantic search.">
<meta property="og:site_name" content="SLMAgents">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="SLM Embeddings Server: High-Throughput Dense Vector Generator | SLMAgents">
<meta name="twitter:description" content="Ultra-fast local ONNX embedding generation providing 384-dimensional dense semantic vectors for RAG and semantic search.">
<!-- Structured Data / Schema Markup -->
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "TechArticle",
"headline": "SLM Embeddings Server: High-Throughput Dense Vector Generator | SLMAgents",
"description": "Ultra-fast local ONNX embedding generation providing 384-dimensional dense semantic vectors for RAG and semantic search.",
"url": "https://www.slmagents.ai/embeddings_server.html",
"mainEntityOfPage": "https://www.slmagents.ai/embeddings_server.html",
"articleSection": "Data & Utilities",
"author": {
"@type": "Organization",
"name": "SLMAgents Community",
"url": "https://github.com/t00114218-stack/SLMAgents"
},
"publisher": {
"@type": "Organization",
"name": "SLMAgents",
"logo": {
"@type": "ImageObject",
"url": "https://www.slmagents.ai/flowchart.png"
}
}
}
</script>
</head>
<body>
<!-- Top Engineering Header -->
<header class="site-top-header web-only-nav">
<div class="container nav-container">
<div style="display: flex; align-items: center; gap: 10px;">
<button class="sidebar-toggle" onclick="toggleSidebar()" aria-label="Toggle Sidebar">
<svg width="20" height="20" fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" d="M4 6h16M4 12h16M4 18h16"/>
</svg>
</button>
<a href="index.html" class="logo">
<div class="logo-icon">
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round">
<path d="M12 2L2 7l10 5 10-5-10-5zM2 17l10 5 10-5M2 12l10 5 10-5"/>
</svg>
</div>
<span>SLMAgents</span>
</a>
</div>
<nav>
<ul class="nav-links">
<li><a href="index.html" id="topnav-home">Home</a></li>
<li><a href="chat.html" id="topnav-chat">Chat Studio</a></li>
<li><a href="playground.html" id="topnav-playground">Playground</a></li>
<li><a href="blogs.html" id="topnav-blogs">Blogs</a></li>
<li>
<button type="button" onclick="openDownloadModal()" class="btn-nav-download">
<svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="7 10 12 15 17 10"/><line x1="12" y1="15" x2="12" y2="3"/></svg>
Download App
</button>
</li>
<li>
<a href="https://github.com/t00114218-stack/SLMAgents" target="_blank" class="btn-github-nav">
<svg width="14" height="14" fill="currentColor" viewBox="0 0 16 16">
<path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27s1.36.09 2 .27c1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.01 8.01 0 0 0 16 8c0-4.42-3.58-8-8-8"/>
</svg>
GitHub
</a>
</li>
</ul>
</nav>
</div>
</header>
<div class="layout-container">
<aside class="sidebar" id="sidebar">
<div class="sidebar-group">
<div class="sidebar-group-title">Active Libraries</div>
<ul class="sidebar-list" id="sidebar-active-list"></ul>
</div>
</aside>
<main class="main-content">
<div class="doc-content-wrapper">
<section class="doc-section" style="margin-top: 0;">
<div class="breadcrumb" style="font-family: var(--font-mono); font-size: 0.75rem; color: var(--text-muted); margin-bottom: 0.75rem;"><a href="index.html" style="color: var(--text-secondary); text-decoration: none;">Overview</a> <span style="margin: 0 6px;">/</span> <span>slm_embeddings</span> <span style="margin: 0 6px;">/</span> <span style="color: var(--text-primary); font-weight: 600;">SLM Embeddings Server</span></div>
<div class="badge-pill">slm_embeddings · Data & Utilities · 10-Plane Pure-CPU Agent</div>
<h1 style="font-size: 2.25rem; color: #0f172a; font-weight: 800; margin-bottom: 0.75rem;">SLM Embeddings Server</h1>
<p style="font-size: 1.05rem; color: #334155; line-height: 1.65; font-weight: 500; margin: 0;">Starts a local CPU-optimized embedding server to compute dense document and query vectors on standard hardware.</p>
</section>
<nav class="doc-nav">
<a href="#overview">01 / Overview</a>
<a href="#install">02 / OS Installers &amp; CLI</a>
<a href="#harness">03 / 10-Plane Harness</a>
<a href="#git">04 / Git Checkout</a>
<a href="#config">05 / API Reference</a>
</nav>
<section class="doc-section" id="overview">
<h2>01 / Overview &amp; Capabilities</h2>
<p>Starts a local CPU-optimized embedding server to compute dense document and query vectors on standard hardware.</p>
<h3 style="margin-top: 1.5rem; font-size: 1.15rem; color: #0f172a; font-weight: 700;">Key Features</h3>
<ul>
<li><strong>Loads quantized mini-LM or BGE embeddings locally</strong></li>
<li><strong>High-speed cosine similarity index built directly in memory</strong></li>
<li><strong>Provides local HTTP API endpoint for integration</strong></li>
<li><strong>Under 200 MB RAM memory usage footprint during idle states</strong></li>
</ul>
</section>
<!-- INSTALLATION & STANDALONE OS HARNESS INSTALLERS -->
<section class="doc-section" id="install">
<h2>02 / Standalone OS Harness Installers, CLI &amp; Python SDK</h2>
<p>In <strong>SLMAgents 2.0</strong>, <strong>SLM Embeddings Server</strong> executes inside the <strong>10-Plane Pure-CPU Agent Harness (<code>slm_harness</code>)</strong>. Download a self-extracting <strong>Individual Agent + Harness OS Installer</strong> for macOS, Windows, or Linux (bundling <code>slm_embeddings</code> + <code>slm_harness</code> + the <code>./slm</code> CLI), install the full 26-agent desktop suite, or install via Python <code>pip</code>:</p>
<h3>1. Individual SLM Embeddings Server + 10-Plane Harness Installers (All OSes)</h3>
<div class="subpage-installer-grid">
<a href="/api/installers/download/agent-macos?agent=embeddings_server" class="subpage-installer-card">
<div class="subpage-installer-title">
<span>macOS Installer</span>
<span class="subpage-installer-ext">.command</span>
</div>
<div class="subpage-installer-desc">Self-extracting macOS bundle with <code>slm_embeddings</code> + <code>slm_harness</code> + <code>./slm</code> CLI</div>
</a>
<a href="/api/installers/download/agent-windows?agent=embeddings_server" class="subpage-installer-card">
<div class="subpage-installer-title">
<span>Windows Installer</span>
<span class="subpage-installer-ext">.exe</span>
</div>
<div class="subpage-installer-desc">Native 64-bit Windows Setup with portable Python 3.11 + <code>slm_embeddings</code> + <code>slm_harness</code></div>
</a>
<a href="/api/installers/download/agent-linux?agent=embeddings_server" class="subpage-installer-card">
<div class="subpage-installer-title">
<span>Linux Installer</span>
<span class="subpage-installer-ext">.AppImage</span>
</div>
<div class="subpage-installer-desc">Standalone Linux AppImage with <code>slm_embeddings</code> + <code>slm_harness</code> + <code>./slm</code> CLI</div>
</a>
</div>
<h3>2. Full 26-Agent Studio &amp; Desktop OS Installers</h3>
<div class="subpage-suite-links">
<a href="/api/installers/download/macos-dmg" class="subpage-suite-link">macOS .dmg (280 MB)</a>
<a href="/api/installers/download/windows" class="subpage-suite-link">Windows .exe (2.8 MB)</a>
<a href="/api/installers/download/linux" class="subpage-suite-link">Linux .AppImage (3.4 MB)</a>
<button onclick="openDownloadModal()" class="btn btn-primary" style="padding: 0.38rem 0.8rem; font-size: 0.76rem;">All OS Installers Modal</button>
</div>
<h3>3. 10-Plane Harness CLI (<code>./slm</code>) &amp; Python <code>pip</code> Installation</h3>
<div class="code-panel" style="margin: 0.85rem 0;">
<div class="code-header">
<div class="code-dots"><div class="code-dot"></div><div class="code-dot"></div><div class="code-dot"></div></div>
<div class="code-title">Terminal — 10-Plane Harness CLI &amp; Pip</div>
</div>
<div class="code-content" style="display:block;">
<pre><span style="color:#64748b;"># Option A: Install via PyPI or editable source with the 10-Plane Harness</span>
<span style="color:#34d399;">$</span> pip install slm-embeddings
<span style="color:#34d399;">$</span> pip install -e ./slm_harness -e ./slm_embeddings
<span style="color:#64748b;"># Option B: Run directly through the Unified 10-Plane Pure-CPU Harness CLI (./slm)</span>
<span style="color:#34d399;">$</span> ./slm serve --port 7860 # Exposes /v1/embeddings (1024-d INT8 mxbai-embed-large)
<span style="color:#34d399;">$</span> ./slm chat <span style="color:#64748b;"># Interactive Rich TUI with 18 slash commands (/trace, /models, /arena, /undo)</span>
<span style="color:#34d399;">$</span> ./slm serve <span style="color:#64748b;"># 100% Offline Web AI Studio + OpenAI /v1 API at http://127.0.0.1:7860</span></pre>
</div>
</div>
</section>
<!-- 10-PLANE PURE-CPU HARNESS ARCHITECTURE -->
<section class="doc-section" id="harness">
<h2>03 / 10-Plane Pure-CPU Harness Integration (<code>slm_harness</code>)</h2>
<p>Rather than executing as an isolated prompt wrapper, <strong>SLM Embeddings Server</strong> is natively orchestrated by the <strong>SLMAgents 2.0 10-Plane Pure-CPU Harness</strong> (via <code>slm_harness.runtime.legacy_bridge</code> and <code>CascadeRouter</code>). Every call benefits from hybrid <code>llama.cpp</code> GGUF (<code>Q4_K_M</code>) + Quantized INT4/INT8 CPU execution, watermarked LRU memory governance, grammar-constrained decoding, and deterministic verification at <strong>$0 API cost</strong>:</p>
<table class="param-table">
<thead>
<tr><th>Harness Plane</th><th>Subsystem</th><th>How It Powers SLM Embeddings Server on Pure CPU</th></tr>
</thead>
<tbody>
<tr>
<td><strong>Plane 6</strong></td>
<td><code>CascadeRouter</code> (&lt;35ms)</td>
<td>Stage 2 (Vectorization / Cosine Similarity / /v1/embeddings Request)</td>
</tr>
<tr>
<td><strong>Plane 2 &amp; 5</strong></td>
<td><code>MemoryArena</code> &amp; Hybrid CPU Runtime</td>
<td>Specialist Role: <code>embeddings (mxbai-embed-large-v1 1024-d INT8) + reranker (bge-reranker-base INT8)</code>. Managed inside Watermark LRU <code>MemoryArena</code> (<code>lite</code> 2.6 GB / <code>balanced</code> 4.7 GB / <code>power</code> 8.2 GB) with zero-copy session sharing and automatic OOM tier step-down (<code>8B -&gt; 4B -&gt; 3B -&gt; 0.8B</code>).</td>
</tr>
<tr>
<td><strong>Plane 7 &amp; 8</strong></td>
<td>Deterministic Verifier &amp; Governance</td>
<td>Plane 5 MemoryArena Warm Pinning (0.03ms warm latency) + L2 Normalization & Dimension (1024-d) Verifier</td>
</tr>
<tr>
<td><strong>Plane 1, 3 &amp; 4</strong></td>
<td>Kernel, Artifact Scratchpad &amp; 4-Tier Memory</td>
<td>SQLite WAL <code>CheckpointStore</code> (<code>/fork</code>, <code>/rewind</code>, <code>/undo</code>), content-addressed <code>artifact://&lt;sha12&gt;</code> handles to prevent prompt bloat, and hybrid SQLite FTS5 BM25 + HNSW vector memory.</td>
</tr>
</tbody>
</table>
</section>
<!-- GIT CHECKOUT -->
<section class="doc-section" id="git">
<h2>04 / Source Checkout (Git Sparse-Checkout)</h2>
<p>Clone this agent alongside the <strong>10-Plane Pure-CPU Harness (<code>slm_harness</code>)</strong> from the monorepo using Git sparse-checkout — no need to download all 26 agents:</p>
<h3>Option 1 — Sparse Checkout with 10-Plane Harness (Recommended)</h3>
<div class="code-panel" style="margin: 0.85rem 0;">
<div class="code-header">
<div class="code-dots"><div class="code-dot"></div><div class="code-dot"></div><div class="code-dot"></div></div>
<div class="code-title">Terminal — Git Sparse Checkout</div>
</div>
<div class="code-content" style="display:block;">
<pre><span style="color:#64748b;"># 1. Create and enter a new directory</span>
<span style="color:#34d399;">$</span> mkdir slm_embeddings_workspace &amp;&amp; cd slm_embeddings_workspace
<span style="color:#64748b;"># 2. Initialise empty git repo and add remote</span>
<span style="color:#34d399;">$</span> git init
<span style="color:#34d399;">$</span> git remote add origin https://github.com/t00114218-stack/SLMAgents.git
<span style="color:#64748b;"># 3. Enable sparse-checkout for both slm_harness and slm_embeddings</span>
<span style="color:#34d399;">$</span> git sparse-checkout init --cone
<span style="color:#34d399;">$</span> git sparse-checkout set slm_harness slm_embeddings
<span style="color:#64748b;"># 4. Pull the 10-Plane Harness and agent source</span>
<span style="color:#34d399;">$</span> git pull origin main</pre>
</div>
</div>
<h3>Option 2 — Full Repository Clone</h3>
<div class="code-panel" style="margin: 0.85rem 0;">
<div class="code-header">
<div class="code-dots"><div class="code-dot"></div><div class="code-dot"></div><div class="code-dot"></div></div>
<div class="code-title">Terminal — Full Clone</div>
</div>
<div class="code-content" style="display:block;">
<pre><span style="color:#34d399;">$</span> git clone https://github.com/t00114218-stack/SLMAgents.git
<span style="color:#34d399;">$</span> cd SLMAgents
<span style="color:#34d399;">$</span> ./slm doctor</pre>
</div>
</div>
<p style="margin-top: 1.25rem; font-size: 0.86rem; color: var(--text-secondary); border-left: 2px solid #09090b; padding-left: 1rem;">
<strong>Note:</strong> After checkout, install the harness and agent in editable mode with <code>pip install -e ./slm_harness -e ./slm_embeddings</code> or download the pre-packaged OS Harness Installer above.
</p>
</section>
<section class="doc-section" id="config">
<h2>05 / Configuration &amp; API Reference</h2>
<h3>Constructor Parameters</h3>
<p>Instantiate <code>SLMEmbeddingsServer</code> with performance options:</p>
<table class="param-table">
<thead>
<tr><th>Parameter</th><th>Type / Default</th><th>Description</th></tr>
</thead>
<tbody>
<tr><td>model_path</td><td>str | None</td><td>Explicit path to GGUF (Q4_K_M) or quantized ONNX model weights. If omitted, managed automatically by the 10-Plane Harness MemoryArena.</td></tr>
<tr><td>cache_dir</td><td>str | None</td><td>Directory to store model weights offline. Defaults to <code>~/.cache/slm-embeddings/</code>. Also settable via <span class="env-tag">SLM_EMBEDDINGS_SERVER_CACHE_DIR</span>.</td></tr>
<tr><td>n_threads</td><td>int | 4</td><td>CPU thread count for 10-Plane Harness inference (auto-tuned to physical P-cores). Optimize for CPU core count. Also settable via <span class="env-tag">SLM_EMBEDDINGS_SERVER_N_THREADS</span>.</td></tr>
</tbody>
</table>
<h3>Methods</h3>
<table class="param-table">
<thead>
<tr><th>Method Signature</th><th>Return Type</th><th>Description</th></tr>
</thead>
<tbody>
<tr><td><code>embed(texts, system_prompt=None, user_input=None)</code></td><td><code>list[float]</code></td><td>Generates dense vectors from standard string lists.</td></tr>
</tbody>
</table>
<h3>Method Parameters (Execution Customization)</h3>
<p>All main execution methods accept optional system routing parameters:</p>
<table class="param-table">
<thead>
<tr><th>Parameter</th><th>Type / Default</th><th>Description</th></tr>
</thead>
<tbody>
<tr><td>system_prompt</td><td>str | None</td><td>Optional custom system prompt instruction to override the default system template response parameters.</td></tr>
<tr><td>user_input</td><td>str | None</td><td>Optional additional user-supplied target text variables or contextual keys.</td></tr>
</tbody>
</table>
<h3>Quick Start</h3>
<pre><code class="language-python">from slm_embeddings import SLMEmbeddingsServer
server = SLMEmbeddingsServer()
vector = server.embed(
["sample test"],
system_prompt="Calculate semantic weights",
user_input="Normalized cosine distance"
)
print(vector)</code></pre>
</section>
<section class="doc-section" id="env-vars">
<h2>Environment Variables</h2>
<p>Configure agent parameters globally using environment values:</p>
<table class="param-table">
<thead>
<tr><th>Environment Variable</th><th>Default</th><th>Purpose</th></tr>
</thead>
<tbody>
<tr><td>SLM_EMBEDDINGS_SERVER_N_THREADS</td><td>4</td><td>Sets CPU inference execution threads.</td></tr>
<tr><td>SLM_EMBEDDINGS_SERVER_CACHE_DIR</td><td>~/.cache/slm-embeddings/</td><td>Default directory to store local GGUF (Q4_K_M) and quantized INT4/INT8 model weights (shared with ~/.slm_harness/models).</td></tr>
</tbody>
</table>
</section>
<section class="doc-section" id="performance">
<h2>CPU Performance Tuning</h2>
<p>To run the <code>SLMEmbeddingsServer</code> engine efficiently on CPU under 1.5 GB memory footprint:</p>
<ul>
<li><strong>Match Threads to Core Count:</strong> Set <code>n_threads</code> or <code>SLM_EMBEDDINGS_SERVER_N_THREADS</code> to match the physical CPU core count.</li>
<li><strong>Sequential Processing:</strong> Avoid concurrent processing when batch files are large.</li>
<li><strong>Garbage Collection:</strong> Clear variables and run <code>gc.collect()</code> to release model RAM blocks after execution.</li>
</ul>
</section>
<section class="doc-section">
<h2>Verified Input & Output Logs</h2>
<p>Diagnostic execution console response running locally on CPU:</p>
<div class="tip-box" style="background: rgba(30, 41, 59, 0.7); border: 1px solid rgba(255, 255, 255, 0.08);">
<pre><code style="color: #38bdf8; font-family: monospace; font-size: 0.9rem; line-height: 1.5; display: block; white-space: pre-wrap;">→ INPUT:
"sample test"
← OUTPUT:
"Vector dimension check: 1024"</code></pre>
</div>
</section>
<footer>
<div class="footer-row">
<div><strong style="color: var(--text-primary);">SLMAgents 2.0</strong> · Powered by the 10-Plane Pure-CPU Agent Harness (<code>slm_harness</code>) · MIT License</div>
<div class="footer-links">
<a href="index.html">Home</a>
<a href="index.html#os-installers">OS Installers</a>
<a href="chat.html">Chat Studio</a>
<a href="playground.html">Playground</a>
<a href="blogs.html">Blogs</a>
<a href="https://github.com/t00114218-stack/SLMAgents" target="_blank">GitHub</a>
</div>
</div>
</footer>
</div>
</main>
</div>
<script src="app.js?v=100.0"></script>
</body>
</html>