opendsec / index.html
zhyb99's picture
Replace the template page with the OpenDsec project page
639191c verified
Raw History Blame Contribute Delete
9.53 kB
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>OpenDsec — self-hosted sandboxes for AI agents and RL</title>
<meta name="description" content="OpenDsec is an open-source, self-hosted sandbox platform for reinforcement-learning rollouts and code-executing AI agents. An independent implementation of the DSec paper (arXiv:2609.22978).">
<style>
:root {
--bg: #ffffff;
--fg: #12151a;
--muted: #5c6470;
--line: #e3e6ea;
--accent: #1f5eff;
--code-bg: #f6f7f9;
}
@media (prefers-color-scheme: dark) {
:root {
--bg: #0f1115;
--fg: #e8eaee;
--muted: #9aa3af;
--line: #262b33;
--accent: #6f97ff;
--code-bg: #171a20;
}
}
* { box-sizing: border-box; }
body {
margin: 0;
background: var(--bg);
color: var(--fg);
font: 16px/1.65 -apple-system, BlinkMacSystemFont, "Segoe UI", Inter, Roboto, "Helvetica Neue", Arial, sans-serif;
-webkit-font-smoothing: antialiased;
}
.wrap { max-width: 780px; margin: 0 auto; padding: 64px 24px 96px; }
header { border-bottom: 1px solid var(--line); padding-bottom: 32px; margin-bottom: 40px; }
h1 { font-size: 2.1rem; line-height: 1.2; margin: 0 0 12px; letter-spacing: -0.02em; }
.lede { font-size: 1.08rem; color: var(--muted); margin: 0; }
h2 { font-size: 1.22rem; margin: 44px 0 14px; letter-spacing: -0.01em; }
p { margin: 0 0 14px; }
a { color: var(--accent); text-decoration: none; }
a:hover { text-decoration: underline; }
.note {
border-left: 3px solid var(--line);
padding: 2px 0 2px 16px;
color: var(--muted);
font-size: 0.94rem;
margin: 24px 0;
}
ul { margin: 0 0 14px; padding-left: 20px; }
li { margin-bottom: 8px; }
code {
font: 0.88em/1.5 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
background: var(--code-bg);
padding: 1px 5px;
border-radius: 4px;
}
pre {
background: var(--code-bg);
border: 1px solid var(--line);
border-radius: 8px;
padding: 16px 18px;
overflow-x: auto;
font: 0.85rem/1.6 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
}
pre code { background: none; padding: 0; }
table { width: 100%; border-collapse: collapse; margin: 18px 0; font-size: 0.94rem; }
th, td { text-align: left; padding: 9px 12px; border-bottom: 1px solid var(--line); }
th { font-weight: 600; color: var(--muted); font-size: 0.82rem; text-transform: uppercase; letter-spacing: 0.04em; }
td:last-child, th:last-child { text-align: right; font-variant-numeric: tabular-nums; }
.links { display: flex; flex-wrap: wrap; gap: 10px; margin-top: 18px; }
.links a {
border: 1px solid var(--line);
border-radius: 6px;
padding: 8px 14px;
font-size: 0.92rem;
}
footer { margin-top: 64px; padding-top: 24px; border-top: 1px solid var(--line); color: var(--muted); font-size: 0.86rem; }
</style>
</head>
<body>
<div class="wrap">
<header>
<h1>OpenDsec</h1>
<p class="lede">Self-hosted sandboxes for AI agents and reinforcement-learning rollouts: Firecracker microVMs, containers and full VMs behind one API, with memory-preserving fork and an E2B-compatible gateway.</p>
<div class="links">
<a href="https://github.com/Kernflux/opendsec">Source code</a>
<a href="https://kernflux.github.io/">Website and docs</a>
<a href="https://kernflux.github.io/en/docs/quickstart">Quickstart</a>
<a href="https://arxiv.org/abs/2609.22978">The DSec paper</a>
</div>
</header>
<div class="note">
OpenDsec is an independent implementation of the architecture described in <em>DeepSeek Elastic Compute (DSec): A Sandbox Infrastructure for Effective Agentic Training at Scale</em> (arXiv:2609.22978). It is not DeepSeek code, not an official release, and not affiliated with or endorsed by that paper's authors.
</div>
<h2>What it is</h2>
<p>OpenDsec runs many isolated sandboxes on your own Linux machines, from a single host to a cluster, for reinforcement-learning rollouts and agents that execute code. It ships a Python SDK, a CLI, and an API compatible with E2B's SDKs, under AGPL-3.0.</p>
<ul>
<li><strong>Good fit:</strong> teams whose sandboxes must stay in their own data centre or an offline network; RL training that starts thousands of environments at once; workflows that need to fork a running environment with its memory; keeping existing E2B SDK code but moving it onto your own infrastructure.</li>
<li><strong>Poor fit:</strong> anyone who wants a managed service with no machines to operate (there is no hosted edition), or non-x86_64 Linux hosts.</li>
</ul>
<h2>Architecture</h2>
<ul>
<li>A <strong>stateless apiserver</strong> handles authentication, quota and placement, choosing nodes from the state a watcher aggregates.</li>
<li><strong>IAM</strong> keeps identities, projects and quotas in a Raft-replicated store.</li>
<li>Each compute node runs an <strong>edge</strong> that makes the final admission decision against its real resources.</li>
<li><strong>Aether</strong> runs inside every sandbox for the runtime channel; <strong>Chronus</strong> keeps persistent shell sessions. Exec, files, HTTP, processes and PTY share one runtime API.</li>
<li>Components talk over gRPC with mTLS, and the sandbox id encodes its edge, so no per-sandbox routing table is needed. Any component can restart without disturbing running sandboxes.</li>
</ul>
<h2>Lifecycle</h2>
<p>A microVM can be paused into a memory snapshot, restored, or forked while running. Copies on the same node share memory pages; a fork on another node pulls the snapshot from the source. A sandbox can also be packed back into a new image from its changes.</p>
<pre><code>import asyncio
from opendsec import DSecClient, DSecRunArgs
async def main():
async with DSecClient() as client:
sb = await client.run_sandbox(DSecRunArgs(
backend="microvm", image="root/py:3.12",
cpu_cores_limit=1, memory_limit_mb=1024,
network_rules={"pypi": True})) # outbound denied unless allowed
try:
print((await sb.run_shell("python3 -c 'print(6*7)'")).stdout)
forks = await sb.fork(count=4) # four copies, memory included
for f in forks:
await f.stop()
finally:
await sb.stop()
asyncio.run(main())</code></pre>
<h2>Measured on our hardware</h2>
<p>These are measurements of OpenDsec on its own machines, not figures from the paper. Method and conditions are in <code>docs/SPEC.md</code>.</p>
<table>
<thead>
<tr><th>Measurement</th><th>Value</th></tr>
</thead>
<tbody>
<tr><td>microVM cold start</td><td>~0.1 s</td></tr>
<tr><td>Restore from memory snapshot</td><td>~55 ms</td></tr>
<tr><td>Resume after pause</td><td>~50 ms</td></tr>
<tr><td>Source VM stall during fork</td><td>~10 ms</td></tr>
<tr><td>Burst of microVMs, one 384-thread node</td><td>5,120</td></tr>
<tr><td>Burst of containers, two nodes</td><td>8,192, no failures</td></tr>
</tbody>
</table>
<h2>Where it differs from the paper</h2>
<ul>
<li>The network policy is enforced with nftables and a DNS proxy rather than eBPF.</li>
<li>Images live on any shared filesystem (NFS, CephFS, JuiceFS) instead of 3FS.</li>
<li>Firecracker is the primary backend rather than one option among several.</li>
</ul>
<p>Where the paper leaves a decision open, <code>docs/SPEC.md</code> marks each point as either taken from the paper, with its section, or the project's own decision.</p>
<h2>Known limits</h2>
<ul>
<li>The image store's availability is whatever the chosen shared filesystem provides.</li>
<li>A host reboot loses that node's sandboxes, paused microVMs included.</li>
<li>QEMU PCI passthrough has not been tested on real devices.</li>
<li>The E2B gateway does not yet cover directory watching, signed file URLs, unauthenticated browser access to sandbox ports, or template building.</li>
</ul>
<h2>Try it</h2>
<p>One Linux x86_64 machine with systemd and cgroup v2 is enough. MicroVMs need <code>/dev/kvm</code>; without it the script sets up containers only.</p>
<pre><code>git clone https://github.com/Kernflux/opendsec
cd opendsec
deploy/quickstart.sh # or --no-microvm for containers only</code></pre>
<p>Existing E2B programs need only three environment variables pointed at the gateway:</p>
<pre><code>export E2B_API_URL=https://sandbox.internal:7400
export E2B_SANDBOX_URL=https://sandbox.internal:7400
export E2B_API_KEY=dsk.…</code></pre>
<h2>Links</h2>
<ul>
<li><a href="https://github.com/Kernflux/opendsec">Source and changelog</a></li>
<li><a href="https://kernflux.github.io/">Website</a></li>
<li><a href="https://kernflux.github.io/en/docs/quickstart">Quickstart</a></li>
<li><a href="https://kernflux.github.io/en/docs/e2b">Moving from E2B</a></li>
<li><a href="https://huggingface.co/papers/2609.22978">The DSec paper on Hugging Face</a></li>
</ul>
<h2>Licensing</h2>
<p>The server and tools are AGPL-3.0. The client parts (the Python SDK, the Go client, and the API definitions) are Apache-2.0, so programs using OpenDsec are not bound by the AGPL.</p>
<footer>
"DeepSeek" and "DSec" are names of their respective owners. OpenDsec uses them only to identify the paper it implements.
</footer>
</div>
</body>
</html>