File size: 9,527 Bytes
9baba57
639191c
9baba57
639191c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9baba57
 
639191c
 
 
 
 
 
 
 
 
 
9baba57
639191c
9baba57
639191c
 
9baba57
 
639191c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9baba57
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>OpenDsec — self-hosted sandboxes for AI agents and RL</title>
<meta name="description" content="OpenDsec is an open-source, self-hosted sandbox platform for reinforcement-learning rollouts and code-executing AI agents. An independent implementation of the DSec paper (arXiv:2609.22978).">
<style>
  :root {
    --bg: #ffffff;
    --fg: #12151a;
    --muted: #5c6470;
    --line: #e3e6ea;
    --accent: #1f5eff;
    --code-bg: #f6f7f9;
  }
  @media (prefers-color-scheme: dark) {
    :root {
      --bg: #0f1115;
      --fg: #e8eaee;
      --muted: #9aa3af;
      --line: #262b33;
      --accent: #6f97ff;
      --code-bg: #171a20;
    }
  }
  * { box-sizing: border-box; }
  body {
    margin: 0;
    background: var(--bg);
    color: var(--fg);
    font: 16px/1.65 -apple-system, BlinkMacSystemFont, "Segoe UI", Inter, Roboto, "Helvetica Neue", Arial, sans-serif;
    -webkit-font-smoothing: antialiased;
  }
  .wrap { max-width: 780px; margin: 0 auto; padding: 64px 24px 96px; }
  header { border-bottom: 1px solid var(--line); padding-bottom: 32px; margin-bottom: 40px; }
  h1 { font-size: 2.1rem; line-height: 1.2; margin: 0 0 12px; letter-spacing: -0.02em; }
  .lede { font-size: 1.08rem; color: var(--muted); margin: 0; }
  h2 { font-size: 1.22rem; margin: 44px 0 14px; letter-spacing: -0.01em; }
  p { margin: 0 0 14px; }
  a { color: var(--accent); text-decoration: none; }
  a:hover { text-decoration: underline; }
  .note {
    border-left: 3px solid var(--line);
    padding: 2px 0 2px 16px;
    color: var(--muted);
    font-size: 0.94rem;
    margin: 24px 0;
  }
  ul { margin: 0 0 14px; padding-left: 20px; }
  li { margin-bottom: 8px; }
  code {
    font: 0.88em/1.5 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
    background: var(--code-bg);
    padding: 1px 5px;
    border-radius: 4px;
  }
  pre {
    background: var(--code-bg);
    border: 1px solid var(--line);
    border-radius: 8px;
    padding: 16px 18px;
    overflow-x: auto;
    font: 0.85rem/1.6 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
  }
  pre code { background: none; padding: 0; }
  table { width: 100%; border-collapse: collapse; margin: 18px 0; font-size: 0.94rem; }
  th, td { text-align: left; padding: 9px 12px; border-bottom: 1px solid var(--line); }
  th { font-weight: 600; color: var(--muted); font-size: 0.82rem; text-transform: uppercase; letter-spacing: 0.04em; }
  td:last-child, th:last-child { text-align: right; font-variant-numeric: tabular-nums; }
  .links { display: flex; flex-wrap: wrap; gap: 10px; margin-top: 18px; }
  .links a {
    border: 1px solid var(--line);
    border-radius: 6px;
    padding: 8px 14px;
    font-size: 0.92rem;
  }
  footer { margin-top: 64px; padding-top: 24px; border-top: 1px solid var(--line); color: var(--muted); font-size: 0.86rem; }
</style>
</head>
<body>
<div class="wrap">

  <header>
    <h1>OpenDsec</h1>
    <p class="lede">Self-hosted sandboxes for AI agents and reinforcement-learning rollouts: Firecracker microVMs, containers and full VMs behind one API, with memory-preserving fork and an E2B-compatible gateway.</p>
    <div class="links">
      <a href="https://github.com/Kernflux/opendsec">Source code</a>
      <a href="https://kernflux.github.io/">Website and docs</a>
      <a href="https://kernflux.github.io/en/docs/quickstart">Quickstart</a>
      <a href="https://arxiv.org/abs/2609.22978">The DSec paper</a>
    </div>
  </header>

  <div class="note">
    OpenDsec is an independent implementation of the architecture described in <em>DeepSeek Elastic Compute (DSec): A Sandbox Infrastructure for Effective Agentic Training at Scale</em> (arXiv:2609.22978). It is not DeepSeek code, not an official release, and not affiliated with or endorsed by that paper's authors.
  </div>

  <h2>What it is</h2>
  <p>OpenDsec runs many isolated sandboxes on your own Linux machines, from a single host to a cluster, for reinforcement-learning rollouts and agents that execute code. It ships a Python SDK, a CLI, and an API compatible with E2B's SDKs, under AGPL-3.0.</p>
  <ul>
    <li><strong>Good fit:</strong> teams whose sandboxes must stay in their own data centre or an offline network; RL training that starts thousands of environments at once; workflows that need to fork a running environment with its memory; keeping existing E2B SDK code but moving it onto your own infrastructure.</li>
    <li><strong>Poor fit:</strong> anyone who wants a managed service with no machines to operate (there is no hosted edition), or non-x86_64 Linux hosts.</li>
  </ul>

  <h2>Architecture</h2>
  <ul>
    <li>A <strong>stateless apiserver</strong> handles authentication, quota and placement, choosing nodes from the state a watcher aggregates.</li>
    <li><strong>IAM</strong> keeps identities, projects and quotas in a Raft-replicated store.</li>
    <li>Each compute node runs an <strong>edge</strong> that makes the final admission decision against its real resources.</li>
    <li><strong>Aether</strong> runs inside every sandbox for the runtime channel; <strong>Chronus</strong> keeps persistent shell sessions. Exec, files, HTTP, processes and PTY share one runtime API.</li>
    <li>Components talk over gRPC with mTLS, and the sandbox id encodes its edge, so no per-sandbox routing table is needed. Any component can restart without disturbing running sandboxes.</li>
  </ul>

  <h2>Lifecycle</h2>
  <p>A microVM can be paused into a memory snapshot, restored, or forked while running. Copies on the same node share memory pages; a fork on another node pulls the snapshot from the source. A sandbox can also be packed back into a new image from its changes.</p>

  <pre><code>import asyncio
from opendsec import DSecClient, DSecRunArgs

async def main():
    async with DSecClient() as client:
        sb = await client.run_sandbox(DSecRunArgs(
            backend="microvm", image="root/py:3.12",
            cpu_cores_limit=1, memory_limit_mb=1024,
            network_rules={"pypi": True}))   # outbound denied unless allowed
        try:
            print((await sb.run_shell("python3 -c 'print(6*7)'")).stdout)
            forks = await sb.fork(count=4)   # four copies, memory included
            for f in forks:
                await f.stop()
        finally:
            await sb.stop()

asyncio.run(main())</code></pre>

  <h2>Measured on our hardware</h2>
  <p>These are measurements of OpenDsec on its own machines, not figures from the paper. Method and conditions are in <code>docs/SPEC.md</code>.</p>
  <table>
    <thead>
      <tr><th>Measurement</th><th>Value</th></tr>
    </thead>
    <tbody>
      <tr><td>microVM cold start</td><td>~0.1 s</td></tr>
      <tr><td>Restore from memory snapshot</td><td>~55 ms</td></tr>
      <tr><td>Resume after pause</td><td>~50 ms</td></tr>
      <tr><td>Source VM stall during fork</td><td>~10 ms</td></tr>
      <tr><td>Burst of microVMs, one 384-thread node</td><td>5,120</td></tr>
      <tr><td>Burst of containers, two nodes</td><td>8,192, no failures</td></tr>
    </tbody>
  </table>

  <h2>Where it differs from the paper</h2>
  <ul>
    <li>The network policy is enforced with nftables and a DNS proxy rather than eBPF.</li>
    <li>Images live on any shared filesystem (NFS, CephFS, JuiceFS) instead of 3FS.</li>
    <li>Firecracker is the primary backend rather than one option among several.</li>
  </ul>
  <p>Where the paper leaves a decision open, <code>docs/SPEC.md</code> marks each point as either taken from the paper, with its section, or the project's own decision.</p>

  <h2>Known limits</h2>
  <ul>
    <li>The image store's availability is whatever the chosen shared filesystem provides.</li>
    <li>A host reboot loses that node's sandboxes, paused microVMs included.</li>
    <li>QEMU PCI passthrough has not been tested on real devices.</li>
    <li>The E2B gateway does not yet cover directory watching, signed file URLs, unauthenticated browser access to sandbox ports, or template building.</li>
  </ul>

  <h2>Try it</h2>
  <p>One Linux x86_64 machine with systemd and cgroup v2 is enough. MicroVMs need <code>/dev/kvm</code>; without it the script sets up containers only.</p>
  <pre><code>git clone https://github.com/Kernflux/opendsec
cd opendsec
deploy/quickstart.sh        # or --no-microvm for containers only</code></pre>
  <p>Existing E2B programs need only three environment variables pointed at the gateway:</p>
  <pre><code>export E2B_API_URL=https://sandbox.internal:7400
export E2B_SANDBOX_URL=https://sandbox.internal:7400
export E2B_API_KEY=dsk.…</code></pre>

  <h2>Links</h2>
  <ul>
    <li><a href="https://github.com/Kernflux/opendsec">Source and changelog</a></li>
    <li><a href="https://kernflux.github.io/">Website</a></li>
    <li><a href="https://kernflux.github.io/en/docs/quickstart">Quickstart</a></li>
    <li><a href="https://kernflux.github.io/en/docs/e2b">Moving from E2B</a></li>
    <li><a href="https://huggingface.co/papers/2609.22978">The DSec paper on Hugging Face</a></li>
  </ul>

  <h2>Licensing</h2>
  <p>The server and tools are AGPL-3.0. The client parts (the Python SDK, the Go client, and the API definitions) are Apache-2.0, so programs using OpenDsec are not bound by the AGPL.</p>

  <footer>
    "DeepSeek" and "DSec" are names of their respective owners. OpenDsec uses them only to identify the paper it implements.
  </footer>

</div>
</body>
</html>