Leon4gr45 commited on
Commit
ffddb2c
Β·
verified Β·
1 Parent(s): a07b1b4

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +549 -0
README.md ADDED
@@ -0,0 +1,549 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: BrainAPI
3
+ emoji: 🧠
4
+ colorFrom: blue
5
+ colorTo: green
6
+ sdk: docker
7
+ app_port: 7860
8
+ pinned: false
9
+ ---
10
+
11
+ <p align="center">
12
+ <a href="https://discord.gg/VTngQTaeDf"><img src="https://img.shields.io/badge/Discord-Join%20Lumen%20Brain-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Discord"/></a>
13
+ <img src="https://img.shields.io/badge/version-2.19.0--dev-blue?style=for-the-badge" alt="Version"/>
14
+ <img src="https://img.shields.io/badge/python-3.11+-green?style=for-the-badge&logo=python&logoColor=white" alt="Python"/>
15
+ <img src="https://img.shields.io/badge/license-BUSL--1.1-blue?style=for-the-badge" alt="License"/>
16
+ </p>
17
+
18
+ <h1 align="center">🧠 BrainAPI</h1>
19
+
20
+ <p align="center">
21
+ <strong>Turn raw text into a living knowledge graph β€” automatically.</strong>
22
+ <br/><br/>
23
+ <em>You send text. A swarm of agents reads it, takes notes, and draws the graph for you.<br/>
24
+ Then you ask questions and get answers backed by a traceable path β€” not a similarity guess.</em>
25
+ </p>
26
+
27
+ <p align="center">
28
+ <a href="https://brainapi.lumen-labs.ai/">BrainAPI Cloud</a> β€’
29
+ <a href="#-quickstart-fastest-way-to-run-it">Quickstart</a> β€’
30
+ <a href="#-what-is-brainapi">What it is</a> β€’
31
+ <a href="#️-how-it-works-the-agent-swarm">How it works</a> β€’
32
+ <a href="#-where-you-can-use-it">Use cases</a> β€’
33
+ <a href="https://brainapi.lumen-labs.ai/docs/v2">Docs β†—</a>
34
+ </p>
35
+
36
+ <a href="https://youtu.be/ECOleTRjl64?si=fBUALoYvsiUl-BPC">
37
+ <img src=".github/assets/screen-video.png" width="100%" alt="Watch the demo" />
38
+ </a>
39
+
40
+ <p align="center">
41
+ <img src="https://pbs.twimg.com/media/G6jXa2HXsAAJhDO?format=jpg&name=4096x4096" alt="BrainAPI Concept Art" width="1000" style="border-radius:4px;box-shadow:0 2px 12px rgba(0,0,0,0.09);margin-bottom:16px;"/>
42
+ </p>
43
+
44
+ ---
45
+
46
+ ## πŸ“– What is BrainAPI?
47
+
48
+ **BrainAPI reads your text and builds a knowledge graph out of it β€” by itself.**
49
+
50
+ You feed it documents, notes, messages or events. Behind the scenes a group of AI agents reads everything, understands what happened, writes down the facts, and connects them into a queryable graph. You then ask questions in plain language and get answers grounded in the actual connections it found.
51
+
52
+ The trick is that BrainAPI is **event-centric**. Instead of just storing "A is related to B," it captures _who did what, to whom, when, and in what context_. That's what lets it answer multi-step questions and always show you the path it used to get there.
53
+
54
+ Feed it one sentence:
55
+
56
+ ```
57
+ "Emily organized the AI Ethics Meetup in London on March 8, 2024."
58
+ ```
59
+
60
+ Ask something it was never explicitly told:
61
+
62
+ ```python
63
+ result = client.retrieveContext("Who organized AI events in London in Q1 2024?")
64
+ # β†’ "Emily organized the AI Ethics Meetup in London on 2024-03-08."
65
+ # β†’ result.triples shows the full graph path used to derive this
66
+ ```
67
+
68
+ That trace is the difference. Not a nearest-neighbour guess β€” a **reasoned, walkable path through a graph it built for you.**
69
+
70
+ > **[β–Ά Watch the full demo β†’](https://youtu.be/ECOleTRjl64?si=fBUALoYvsiUl-BPC)**
71
+
72
+ ---
73
+
74
+ ## πŸƒ Quickstart (fastest way to run it)
75
+
76
+ The quickest way to get BrainAPI running is the **`brainapi` TUI** β€” a CLI that clones the project, sets up a Python environment, configures services, starts backing services, and launches everything for you. Run the interactive wizard, or pass flags to skip prompts (hybrid / non-interactive).
77
+
78
+ ```sh
79
+ npm install -g brainapi-tui
80
+ brainapi init # clone, install deps, setup wizard (or pass flags to skip prompts)
81
+ brainapi start # backing services + API + MCP + worker + console
82
+ ```
83
+
84
+ When it's up:
85
+
86
+ - **API** β†’ `http://localhost:8000`
87
+ - **Console (web UI)** β†’ `http://localhost:8000/console`
88
+ - **MCP server** β†’ `http://localhost:8001/mcp`
89
+
90
+ Set `PUBLIC_BASE_URL` to the externally visible, scheme-qualified deployment URL
91
+ (without a trailing slash) when BrainAPI is served behind a proxy. BrainAPI uses it
92
+ to publish stable workspace API and Console URLs such as
93
+ `$PUBLIC_BASE_URL/brains/company/api` and
94
+ `$PUBLIC_BASE_URL/console/w/company/`. If it is unset, discovery responses use the
95
+ request base URL.
96
+
97
+ ### Workspace API base
98
+
99
+ Every active workspace publishes a stable API base on the same BrainAPI
100
+ deployment:
101
+
102
+ ```text
103
+ https://HOST/brains/aux-research/api
104
+ ```
105
+
106
+ Authenticate with either `Authorization: Bearer $BRAIN_PAT` or the `BrainPAT`
107
+ header. The workspace path selects the brain, so `X-Brain-ID` is not required.
108
+ If a caller supplies `X-Brain-ID` or a query/body `brain_id` that conflicts with
109
+ the workspace path, BrainAPI returns `409 BRAIN_SCOPE_CONFLICT`. Existing clients
110
+ using legacy endpoints plus `X-Brain-ID` remain supported.
111
+
112
+ The system PAT may access every active workspace. A per-brain PAT may access only
113
+ the workspace backed by that brain; using it with another workspace returns
114
+ `403`. Unknown credentials return `401`, unknown workspace slugs return `404`,
115
+ and archived workspaces return `410`. These rules apply consistently to the
116
+ native API, Agent API, and workspace MCP endpoint.
117
+
118
+ Useful discovery URLs are:
119
+
120
+ ```text
121
+ GET /brains/aux-research/api
122
+ GET /brains/aux-research/api/openapi.json
123
+ ```
124
+
125
+ The workspace facade includes ingest, retrieval/data, graph model, task, and
126
+ brain-specific metadata routes. System management routes are never mirrored.
127
+
128
+ | Hosted route family | Methods | Scope/behavior |
129
+ | --- | --- | --- |
130
+ | `/ingest/`, `/ingest/structured`, `/ingest/file` | `POST` | Queues the existing ingestion workers with the resolved workspace `brain_id`. |
131
+ | `/retrieve/*` | `GET`, `POST` | Reuses the existing context, search, graph, observation, data, recommendation, and vector handlers. |
132
+ | `/model/entity`, `/model/relationship` | `POST`, `PUT` | Reuses the existing graph mutation handlers. |
133
+ | `/tasks/`, `/tasks/{task_id}` | `GET` | Reads only the task namespace for the resolved workspace; foreign task IDs return `404`. |
134
+ | `/meta/relationships-properties`, `/meta/entity-labels`, `/meta/entity-properties` | `GET` | Exposes brain-specific metadata only. `/meta/login-info` is intentionally excluded. |
135
+
136
+ All paths in this table are relative to
137
+ `/brains/{workspace_slug}/api`. Management (`/system/*`), health, demo, and
138
+ plugin routes are not implicitly exposed. The discovery endpoint describes the
139
+ active workspace, while its OpenAPI document contains only this allowlisted
140
+ facade.
141
+
142
+ ### Using a Brain Workspace from an Agent
143
+
144
+ An external agent only needs a deployment URL, workspace slug, and existing
145
+ BrainPAT:
146
+
147
+ ```ini
148
+ BRAINAPI_BASE_URL=https://HOST
149
+ BRAINAPI_WORKSPACE=aux-research
150
+ BRAINAPI_PAT=...
151
+ ```
152
+
153
+ These values derive three stable surfaces:
154
+
155
+ ```text
156
+ Native API: https://HOST/brains/aux-research/api
157
+ Agent API: https://HOST/brains/aux-research/agent
158
+ MCP: https://HOST/brains/aux-research/mcp
159
+ ```
160
+
161
+ Retrieve compact agent context without knowing BrainAPI's internal route tree:
162
+
163
+ ```bash
164
+ curl \
165
+ -H "Authorization: Bearer $BRAINAPI_PAT" \
166
+ -H "Content-Type: application/json" \
167
+ https://HOST/brains/aux-research/agent/context \
168
+ -d '{"query":"What do we know about X?"}'
169
+ ```
170
+
171
+ Queue a memory write through the existing asynchronous ingestion pipeline:
172
+
173
+ ```bash
174
+ curl \
175
+ -H "Authorization: Bearer $BRAINAPI_PAT" \
176
+ -H "Content-Type: application/json" \
177
+ -H "Idempotency-Key: hermes-session-42-message-7" \
178
+ https://HOST/brains/aux-research/agent/memory \
179
+ -d '{"text":"The customer prefers annual billing.","metadata":{"source":"hermes","session_id":"42"}}'
180
+ ```
181
+
182
+ Agent memory metadata is forwarded through the ingestion model's existing
183
+ `meta_keys` support. Arbitrary metadata is retained as source metadata where the
184
+ configured ingestion backend already supports it; no separate metadata store is
185
+ created. Specifically, standard text ingestion persists it on the source
186
+ `TextChunk`, copies it into the data-vector metadata, and attaches it to generated
187
+ observations. Stored chunks can be filtered through the existing
188
+ `/retrieve/text-chunks?metadata_eq=...` API. `Idempotency-Key` stabilizes the
189
+ returned task identifier, but it is not a distributed duplicate-content
190
+ guaranteeβ€”retries after a successfully queued request may still process the
191
+ content again.
192
+
193
+ Agent memory writes are asynchronous: a successful request returns `202`,
194
+ `status: "queued"`, and the actual ingestion `task_id`. `Task-Identifier` and
195
+ `Idempotency-Key` use their value as that task identifier, so repeated requests
196
+ with the same key return the same identifier; they do not provide an exactly-once
197
+ persistence guarantee.
198
+
199
+ The agent schema is available at
200
+ `/brains/aux-research/agent/openapi.json`, and machine-readable capabilities at
201
+ `/brains/aux-research/agent/capabilities`.
202
+
203
+ BrainAPI's existing stateless Streamable HTTP MCP server remains one process on
204
+ the MCP port. The bundled reverse proxy maps the workspace-aware transport at
205
+ `https://HOST/brains/aux-research/mcp` to that server. The tools are
206
+ `memory_search`, `memory_context`, `memory_store`, `memory_observe`,
207
+ `memory_neighbors`, `memory_entity`, and `memory_task_status`; workspace scope is
208
+ derived from the URL and is not a tool argument. Direct development connections
209
+ without the reverse proxy use the MCP server port with the same path, for example
210
+ `http://localhost:8001/brains/aux-research/mcp`.
211
+
212
+ The workspace endpoint uses a semantic-only MCP registry. Legacy raw MCP tools
213
+ remain available at `/mcp` for backward compatibility, but they are not listed
214
+ or callable through `/brains/{workspace_slug}/mcp`. Plugins likewise appear in
215
+ the workspace registry only when they explicitly use the workspace-safe MCP
216
+ registration contract.
217
+
218
+ Log in to the console with the `BRAINPAT_TOKEN` generated during setup. That's it β€” you have a working brain.
219
+
220
+ ### TUI commands
221
+
222
+ | Command | What it does |
223
+ | ----------------- | ------------------------------------------------------------------------------------------- |
224
+ | `brainapi init` | Clone into `~/.brainapi/source/`, create a venv, run setup (flags skip individual prompts) |
225
+ | `brainapi start` | Bring up backing services and run the API, MCP server, and Celery worker (Ctrl-C stops all) |
226
+ | `brainapi config` | Re-open the setup wizard to change one area (databases, models, pipeline, plugins, …) |
227
+ | `brainapi doctor` | Check Python, Docker, Ollama, cloud credentials, and configured services |
228
+ | `brainapi update` | `git pull` the install and reinstall Python dependencies |
229
+
230
+ Running any command (except `help`) before `init` will run `init` first automatically.
231
+
232
+ **Useful `brainapi init` flags:** `--defaults`, `--vector-db`, `--models-mode`, provider keys/models, `--brainpat-token`, `--no-plugins`, `--no-start-services` (any passed value skips that wizard prompt). **`brainapi start` flags:** `--pipeline accurate|lightweight`, `--no-services`, `--no-api` / `--no-mcp` / `--no-worker`, `--only api,mcp,worker`. Full reference β†’ [`tui/README.md`](tui/README.md).
233
+
234
+ > The TUI installs into `~/.brainapi/` (source, venv, `.env`, and `state.json`) β€” separate from any git clone. Override with `BRAINAPI_HOME`, `BRAINAPI_REPO_URL`, or `BRAINAPI_BRANCH`.
235
+
236
+ ---
237
+
238
+ ## πŸ› οΈ Run it locally (dev mode)
239
+
240
+ If you're contributing or hacking on BrainAPI, run it straight from the repo with `make`:
241
+
242
+ ```sh
243
+ git clone https://github.com/lumen-labs/brainapi2.git && cd brainapi2
244
+ poetry install
245
+ make start-all # backing services + API (:8000) + MCP (:8001) + Celery worker
246
+ ```
247
+
248
+ `make start-all` runs everything from source with hot reload enabled, so code changes are picked up live.
249
+
250
+ **Console in dev mode:**
251
+
252
+ ```sh
253
+ make build-console && make start-api # serve console at http://localhost:8000/console
254
+ # β€” or hot-reload the console UI separately β€”
255
+ make start-api # terminal 1
256
+ make start-console-dev # terminal 2 β†’ http://localhost:5173/console/
257
+ ```
258
+
259
+ **Other handy targets:** `make start-api`, `make start-mcp`, `make stop-all`, and `make install-extras` (auto-installs heavy ML extras like local OCR based on your `.env`).
260
+
261
+ **Docker only** (pick a [deploy profile](deploy/README.md)):
262
+
263
+ ```sh
264
+ # light β€” Postgres + DeepSeek + OpenAI embeddings
265
+ cp deploy/env.light.example .env # fill in secrets, then:
266
+ docker compose -f deploy/docker-compose.light.yaml up -d
267
+
268
+ # heavy β€” Neo4j + Milvus + Mongo + GCP/Azure (full resources)
269
+ cp deploy/env.heavy.example .env
270
+ docker compose -f deploy/docker-compose.heavy.yaml up -d
271
+ ```
272
+
273
+ Then ingest your first data point:
274
+
275
+ ```sh
276
+ curl -X POST http://localhost:8000/ingest/ \
277
+ -H "Authorization: Bearer $BRAINPAT_TOKEN" \
278
+ -H "Content-Type: application/json" \
279
+ -d '{"data": {"data_type": "text", "text_data": "Emily organized the AI Ethics Meetup in London on 2024-03-08."}, "brain_id": "default"}'
280
+ ```
281
+
282
+ ---
283
+
284
+ ## πŸ–₯️ The Console
285
+
286
+ The Console is a web UI for exploring everything inside your brain: the **graph**, **text chunks**, **vectors**, **observations**, **tasks**, and an **ingest** page to push new data. It's served on the same port as the API at `http://localhost:8000/console` and you log in with your `BRAINPAT_TOKEN`.
287
+
288
+ <p align="center">
289
+ <img src=".github/assets/console-sh.png" width="100%" alt="BrainAPI Console (screenshot coming soon)" />
290
+ </p>
291
+
292
+ > Set `CONSOLE_ENABLED=false` to disable static serving in environments where you don't want the UI exposed.
293
+
294
+ ---
295
+
296
+ ## βš™οΈ How It Works: The Agent Swarm
297
+
298
+ Every piece of text you send is read and discussed by a **swarm of specialized agents**. After they understand it, they write down notes and draw the graph. Each agent has one narrow job, which keeps the pipeline reliable and auditable end to end.
299
+
300
+ ```mermaid
301
+ flowchart TD
302
+ Text["Raw text"] --> Scout["πŸ” Scout<br/>extract ALL entities"]
303
+ Text --> Observer["πŸ“ Observations Agent<br/>write notes about facts<br/>using prior context"]
304
+ Scout -->|entities| Architect["πŸ›οΈ Architect<br/>build event-centric<br/>relationships"]
305
+ Architect -->|relationships + entities| Janitor["🧹 Janitor<br/>verify correctness<br/>+ check for duplicates"]
306
+ Janitor -->|"errors + instructions"| Architect
307
+ Janitor -->|"OK"| Graph["πŸ“Š Knowledge Graph<br/>(append-only)"]
308
+ Observer --> Graph
309
+ ```
310
+
311
+ 1. **πŸ” Scout** reads the text and extracts **all** the entities it contains β€” people, things, events, quantities.
312
+ 2. **πŸ“ Observations Agent** runs alongside and writes down notes about the facts in the text, taking the previously known context into account.
313
+ 3. **πŸ›οΈ Architect** takes the Scout's entities, re-reads the text, and creates the connections between them as **event-centric relationships** β€” `subject β€”relβ†’ event β€”relβ†’ object`. Actions become first-class event hubs rather than flat edges.
314
+ 4. **🧹 Janitor** checks the Architect's work: are the relationships correct, and do similar nodes (the same entity) or triplets (the same fact) already exist in the graph? If something is wrong, the Janitor tells the Architect exactly what to fix and expects a corrected output, then checks again β€” looping until it's clean.
315
+
316
+ ### The Triangle of Attribution
317
+
318
+ Every action is modeled as a central **Event Hub** connecting three points, which is what makes multi-hop, traceable answers possible:
319
+
320
+ ```
321
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
322
+ β”‚ EVENT HUB β”‚
323
+ β”‚ (Action Node) β”‚
324
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜
325
+ β”‚
326
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
327
+ β–Ό β–Ό β–Ό
328
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
329
+ β”‚ ACTOR β”‚ β”‚ TARGET β”‚ β”‚ CONTEXT β”‚
330
+ β”‚ (Source) β”‚ β”‚ (Recipient) β”‚ β”‚ (Anchor) β”‚
331
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
332
+ :MADE :TARGETED :OCCURRED_WITHIN
333
+ ```
334
+
335
+ So `"Emily organized the AI Ethics Meetup in London"` becomes:
336
+
337
+ ```
338
+ (Emily)-[:MADE]->(Organizing Event)-[:TARGETED]->(AI Ethics Meetup)
339
+ \-[:OCCURRED_WITHIN]->(London)
340
+ ```
341
+
342
+ ### Append-only by design
343
+
344
+ BrainAPI follows an **append-only philosophy**: nothing is deleted from the graph. Every new action is appended as its own event node rather than overwriting what was already there. This keeps the full history intact and lets you **reconstruct how facts evolved over time** β€” events are never merged away, so the provenance trail always survives.
345
+
346
+ ### Two pipeline modes
347
+
348
+ - **`accurate`** (default) β€” the full swarm runs: Scout extracts every entity, the Observations agent writes notes, and the Janitor validates and deduplicates the Architect's relationships.
349
+ - **`lightweight`** β€” a faster, cheaper path that extracts only the most important entities and skips the heavier validation steps. Switch with `--pipeline lightweight` or `PIPELINE_MODE` in `.env`.
350
+
351
+ Search (`GET|POST /retrieve/search`) is **off by default**. Set `SEARCH_ENABLED=true` to register the hit-list API. BM25 requires `DATA_DB=postgresql`; dense-only search (`SEARCH_USE_BM25=false`) is supported by either production profile. PostgreSQL writes BM25 `tsvector` indexes and uses halfvec HNSW when embeddings are >2000-d. Default light-profile retrieval is both channels fused (`SEARCH_FUSION=rrf`). The search p50 SLO is **< 200 ms excluding `embed.query`** β€” ask for `profile_stages` to split embed RTT from retrieve. `/retrieve/context` stays dense βˆͺ ILIKE unless search is on and `CONTEXT_PASSAGE_MODE` is `hybrid` / `bm25` / `dense`.
352
+
353
+ ---
354
+
355
+ ## πŸ”Ž Retrieving Knowledge
356
+
357
+ Once your data is in, BrainAPI exposes purpose-built retrieval endpoints (REST, on port `8000`):
358
+
359
+ | Endpoint | Method | What you get |
360
+ | ---------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------- |
361
+ | `/retrieve/context` | `POST` | **Relevant information for a piece of text** β€” one-shot graph triples, passages, historical context, and optional provenance; p50 target < 1000 ms |
362
+ | `/retrieve/search` | `GET`/`POST` | **Ranked search hits** (opt-in: `SEARCH_ENABLED=true`) β€” passages (BM25 βˆͺ dense) by default; optional `entities` / `events` / `communities` channels and `expand=neighbors`; p50 < 200 ms excluding embed |
363
+ | `/retrieve/entity/status` | `GET` | **Existence check** for a specific entity β€” returns whether it exists, its node, relationships, and observations |
364
+ | `/retrieve/entity/synergies` | `GET` | **Sibling entities (Preview)** β€” shared event hubs + embedding similarity (`top_k`, `labels`, `polarity`); quality thresholds are not yet release-stable |
365
+ | `/retrieve/recommend` | `GET`/`POST` | **Train-free recommendations (Preview)** β€” synergies plus asymmetric walks, multi-interest, and optional attribute preferences; endpoint compatibility is retained while the quality gate is defined |
366
+
367
+ **Get context for a question:**
368
+
369
+ ```sh
370
+ curl -X POST http://localhost:8000/retrieve/context \
371
+ -H "Authorization: Bearer $BRAINPAT_TOKEN" \
372
+ -H "Content-Type: application/json" \
373
+ -d '{"text": "Who organized AI events in London in March 2024?", "brain_id": "default"}'
374
+ # β†’ text_context + triples + historical_context + source_passages
375
+ # (optional: graph_session_ids, topics, paths; tune with max_facts / max_passages)
376
+ ```
377
+
378
+ **Check if an entity exists:**
379
+
380
+ ```sh
381
+ curl "http://localhost:8000/retrieve/entity/status?target=Emily" \
382
+ -H "Authorization: Bearer $BRAINPAT_TOKEN"
383
+ # β†’ { "exists": true, "node": {...}, "relationships": [...], "observations": [...] }
384
+ ```
385
+
386
+ **Find synergies (same-type siblings, Preview):**
387
+
388
+ ```sh
389
+ curl "http://localhost:8000/retrieve/entity/synergies?target=Neural%20Networks%20101&top_k=20" \
390
+ -H "Authorization: Bearer $BRAINPAT_TOKEN"
391
+ # β†’ similar entities ranked by association_score
392
+ ```
393
+
394
+ **Rank recommendations (product / next-item style):**
395
+
396
+ ```sh
397
+ curl "http://localhost:8000/retrieve/recommend?target=u01&top_k=20&labels=PRODUCT&exclude_seen=true" \
398
+ -H "Authorization: Bearer $BRAINPAT_TOKEN"
399
+ # β†’ ranked recommendations with channel provenance (synergy / asymmetric / multi_interest / …)
400
+ ```
401
+
402
+
403
+ These endpoints cover the most common needs, but there are more (`/retrieve/hops`, `/retrieve/entities/neighbors`, `/retrieve/text-chunks`, …). Start with [Context retrieval](https://brainapi.lumen-labs.ai/docs/v2/retrieval/context) and the [docs hub](https://brainapi.lumen-labs.ai/docs/v2).
404
+
405
+ ---
406
+
407
+ ## 🧩 Plugins: Make BrainAPI Fit Your Use Case
408
+
409
+ Out of the box BrainAPI is a general-purpose understanding engine. **Plugins** let you reshape it into something far more specific β€” without forking the core. Because plugins can add routes, swap agent prompts, and register MCP tools, the same engine can become:
410
+
411
+ - **A cheap semantic search engine** β€” extract entities once, then serve fast graph-backed search instead of paying for vector queries every time.
412
+ - **A recommendation system** β€” use `/retrieve/recommend` (and `/retrieve/entity/synergies` for same-type siblings) or plugins like `features-rec` / `recsys-gnn`.
413
+ - **A domain-specific extractor** β€” override the Scout/Architect prompts so the swarm understands your ontology (CRM, legal, medical, …).
414
+ - **An agent toolset** β€” register custom MCP tools so any LLM runtime can read and write your brain.
415
+
416
+ A plugin can:
417
+
418
+ - Add custom REST routes to the BrainAPI server
419
+ - Override or extend the agents' prompts for domain-specific extraction
420
+ - Register new MCP tools for agent runtimes
421
+ - Ship its own background (Celery) tasks
422
+
423
+ ### Install & publish
424
+
425
+ ```sh
426
+ poetry run brainapi plugins install @community/crm-entities
427
+ poetry run brainapi plugins list --remote
428
+
429
+ poetry run brainapi plugins register # become a publisher
430
+ poetry run brainapi plugins publish ./my-plugin.tar.gz
431
+ ```
432
+
433
+ Browse and publish at the [Plugin Registry β†’](https://registry.brain-api.dev/app)
434
+
435
+ ---
436
+
437
+ ## πŸš€ Where You Can Use It
438
+
439
+ BrainAPI shines wherever messy, ever-changing text needs to become structured, queryable understanding:
440
+
441
+ - **AI memory for agents** β€” persistent, structured memory across sessions and documents, with provenance.
442
+ - **Recommendation systems** β€” grounded in real behavioural paths, not click co-occurrence.
443
+ - **Semantic search** β€” retrieve by meaning and multi-hop reasoning across docs, tickets and chat.
444
+ - **Expert & community mapping** β€” "Who in our org has worked on Kubernetes and ML pipelines together?"
445
+ - **Business intelligence from qualitative data** β€” turn feedback and support conversations into queryable signals.
446
+ - **Investigation & compliance** β€” connect people, events, documents and timestamps into a coherent, traceable graph.
447
+
448
+ βœ… **Reach for BrainAPI when** you need structured knowledge from messy sources, long-term explainable memory, or multi-hop queries with a traceable answer.
449
+
450
+ ⚠️ **Skip it when** your data already fits a fixed schema you control, or a single SQL query solves the problem.
451
+
452
+ ---
453
+
454
+ ## πŸ”Œ Connecting Claude Desktop via MCP
455
+
456
+ The MCP server runs as a separate process on port **8001** (`http://localhost:8001/mcp`).
457
+
458
+ 1. Start it: `make start-mcp` (or it's already running via `brainapi start`).
459
+ 2. In Claude Desktop β†’ **Settings β†’ Developer β†’ Edit Config**, add:
460
+
461
+ ```json
462
+ "brainapi-local": {
463
+ "command": "/path/to/node/bin/npx",
464
+ "args": ["-y", "@pyroprompts/mcp-stdio-to-streamable-http-adapter"],
465
+ "env": {
466
+ "URI": "http://localhost:8001/mcp",
467
+ "MCP_NAME": "brainapi-local",
468
+ "PATH": "/path/to/node/bin:/usr/local/bin:/usr/bin:/bin",
469
+ "BEARER_TOKEN": "your-pat-here"
470
+ }
471
+ }
472
+ ```
473
+
474
+ 3. Make sure URL-based MCP servers are enabled in Claude Desktop settings.
475
+
476
+ ---
477
+
478
+ ## πŸ“¦ SDKs
479
+
480
+ | Platform | Package | Status |
481
+ | ----------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
482
+ | **Python** | [`lumen_brain`](https://pypi.org/project/lumen_brain/) | ![PyPI](https://img.shields.io/pypi/v/lumen_brain?label=v0.x&color=orange) ⚠️ Pre-release |
483
+ | **Node.js** | [`lumen-brain`](https://www.npmjs.com/package/@lumenlabs/lumen-brain) | ![npm](https://img.shields.io/npm/v/lumen-brain?label=v0.x&color=orange) ⚠️ Pre-release |
484
+
485
+ ```python
486
+ from lumen_brain import BrainAPI
487
+
488
+ client = BrainAPI(api_key="your-key")
489
+ client.ingest.text("Emily organized the AI Ethics Meetup in London on 2024-03-08.")
490
+
491
+ result = client.retrieveContext("Who organized AI events in London in March 2024?")
492
+ print(result.text_context) # the answer
493
+ print(result.triples) # the graph path used to derive it
494
+ ```
495
+
496
+ > Both SDKs are pre-1.0 and under active development. For production, use the [REST API](https://brainapi.lumen-labs.ai/docs/v2) directly until v1.0 ships. You can mix modes freely β€” ingest over REST, retrieve via MCP inside an agent runtime, or use the SDKs for everything.
497
+
498
+ ---
499
+
500
+ <p align="center">
501
+ <a href="https://www.youtube.com/watch?v=wWwTFU5-qeA">
502
+ <img src="https://img.youtube.com/vi/wWwTFU5-qeA/0.jpg" alt="BrainAPI Overview Video" width="100%" style="aspect-ratio: 16/9;object-fit: cover;border-radius:4px;box-shadow:0 2px 12px rgba(0,0,0,0.09);margin-bottom:16px;"/>
503
+ <br/>
504
+ <strong>β–Ά Watch: BrainAPI (non-technical) Overview Video</strong>
505
+ </a>
506
+ </p>
507
+
508
+ ---
509
+
510
+ ## πŸ“š Resources
511
+
512
+ | Resource | Link |
513
+ | ---------------------- | ------------------------------------------------------------------------------------------------ |
514
+ | πŸ–₯️ Local CLI (TUI) | [`tui/README.md`](tui/README.md) β€” `npm install -g brainapi-tui` |
515
+ | πŸ“– Documentation | [brainapi.lumen-labs.ai/docs/v2](https://brainapi.lumen-labs.ai/docs/v2) |
516
+ | ⚑ Quick Start Guide | [brainapi.lumen-labs.ai/docs/v2](https://brainapi.lumen-labs.ai/docs/v2) |
517
+ | πŸ”Œ Plugin Registry | [registry.brain-api.dev/app](https://registry.brain-api.dev/app) |
518
+ | πŸ› οΈ Context API | [brainapi.lumen-labs.ai/docs/v2/retrieval/context](https://brainapi.lumen-labs.ai/docs/v2/retrieval/context) |
519
+ | 🐍 Python SDK (PyPI) | [pypi.org/project/lumen_brain](https://pypi.org/project/lumen_brain/) |
520
+ | πŸ“¦ Node.js SDK (npm) | [npmjs.com/package/@lumenlabs/lumen-brain](https://www.npmjs.com/package/@lumenlabs/lumen-brain) |
521
+ | πŸ’¬ Community & Support | [Discord](https://discord.gg/VTngQTaeDf) |
522
+
523
+ ---
524
+
525
+ ## 🀝 Contributing
526
+
527
+ 1. Fork the repository
528
+ 2. Create a feature branch (`git checkout -b feature/amazing-feature`)
529
+ 3. Commit your changes (`git commit -m 'Add amazing feature'`)
530
+ 4. Push to the branch (`git push origin feature/amazing-feature`)
531
+ 5. Open a Pull Request
532
+
533
+ ---
534
+
535
+ ## πŸ“„ License
536
+
537
+ This project is licensed under the **[Business Source License 1.1](LICENSE)**.
538
+
539
+ - **Free for non-production use** β€” copy, modify, redistribute, and run BrainAPI for development, evaluation, and other non-production purposes.
540
+ - **Production use requires a commercial license** β€” including self-host, sidecar, and running BrainAPI on your own cloud as part of a live product or service.
541
+ - **This version becomes Apache 2.0 on 13 August 2030.** After that date, this release is under the [Apache License, Version 2.0](licenses/Apache-2.0.txt) and production use of *this* version is free. Later releases keep their own BSL terms until their Change Date.
542
+
543
+ Contact [hello@lumen-labs.ai](mailto:hello@lumen-labs.ai) for a commercial license.
544
+
545
+ ---
546
+
547
+ <p align="center">
548
+ <sub>Built with ❀️ by <a href="https://github.com/lumen-labs">Lumen Labs</a></sub>
549
+ </p>