Hugging Face's logo

edgeimpulse
/
edgeimpulse-docs-rag

Text Generation
Transformers
edge-impulse
rag
retrieval-augmented-generation
faiss
qwen
documentation
tinyml
edge-ai
Model card Files
xet
 Community
edgeimpulse-docs-rag
File size: 4,306 Bytes
8c14673
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
---
license: apache-2.0
base_model: edgeimpulse/edgeimpulse-docs-qwen-0.5b
pipeline_tag: text-generation
library_name: transformers
tags:
  - edge-impulse
  - rag
  - retrieval-augmented-generation
  - faiss
  - qwen
  - documentation
  - tinyml
  - edge-ai
---

# Edge Impulse Docs — RAG Assistant

A retrieval-augmented assistant for the [Edge Impulse](https://edgeimpulse.com)
documentation. It grounds every answer in a prebuilt vector index of the docs and
generates with the small quantized model
[`edgeimpulse/edgeimpulse-docs-qwen-0.5b`](https://huggingface.co/edgeimpulse/edgeimpulse-docs-qwen-0.5b),
so it runs comfortably on a laptop.

- **Retrieval:** FAISS (inner-product) over `data/index`, embedded with
  `sentence-transformers/all-MiniLM-L6-v2` (384-dim, the same model the index was
  built with).
- **Generation:** the 0.5B GGUF, served through any OpenAI-compatible endpoint
  (llama.cpp `llama-server` or Ollama). No training stack required.
- **Grounded + cited:** answers are constrained to the retrieved context and each
  response lists its source documents.

This repo ships only what you need to **run** the assistant — the prebuilt index
and the inference code. The raw document corpus and the index-building pipeline
are not included.

## Contents

| File | Purpose |
| --- | --- |
| `data/index/edge_impulse_docs.faiss` | FAISS inner-product index of the docs |
| `data/index/chunks.pkl` | Chunk text + source metadata (aligned to the index) |
| `data/index/metadata.json` | Embedding model + index parameters |
| `rag.py` | Retrieval + grounded generation (CLI + importable) |
| `serve.py` | Minimal Flask HTTP API (`POST /ask`) |
| `requirements.txt` | Runtime dependencies |

## Quickstart

**1. Install dependencies and download this repo**

```bash
pip install -r requirements.txt
hf download edgeimpulse/edgeimpulse-docs-rag --local-dir edgeimpulse-docs-rag
cd edgeimpulse-docs-rag
```

**2. Start the generator** (pick one)

llama.cpp:

```bash
hf download edgeimpulse/edgeimpulse-docs-qwen-0.5b qwen-edgeai-q4_k_m.gguf --local-dir .
llama-server -m qwen-edgeai-q4_k_m.gguf -c 4096 --port 8080 --jinja
```

Ollama:

```bash
ollama run hf.co/edgeimpulse/edgeimpulse-docs-qwen-0.5b
# then point rag.py at Ollama's OpenAI-compatible port:
export RAG_API_BASE=http://127.0.0.1:11434/v1
export RAG_MODEL=hf.co/edgeimpulse/edgeimpulse-docs-qwen-0.5b
```

**3. Ask a question**

```bash
python rag.py "How do I deploy a model to run on a Linux target as an .eim file?"
```

Only see what was retrieved (no generation):

```bash
python rag.py "How do I create an API key?" --no-generate
```

Serve it over HTTP:

```bash
python serve.py --host 0.0.0.0 --port 8000
curl -s localhost:8000/ask -H 'content-type: application/json' \
  -d '{"question": "What is the data forwarder?"}'
```

## Configuration

`rag.py` reads these environment variables (all optional):

| Variable | Default | Meaning |
| --- | --- | --- |
| `RAG_INDEX_DIR` | `data/index` | Location of the FAISS index + chunks |
| `RAG_API_BASE` | `http://127.0.0.1:8080/v1` | OpenAI-compatible generation endpoint |
| `RAG_MODEL` | `edgeimpulse/edgeimpulse-docs-qwen-0.5b` | Model name passed to the endpoint |

## How it works

```
question ──▶ MiniLM embed ──▶ FAISS top-k ──▶ context + question
                                                    │
                                                    â–¼
                             edgeimpulse-docs-qwen-0.5b (llama.cpp / Ollama)
                                                    │
                                                    â–¼
                                    grounded answer + cited sources
```

The generator is a small model, so retrieval quality matters: the assistant is
most accurate when the right chunk is retrieved, and it may be terse or repeat
itself on out-of-scope questions. Sampling defaults (`temperature 0.3`,
`repeat_penalty 1.2`) are tuned to keep it from looping.

## Related

- Generator model: [`edgeimpulse/edgeimpulse-docs-qwen-0.5b`](https://huggingface.co/edgeimpulse/edgeimpulse-docs-qwen-0.5b)
- API-scoped variant: [`edgeimpulse/edgeimpulse-api-docs-rag`](https://huggingface.co/edgeimpulse/edgeimpulse-api-docs-rag)

## License

Apache-2.0. Documentation content belongs to Edge Impulse.