mxkuzn commited on
Commit
41c7f6c
·
verified ·
1 Parent(s): 05b8a9f

solvi 1.0.0

Browse files
Files changed (4) hide show
  1. README.md +3 -2
  2. app.js +2 -2
  3. solvi_docs.py +1 -1
  4. tests/make_ref.py +1 -1
README.md CHANGED
@@ -27,6 +27,7 @@ Typed answers from documents with [solvi](https://github.com/solvi-ai/solvi), co
27
  - **Fields are defined by a plain-English description.** No training, no labels. Add a field and it is extracted at once.
28
  - **Every answer cites the exact span**: the field is highlighted in the document at its character offsets.
29
  - **Rules and hard checks decide**, in plain Python you can edit on the page.
 
30
  - **The system abstains instead of guessing**: missing field, failed parse or an answer outside the options gives no answer.
31
  - **The trace re-verifies**: every run writes a hash-chained trace and replays it; "Tamper with a copy" shows the replay
32
  catching an edited fact.
@@ -46,13 +47,13 @@ A use case is one file in `usecases/` (fields as `[name, description]`, document
46
  1. **Tokenizer** (`tokenizer.js`): ModernBERT byte-level BPE reimplemented in JavaScript so every token has character
47
  offsets (Python code-point offsets for solvi quotes, UTF-16 offsets for highlighting). Identical ids and offsets to
48
  Hugging Face `tokenizers` on all sample documents and descriptions.
49
- 2. **Extractor** (`extractor.worker.js`): reproduces `solvi.extract_long.LongSpanExtractor.predict`: windows
50
  `[CLS] description [SEP] chunk [SEP]` (1024 tokens, stride 128), softmax of start/end logits per window, best span
51
  `start ≤ end < start + 256` by `p_start · p_end`, best over windows, present if the score reaches the threshold from
52
  `solvi_extract.json`. The model file (790 MB) is cached with the Cache API. WebGPU is used when the GPU supports
53
  16-bit float shaders (`shader-f16`); otherwise the model runs on the CPU (multi-threaded when the page is
54
  cross-origin isolated, which the headers above enable).
55
- 3. **Decisions** (`solvi_docs.py` in Pyodide, `solvi` from PyPI): each field becomes an `@cat.extract` part returning a
56
  `Quote(value, start, end, confidence)`; the use case's rule code adds `@cat.fn`, `@cat.check` and `@cat.rule` parts;
57
  `System(cat, QUESTIONS).ask({"doc": text, "today": ...})` answers, and the trace is replayed.
58
 
 
27
  - **Fields are defined by a plain-English description.** No training, no labels. Add a field and it is extracted at once.
28
  - **Every answer cites the exact span**: the field is highlighted in the document at its character offsets.
29
  - **Rules and hard checks decide**, in plain Python you can edit on the page.
30
+ - **Built on solvi 1.0**: the page installs `solvi==1.0.0` from PyPI into its Python.
31
  - **The system abstains instead of guessing**: missing field, failed parse or an answer outside the options gives no answer.
32
  - **The trace re-verifies**: every run writes a hash-chained trace and replays it; "Tamper with a copy" shows the replay
33
  catching an edited fact.
 
47
  1. **Tokenizer** (`tokenizer.js`): ModernBERT byte-level BPE reimplemented in JavaScript so every token has character
48
  offsets (Python code-point offsets for solvi quotes, UTF-16 offsets for highlighting). Identical ids and offsets to
49
  Hugging Face `tokenizers` on all sample documents and descriptions.
50
+ 2. **Extractor** (`extractor.worker.js`): reproduces `solvi.core.extract.LongSpanExtractor.predict`: windows
51
  `[CLS] description [SEP] chunk [SEP]` (1024 tokens, stride 128), softmax of start/end logits per window, best span
52
  `start ≤ end < start + 256` by `p_start · p_end`, best over windows, present if the score reaches the threshold from
53
  `solvi_extract.json`. The model file (790 MB) is cached with the Cache API. WebGPU is used when the GPU supports
54
  16-bit float shaders (`shader-f16`); otherwise the model runs on the CPU (multi-threaded when the page is
55
  cross-origin isolated, which the headers above enable).
56
+ 3. **Decisions** (`solvi_docs.py` in Pyodide, `solvi==1.0.0` from PyPI, pinned twice in `app.js`): each field becomes an `@cat.extract` part returning a
57
  `Quote(value, start, end, confidence)`; the use case's rule code adds `@cat.fn`, `@cat.check` and `@cat.rule` parts;
58
  `System(cat, QUESTIONS).ask({"doc": text, "today": ...})` answers, and the trace is replayed.
59
 
app.js CHANGED
@@ -222,10 +222,10 @@ async function loadPython() {
222
  $("py-line").textContent = "Python (Pyodide) + solvi: installing numpy and solvi…";
223
  await py.loadPackage(["numpy", "micropip"]);
224
  try { // solvi's own code needs only numpy at import time: skip scipy (another ~10 MB)
225
- await py.runPythonAsync("import micropip\nawait micropip.install('solvi==0.8.0', deps=False)\nimport solvi");
226
  } catch (e) {
227
  console.warn("solvi without dependencies failed, installing with dependencies", e);
228
- await py.runPythonAsync("import micropip\nawait micropip.install('solvi==0.8.0')\nimport solvi");
229
  }
230
  const src = await (await fetch(new URL("solvi_docs.py", import.meta.url))).text();
231
  py.FS.writeFile("/home/pyodide/solvi_docs.py", src);
 
222
  $("py-line").textContent = "Python (Pyodide) + solvi: installing numpy and solvi…";
223
  await py.loadPackage(["numpy", "micropip"]);
224
  try { // solvi's own code needs only numpy at import time: skip scipy (another ~10 MB)
225
+ await py.runPythonAsync("import micropip\nawait micropip.install('solvi==1.0.0', deps=False)\nimport solvi");
226
  } catch (e) {
227
  console.warn("solvi without dependencies failed, installing with dependencies", e);
228
+ await py.runPythonAsync("import micropip\nawait micropip.install('solvi==1.0.0')\nimport solvi");
229
  }
230
  const src = await (await fetch(new URL("solvi_docs.py", import.meta.url))).text();
231
  py.FS.writeFile("/home/pyodide/solvi_docs.py", src);
solvi_docs.py CHANGED
@@ -17,7 +17,7 @@ import traceback
17
  from datetime import date, datetime, timedelta
18
 
19
  from solvi import Answer, Catalog, Question, Quote, System
20
- from solvi.runtime import MISSING, vhash
21
 
22
  # ------------------------------------------------------------------------------------------------------------ helpers
23
  NUMBER_WORDS = {"zero": 0, "one": 1, "two": 2, "three": 3, "four": 4, "five": 5, "six": 6, "seven": 7, "eight": 8, "nine": 9,
 
17
  from datetime import date, datetime, timedelta
18
 
19
  from solvi import Answer, Catalog, Question, Quote, System
20
+ from solvi.core.runtime import MISSING, vhash
21
 
22
  # ------------------------------------------------------------------------------------------------------------ helpers
23
  NUMBER_WORDS = {"zero": 0, "one": 1, "two": 2, "three": 3, "four": 4, "five": 5, "six": 6, "seven": 7, "eight": 8, "nine": 9,
tests/make_ref.py CHANGED
@@ -9,7 +9,7 @@ import json
9
  import sys
10
  import time
11
 
12
- from solvi.extract_long import LongSpanExtractor
13
 
14
  import solvi_docs
15
 
 
9
  import sys
10
  import time
11
 
12
+ from solvi.core.extract import LongSpanExtractor
13
 
14
  import solvi_docs
15