join3r commited on
Commit
57a55b5
·
verified ·
1 Parent(s): dae4f76

gte-memory-20261021: skill-selection fine-tune (facts held to epoch2)

Browse files
Files changed (7) hide show
  1. NOTICE +25 -1
  2. README.md +43 -38
  3. SHA256SUMS +6 -5
  4. TRAINING-SOURCES.md +58 -0
  5. config.json +1 -1
  6. onnx/model_quantized.onnx +1 -1
  7. stem-model.json +9 -6
NOTICE CHANGED
@@ -11,7 +11,31 @@ Stem modifications (September 2026): fine-tuned encoder and classification
11
  head for personal-memory relevance using fictional profiles and Czech,
12
  Slovak, German and English queries; exported epoch2 to ONNX and dynamically
13
  quantized weights to int8; calibrated separate fact and skill thresholds.
 
 
 
 
 
14
  The supplied model configuration and tokenizer match the tested export.
15
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  This notice describes attribution and modifications; it does not imply
17
- endorsement by the upstream authors. See LICENSE for Apache License 2.0.
 
 
11
  head for personal-memory relevance using fictional profiles and Czech,
12
  Slovak, German and English queries; exported epoch2 to ONNX and dynamically
13
  quantized weights to int8; calibrated separate fact and skill thresholds.
14
+
15
+ Stem modifications (October 2026, model gte-memory-20261021): further
16
+ fine-tuned the September epoch2 weights for skill selection, holding fact
17
+ scores to the September model by distillation; exported to ONNX and
18
+ dynamically quantized weights to int8; recalibrated the skill thresholds.
19
  The supplied model configuration and tokenizer match the tested export.
20
 
21
+ The October training data included material derived from these public
22
+ datasets and repositories (used for training only; none of it is
23
+ redistributed here):
24
+ - MetaTool (HowieHwong/MetaTool), MIT
25
+ - Berkeley Function Calling Leaderboard (gorilla-llm), Apache-2.0
26
+ - xlam-irrelevance-7.5k (MadeAgents), CC-BY-4.0
27
+ - german-function-calling (flozi00), Apache-2.0
28
+ - SauerkrautLM-Fermented-Irrelevance-GER-DPO (VAGOsolutions), MIT
29
+ - Seal-Tools (casey-martin), MIT
30
+ - ToolACE (Team-ACE), Apache-2.0
31
+ - ToolRet-Queries (mangopy): ToolBench queries, Apache-2.0; API-Bank, MIT
32
+ - Public agent-skill descriptions (name and one-line description) from
33
+ 34 repositories, including anthropics/skills, openai/skills,
34
+ huggingface/skills, vercel-labs/agent-skills, obra/superpowers,
35
+ trailofbits/skills (CC-BY-SA-4.0) and GarethManning/education-agent-skills
36
+ (CC-BY-SA-4.0), under their respective licenses.
37
+ TRAINING-SOURCES.md lists every source with its license.
38
+
39
  This notice describes attribution and modifications; it does not imply
40
+ endorsement by the upstream authors or dataset publishers. See LICENSE for
41
+ Apache License 2.0.
README.md CHANGED
@@ -12,6 +12,7 @@ tags:
12
  - onnx
13
  - reranker
14
  - memory
 
15
  - stem
16
  - quantized
17
  - experimental
@@ -19,40 +20,34 @@ tags:
19
 
20
  # Stem GTE Memory
21
 
22
- A custom GTE reranker fine-tuned for Stem's personal-memory retrieval, with training queries in **Czech, Slovak, German, and English**. It scores how relevant a stored memory is to a user's message. Stem also uses the same weights for skill selection with a separately calibrated threshold.
23
 
24
- This is the selected **epoch2 q8 ONNX** release, model ID `gte-memory-20260905-epoch2`. The five inference files total **358,119,455 bytes (341.53 MiB)**; Stem's dropdown rounds this to approximately 342 MB. The quantized graph alone is 341,032,790 bytes.
25
 
26
- In Stem's fresh synthetic memory benchmark, this checkpoint improved recall from **43.1% to 64.7%** and reduced irrelevant selections from **166 to 87**, compared with the configured Qwen3 Reranker 0.6B baseline. A separate local timing experiment measured **320 ms vs 3,116 ms median** for the benchmark pipeline, approximately **9.74× faster**. These are scoped benchmark results, not universal model rankings or end-to-end chat speed guarantees.
27
 
28
  ## Intended use
29
 
30
- Rerank a shortlist of memory facts, or skill descriptions, before an assistant responds. Training and the reported multilingual evaluations used English stored facts or descriptions with multilingual user requests. Broader language coverage inherited from the base model has not been established by this fine-tune's evaluation.
31
-
32
- This is an experimental, opt-in release. Publishing the files does not enable automatic downloading in existing Stem versions; that app integration is separate.
33
 
34
  ## Files and integrity
35
 
36
  - `onnx/model_quantized.onnx`: selected q8 inference graph, ONNX opset 17.
37
- - `config.json`, `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`: the exact measured inference configuration and tokenizer.
38
- - `stem-model.json`: immutable model identity, upstream revisions, per-file SHA-256 hashes and sizes, and Stem thresholds.
39
- - `example.mjs`: local Node.js inference example using Transformers.js 3.8.1.
40
- - `LICENSE` and `NOTICE`: licensing and modification notices.
41
-
42
- Pin a Hub commit revision when downloading, then verify all five inference files against `stem-model.json`. Hashes are also pinned in Stem's runtime. No training datasets, chat transcripts, benchmark examples, FP32 weights, or training checkpoints are included.
43
 
44
- `config.json` retains an inert temporary export path in `_name_or_path` to preserve its measured hash. It does not need to exist. Its upstream `auto_map` metadata is likewise preserved; the supported ONNX path below does not execute remote Python code.
45
 
46
  ## Inference contract
47
 
48
- Use **Transformers.js 3.8.1 `PreTrainedModel`**, not its auto sequence-classification loader: this export retains the upstream custom `model_type: new`. This repository contains ONNX weights, not a PyTorch checkpoint.
49
 
50
  1. Tokenize a `(query, document)` text pair with the supplied tokenizer.
51
  2. Score **one pair per forward pass**, sequentially. Do not silently truncate. Stem rejects pairs longer than 8,192 tokens, including special tokens; long-context quality has not been validated here.
52
  3. Read the single finite **raw logit**. Higher means more relevant. Do not apply sigmoid or reinterpret it as a probability.
53
 
54
- Install `@huggingface/transformers@3.8.1` in a Node.js project, download this repository's files to a local folder, and run:
55
-
56
  ```sh
57
  node example.mjs /absolute/path/to/downloaded/model
58
  ```
@@ -62,12 +57,13 @@ For Stem's measured configuration:
62
  | Purpose | Raw-logit threshold |
63
  |---|---:|
64
  | Memory facts | -1.6370911598205566 |
65
- | Skill selection | -3.9743599891662598 |
 
66
  | Additional margin for sensitive facts | +2 |
67
 
68
- These thresholds belong to this exact graph, tokenizer, and pipeline. They are not interchangeable or universal cutoffs. Stem uses a Qwen3 Embedding 0.6B shortlist, up to 24 fact candidates and at most 16 relevant selected facts (subject to a smaller user setting). Pinned facts are handled separately.
69
 
70
- For fact retrieval with conversational context, Stem takes at most two preceding eligible user messages in chronological order, clipping each to 400 Unicode code points, and formats the query as:
71
 
72
  ```text
73
  Earlier messages from the user in this conversation:
@@ -77,41 +73,50 @@ Earlier messages from the user in this conversation:
77
  Current message: <current user message>
78
  ```
79
 
80
- With no eligible history, use the current message alone. Skill selection uses the current message alone and its own threshold.
 
 
81
 
82
  ## Training
83
 
84
  Base: [Alibaba-NLP/gte-multilingual-reranker-base](https://huggingface.co/Alibaba-NLP/gte-multilingual-reranker-base), revision `8215cf04918ba6f7b6a62bb44238ce2953d8831c`. Upstream implementation: `Alibaba-NLP/new-impl`, revision `40ced75c3017eb27626c9d4ea981bde21a2662f4`.
85
 
86
- Fine-tuning used **23,684 binary pairs from 88 fictional training profiles**, including 797 relevant pairs. Query-language pair counts: Czech 5,905; Slovak 6,051; German 5,963; English 5,765. Borderline examples were excluded. Real personal-memory benchmarks were not used for training and are not published.
87
 
88
- The encoder and classification head were trained (113,920,513 trainable parameters); token embeddings were frozen. AdamW used learning rate 1e-5, weight decay 0.01, batch size 16, 10% linear warmup, gradient clipping 1.0 and BCE positive weight 3. The maximum tokenized training pair length was 94; no training examples were truncated. Three epochs were planned; epoch2 was selected after q8 export using development data. Feasible settings had to meet the configured Qwen baseline recall overall and per language, then minimize `3 * false negatives + false positives`.
89
 
90
- Export used PyTorch 2.8.0 and ONNX Runtime 1.22.1. Dynamic weight quantization used QInt8, per-channel quantization, reduced range, symmetric weights and constant-matrix-only MatMul quantization. Four synthetic export probes had maximum FP32/PyTorch error 0.0000019074 and q8/FP32 error 0.065004; these probes do not establish broad numerical parity.
 
 
 
 
 
91
 
92
- ## Evaluation and limitations
93
 
94
- The fresh memory test contained 192 requests from 16 fictional profiles, 48 in each language. Its references were synthetic author/reviewer judgments, with disagreements treated as borderline, not human gold. Labels covered the reviewed union of candidate pools, so facts missed by both retrieval paths remained unjudged.
95
 
96
- | Request language | Configured Qwen recall | Stem GTE recall |
97
- |---|---:|---:|
98
- | Czech | 19.4% | 58.1% |
99
- | Slovak | 10.3% | 53.8% |
100
- | German | 48.7% | 64.1% |
101
- | English | 74.1% | 75.9% |
102
 
103
- Two previously seen real-data regression sets showed small recall changes (15.4% to 19.2%, and 23.5% to 25.7%) and fewer irrelevant selections. Their recall-difference intervals included zero; they do not establish a large real-world recall gain. No real-data examples are distributed.
104
 
105
- The timing result above measured 64 warm observations per model on a local Mac. Each observation embedded a contextual query, sequentially scored 24 candidates and applied the gate. Candidate retrieval, database work, IPC, model loading and answer generation were excluded. Repeated observations were correlated. This is not a promise about complete response latency, other hardware, or application-wide memory consumption.
 
 
 
 
 
106
 
107
- Linux validation found numerical differences from macOS despite identical tokenized inputs (maximum absolute logit difference 0.645472 on 16 probes). With the frozen fact threshold, Linux selected 106 useful and 84 irrelevant facts on the same fresh synthetic set, versus macOS GTE's 108 and 87. An English-only same-server rerun selected 42 useful facts with GTE versus 39 with Qwen; other languages' Qwen baselines were not rerun on Linux. Validate your target runtime; do not assume bitwise portability.
108
 
109
- Sensitive-fact selection remains a limitation: in the fresh macOS test, GTE selected 3 irrelevant sensitive facts versus Qwen's 1 while finding more useful sensitive facts. The inherited +2 margin is not equally calibrated across models.
110
 
111
- A separate fresh skill test found the correct skill on 39/48 positive requests versus Qwen's 33/48, with false loads on 11/48 negative requests versus 12/48. GTE still **missed the predeclared 90% correct-loading target** (81.25%). Both models found the correct skill on only 15/24 concise conversational positives. The weights were not trained again for this skill evaluation; only the threshold was calibrated beforehand.
112
 
113
- These experiments do not establish improved final answers, long-input accuracy, performance outside the four training languages, or universal superiority over other rerankers. The model should remain a selectable option while broader evaluation continues.
114
 
115
  ## License and attribution
116
 
117
- Released under Apache License 2.0, following the upstream model's declared license. See `LICENSE` and `NOTICE`. Stem modifications comprise memory-relevance fine-tuning, ONNX export, quantization and task-specific calibration. This release is a Stem adaptation, not an official Alibaba release.
 
12
  - onnx
13
  - reranker
14
  - memory
15
+ - skills
16
  - stem
17
  - quantized
18
  - experimental
 
20
 
21
  # Stem GTE Memory
22
 
23
+ A custom GTE reranker fine-tuned for Stem, a personal assistant. It scores how relevant a stored memory fact, or a saved skill (a step-by-step procedure), is to a user's message. Training requests are in **Czech, Slovak, German, and English**; stored facts and skill descriptions are English.
24
 
25
+ This is the **October 2026 q8 ONNX** release, model ID `gte-memory-20261021`. It continues the September release (`gte-memory-20260905-epoch2`, Hub revision `dae4f76d6974e70cdc007d3ca66d82226791822f`, still downloadable by that revision) with training for **skill selection**, while fact scores were held to the September model. The five inference files total **358,119,414 bytes (341.53 MiB)**; the quantized graph alone is 341,032,790 bytes.
26
 
27
+ On 815 real assistant turns held out from training, at a fixed operating point this release loaded the same number of needed skills as the September model with Stem's example requests (45 of 81) while loading a wrong skill on **18 turns instead of 57**. Fact selection at the unchanged fact threshold stayed within a few percent of the September model. These are scoped results from one user's assistant, not universal model rankings.
28
 
29
  ## Intended use
30
 
31
+ Rerank a shortlist of memory facts, or of skill descriptions and per-skill example requests, before an assistant responds. Fact and skill selection use separately calibrated thresholds.
 
 
32
 
33
  ## Files and integrity
34
 
35
  - `onnx/model_quantized.onnx`: selected q8 inference graph, ONNX opset 17.
36
+ - `config.json`, `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`: the exact measured inference configuration and tokenizer. The tokenizer files are byte-identical to the September release.
37
+ - `stem-model.json`: model identity, upstream and previous revisions, per-file SHA-256 hashes and sizes, and Stem thresholds.
38
+ - `example.mjs`: local Node.js inference example using Transformers.js.
39
+ - `LICENSE`, `NOTICE` and `TRAINING-SOURCES.md`: licensing, modification notices and the public training sources.
 
 
40
 
41
+ Pin a Hub commit revision when downloading, then verify all five inference files against `stem-model.json`. Hashes are also pinned in Stem's runtime. No training data, chat transcripts, benchmark examples, FP32 weights, or training checkpoints are included. `config.json` keeps its upstream `auto_map` metadata; the supported ONNX path below does not execute remote Python code.
42
 
43
  ## Inference contract
44
 
45
+ Use **Transformers.js `PreTrainedModel`**, not its auto sequence-classification loader: this export retains the upstream custom `model_type: new`. This repository contains ONNX weights, not a PyTorch checkpoint.
46
 
47
  1. Tokenize a `(query, document)` text pair with the supplied tokenizer.
48
  2. Score **one pair per forward pass**, sequentially. Do not silently truncate. Stem rejects pairs longer than 8,192 tokens, including special tokens; long-context quality has not been validated here.
49
  3. Read the single finite **raw logit**. Higher means more relevant. Do not apply sigmoid or reinterpret it as a probability.
50
 
 
 
51
  ```sh
52
  node example.mjs /absolute/path/to/downloaded/model
53
  ```
 
57
  | Purpose | Raw-logit threshold |
58
  |---|---:|
59
  | Memory facts | -1.6370911598205566 |
60
+ | Skill, best of description and its example requests | -0.25 |
61
+ | Skill without example requests (description only) | -2.4 |
62
  | Additional margin for sensitive facts | +2 |
63
 
64
+ These thresholds belong to this exact graph, tokenizer and pipeline. They are not interchangeable with the September release's skill thresholds, nor universal cutoffs.
65
 
66
+ **Facts.** Stem takes at most two preceding eligible user messages in chronological order, clipping each to 400 Unicode code points, and formats the query as:
67
 
68
  ```text
69
  Earlier messages from the user in this conversation:
 
73
  Current message: <current user message>
74
  ```
75
 
76
+ With no eligible history, use the current message alone. Candidates come from an embedding shortlist of up to 24 facts.
77
+
78
+ **Skills.** The query is the current message; when it has fewer than 8 words, the previous user message (clipped to 400 code points) is prepended with a newline. Each shortlisted skill is scored as the maximum over separate pairs: `name\ndescription` and `name\n<example request>` for each of up to four example requests that Stem writes per skill. A skill without example requests uses its description pair and the description-only threshold. Stem shortlists four skills by embedding cosine (EmbeddingGemma 2) and loads at most two.
79
 
80
  ## Training
81
 
82
  Base: [Alibaba-NLP/gte-multilingual-reranker-base](https://huggingface.co/Alibaba-NLP/gte-multilingual-reranker-base), revision `8215cf04918ba6f7b6a62bb44238ce2953d8831c`. Upstream implementation: `Alibaba-NLP/new-impl`, revision `40ced75c3017eb27626c9d4ea981bde21a2662f4`.
83
 
84
+ **September stage** (unchanged, see revision `dae4f76`): 23,684 binary fact pairs from 88 fictional training profiles, query languages balanced across Czech, Slovak, German and English.
85
 
86
+ **October stage**, starting from the September epoch2 weights. Skill items were (message, skill) pairs, each holding every document the skill is scored on (description plus example requests), labeled 1 if the message needs that skill. Candidates were each message's top 8 skills by EmbeddingGemma 2 cosine plus any needed skill. Sources:
87
 
88
+ | Source | Messages | Notes |
89
+ |---|---:|---|
90
+ | Real assistant messages, June–September 2026 | 1,469 | from the author's own use; labeled by two blind LLM labelers (96% agreement) with LLM adjudication; not published |
91
+ | Synthetic messages for the author's skills | 2,527 | Czech, Slovak, German and English; kept only where a blind labeler agreed with the writer |
92
+ | Synthetic skill libraries | about 2,700 | 29 libraries of 25 skills for varied fictional users, seven built from public agent-skill descriptions |
93
+ | Public tool-selection data | 3,734 | see TRAINING-SOURCES.md; capped at 12,000 items |
94
 
95
+ Facts were held to the September model by distillation: each step added 16 fact pairs from the September training set with the September model's own logits as targets (mean squared error, weight 20). The loss on skill items was binary cross-entropy on the **maximum** logit over the item's documents, matching how Stem scores skills, with positive weight 3. The encoder and classification head were trained and token embeddings frozen. AdamW, learning rate 1e-5, weight decay 0.01, batch size 16 skill items, 10% linear warmup, gradient clipping 1.0, one epoch (3,793 steps). Training queries longer than 320 tokens were truncated; inference does not truncate.
96
 
97
+ The checkpoint was chosen on development data only: 20% of the real conversations (held out by conversation) plus three whole held-out synthetic libraries (Czech/English, English, German). The 815-turn evaluation set below was never used for training or checkpoint selection; every conversation in it is later than, and disjoint from, the real training messages.
98
 
99
+ Export used ONNX opset 17 and the same dynamic int8 quantization settings as the September release (QInt8, per-channel, reduced range, symmetric weights, constant-matrix-only MatMul).
100
+
101
+ ## Evaluation and limitations
 
 
 
102
 
103
+ **Skills, real turns.** 815 turns from one user's assistant (27 Sep–20 Oct 2026; mostly Slovak and English, some Czech), each labeled by two blind LLM labelers with disagreements adjudicated by hand; 81 turns needed a skill that was in the library at the time. Both models used the same shortlist and the same example requests.
104
 
105
+ | | Needed skills loaded | Turns with a wrong skill loaded |
106
+ |---|---:|---:|
107
+ | September model, its example threshold (+3.0) | 45 / 81 | 57 |
108
+ | This release, example threshold −0.25 | 45 / 81 | 18 |
109
+ | Threshold chosen on half the conversations, applied to the other half (mean of 20 splits): September | 39.7 / 81 | 31.5 |
110
+ | same, this release | 45.2 / 81 | 27.7 |
111
 
112
+ A paired bootstrap over conversations put the gain at about 5 more needed skills loaded at matched wrong-load budgets (probability of no gain about 0.02). Only 60 of the 108 needed skills reached the four-skill embedding shortlist; widening it to eight did not help with either model.
113
 
114
+ **Skills, held-out libraries.** Average precision over shortlisted (message, skill) items: Czech/English 0.873 → 0.952, English 0.839 → 0.921, German 0.856 → 0.928; held-out real conversations 0.263 → 0.401.
115
 
116
+ **Facts.** At the unchanged fact threshold, cost `3 × false negatives + false positives` against the September model: development 171 vs 182, fresh synthetic 221 vs 224, real regression set 1: 162 vs 155, real regression set 2: 377 vs 375. Fact selection is not improved by this release, and set 1 is slightly worse.
117
 
118
+ Limits: the real evaluation comes from one user with one skill library, and labels are LLM judgments (human-adjudicated only where the two labelers disagreed). Thresholds were measured on macOS; the September release found Linux logits differing by up to 0.65 on probes, so validate your target runtime. Long-input accuracy, final-answer quality and languages outside the four training languages were not evaluated.
119
 
120
  ## License and attribution
121
 
122
+ Released under Apache License 2.0, following the upstream model's declared license. See `LICENSE`, `NOTICE` and `TRAINING-SOURCES.md`. Stem modifications comprise memory- and skill-relevance fine-tuning, ONNX export, quantization and task-specific calibration. This release is a Stem adaptation, not an official Alibaba release.
SHA256SUMS CHANGED
@@ -1,10 +1,11 @@
1
  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE
2
- c943475cbb6f4cf48f04e009ba003ebc63b6fc5f7a9c1329bda7ef294bc9bef2 NOTICE
3
- d1ebe6222bfaf23ac515be79aced8ca113d5d95b4bfe96d54a97645ad5d8d86a README.md
4
- e5f2efd1e04e5e4465b8c6c2f6af1f00a5d66a839e825b0af8edba68ceb8b1bb config.json
 
5
  7702e5276936c92f0283150c5032c5a74dfb5bc25e00d53b75c062a01c28f3b8 example.mjs
6
- 5cbb66c672aa71fb6609880ba8e8370ff5be3376fc95215317c92e41e7743e4e onnx/model_quantized.onnx
7
  8c785abebea9ae3257b61681b4e6fd8365ceafde980c21970d001e834cf10835 special_tokens_map.json
8
- 196a6a4a0d4973bbc39095cb8b95d519df1a093fcf84124b720ddcd8096d9b2b stem-model.json
9
  3a56def25aa40facc030ea8b0b87f3688e4b3c39eb8b45d5702b3a1300fe2a20 tokenizer.json
10
  6f00514620aff01ba8b7291b2394e98daca5be264cb743805232d9ae27494b2a tokenizer_config.json
 
1
  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE
2
+ 8d7ae557f6eb6160b731cc353d4360c27caf87cf997542a177491e9085e86ff6 NOTICE
3
+ 11ebb85e616812b3cb51de33eace89fe46f2a1e44ed9f14ce01c947a21bf7471 README.md
4
+ b325e1441ba66f39fa694a91b5cef4a24e10a0c1cfa4ddc3fe7e387e103807db TRAINING-SOURCES.md
5
+ 01a3c997ee3f9ebdf250fa01086487f336ce573a46ad24cbb0a249e22eaf98b6 config.json
6
  7702e5276936c92f0283150c5032c5a74dfb5bc25e00d53b75c062a01c28f3b8 example.mjs
7
+ 1e88d816d1cefef69a7b43df3bb2dabdb92a21b0e3ce92cd6ceb0e753fcda74f onnx/model_quantized.onnx
8
  8c785abebea9ae3257b61681b4e6fd8365ceafde980c21970d001e834cf10835 special_tokens_map.json
9
+ 814375aebb3f6660ab975de74e79aec15e3a5cf7b25d016d5329dca3ff966f26 stem-model.json
10
  3a56def25aa40facc030ea8b0b87f3688e4b3c39eb8b45d5702b3a1300fe2a20 tokenizer.json
11
  6f00514620aff01ba8b7291b2394e98daca5be264cb743805232d9ae27494b2a tokenizer_config.json
TRAINING-SOURCES.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Training sources (gte-memory-20261021)
2
+
3
+ Public material used to train this release. Nothing from these sources is redistributed in this repository. Licenses as stated by each source at collection time (2026-10-20).
4
+
5
+ ## Tool-selection datasets
6
+
7
+ | Dataset | License | Messages used |
8
+ |---|---|---:|
9
+ | [metatool](https://github.com/HowieHwong/MetaTool) | MIT | 500 |
10
+ | [bfcl](https://huggingface.co/datasets/gorilla-llm/Berkeley-Function-Calling-Leaderboard) | Apache-2.0 | 480 |
11
+ | [xlam-irrelevance](https://huggingface.co/datasets/MadeAgents/xlam-irrelevance-7.5k) | CC-BY-4.0 | 400 |
12
+ | [german-fc](https://huggingface.co/datasets/flozi00/german-function-calling) | Apache-2.0 | 413 |
13
+ | [sauerkraut-irrelevance-de](https://huggingface.co/datasets/VAGOsolutions/SauerkrautLM-Fermented-Irrelevance-GER-DPO) | MIT | 400 |
14
+ | [seal-tools](https://huggingface.co/datasets/casey-martin/Seal-Tools) | MIT | 480 |
15
+ | [toolace](https://huggingface.co/datasets/Team-ACE/ToolACE) | Apache-2.0 | 480 |
16
+ | [toolbench](https://huggingface.co/datasets/mangopy/ToolRet-Queries) | Apache-2.0 (ToolBench; ToolRet repo Apache-2.0) | 480 |
17
+ | [api-bank](https://huggingface.co/datasets/mangopy/ToolRet-Queries) | MIT (API-Bank; ToolRet repo Apache-2.0) | 101 |
18
+
19
+ ## Agent-skill descriptions (name and one-line description only)
20
+
21
+ A sample of these was used to build synthetic skill libraries.
22
+
23
+ | Repository | License | Skills collected |
24
+ |---|---|---:|
25
+ | https://github.com/anthropics/skills | Apache-2.0 (per skill; docx/pdf/pptx/xlsx proprietary, excluded) | 24 |
26
+ | https://github.com/obra/superpowers | MIT | 15 |
27
+ | https://github.com/openai/skills | Apache-2.0/MIT per skill (Figma-terms skills excluded) | 26 |
28
+ | https://github.com/huggingface/skills | Apache-2.0 | 11 |
29
+ | https://github.com/vercel-labs/agent-skills | MIT (stated in README) | 6 |
30
+ | https://github.com/trailofbits/skills | CC-BY-SA-4.0 | 134 |
31
+ | https://github.com/K-Dense-AI/claude-scientific-skills | MIT | 125 |
32
+ | https://github.com/wshobson/agents | MIT | 280 |
33
+ | https://github.com/VoltAgent/awesome-claude-code-subagents | MIT | 149 |
34
+ | https://github.com/0xfurai/claude-code-subagents | MIT | 130 |
35
+ | https://github.com/lst97/claude-code-sub-agents | MIT | 5 |
36
+ | https://github.com/Jeffallan/claude-skills | MIT | 15 |
37
+ | https://github.com/github/awesome-copilot | MIT | 380 |
38
+ | https://github.com/alirezarezvani/claude-skills | MIT | 200 |
39
+ | https://github.com/ComposioHQ/awesome-claude-skills | Apache-2.0 (README badge; no root LICENSE file) | 81 |
40
+ | https://github.com/davila7/claude-code-templates | MIT | 450 |
41
+ | https://github.com/google/skills | Apache-2.0 | 60 |
42
+ | https://github.com/phuryn/pm-skills | MIT | 84 |
43
+ | https://github.com/LambdaTest/agent-skills | MIT | 42 |
44
+ | https://github.com/muratcankoylan/Agent-Skills-for-Context-Engineering | MIT | 12 |
45
+ | https://github.com/GarethManning/education-agent-skills | CC-BY-SA-4.0 | 165 |
46
+ | https://github.com/sanjay3290/ai-skills | Apache-2.0 | 10 |
47
+ | https://github.com/zxkane/aws-skills | MIT | 1 |
48
+ | https://github.com/redis/agent-skills | MIT | 2 |
49
+ | https://github.com/qdrant/skills | Apache-2.0 | 11 |
50
+ | https://github.com/prisma/skills | MIT | 8 |
51
+ | https://github.com/obra/superpowers-lab | MIT | 3 |
52
+ | https://github.com/ognjengt/founder-skills | MIT | 14 |
53
+ | https://github.com/mukul975/Anthropic-Cybersecurity-Skills | Apache-2.0 | 120 |
54
+ | https://github.com/trycourier/courier-skills | MIT | 1 |
55
+ | https://github.com/realkimbarrett/advertising-skills | MIT (stated in README) | 12 |
56
+ | https://github.com/resend/resend-skills | MIT | 2 |
57
+ | https://github.com/cypress-io/ai-toolkit | MIT | 3 |
58
+ | https://github.com/tjboudreaux/cc-thinking-skills | MIT | 28 |
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "/private/tmp/stem-gte-finetune-20260905-2008/checkpoints/epoch2",
3
  "architectures": [
4
  "NewForSequenceClassification"
5
  ],
 
1
  {
2
+ "_name_or_path": "join3r/stem-gte-memory",
3
  "architectures": [
4
  "NewForSequenceClassification"
5
  ],
onnx/model_quantized.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5cbb66c672aa71fb6609880ba8e8370ff5be3376fc95215317c92e41e7743e4e
3
  size 341032790
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e88d816d1cefef69a7b43df3bb2dabdb92a21b0e3ce92cd6ceb0e753fcda74f
3
  size 341032790
stem-model.json CHANGED
@@ -1,15 +1,18 @@
1
  {
2
  "schema_version": 1,
3
- "model_id": "gte-memory-20260905-epoch2",
4
  "display_name": "Stem GTE Memory",
5
  "base_model": "Alibaba-NLP/gte-multilingual-reranker-base",
6
  "base_revision": "8215cf04918ba6f7b6a62bb44238ce2953d8831c",
7
  "upstream_code_revision": "40ced75c3017eb27626c9d4ea981bde21a2662f4",
8
- "checkpoint": "epoch2",
 
 
9
  "dtype": "q8",
10
  "scoring": "gte-scalar",
11
  "fact_threshold": -1.6370911598205566,
12
- "skill_threshold": -3.9743599891662598,
 
13
  "sensitive_fact_margin": 2,
14
  "languages": [
15
  "cs",
@@ -19,8 +22,8 @@
19
  ],
20
  "files": {
21
  "config.json": {
22
- "sha256": "e5f2efd1e04e5e4465b8c6c2f6af1f00a5d66a839e825b0af8edba68ceb8b1bb",
23
- "bytes": 1627
24
  },
25
  "tokenizer.json": {
26
  "sha256": "3a56def25aa40facc030ea8b0b87f3688e4b3c39eb8b45d5702b3a1300fe2a20",
@@ -35,7 +38,7 @@
35
  "bytes": 964
36
  },
37
  "onnx/model_quantized.onnx": {
38
- "sha256": "5cbb66c672aa71fb6609880ba8e8370ff5be3376fc95215317c92e41e7743e4e",
39
  "bytes": 341032790
40
  }
41
  }
 
1
  {
2
  "schema_version": 1,
3
+ "model_id": "gte-memory-20261021",
4
  "display_name": "Stem GTE Memory",
5
  "base_model": "Alibaba-NLP/gte-multilingual-reranker-base",
6
  "base_revision": "8215cf04918ba6f7b6a62bb44238ce2953d8831c",
7
  "upstream_code_revision": "40ced75c3017eb27626c9d4ea981bde21a2662f4",
8
+ "previous_model_id": "gte-memory-20260905-epoch2",
9
+ "previous_revision": "dae4f76d6974e70cdc007d3ca66d82226791822f",
10
+ "checkpoint": "t4v3/e1",
11
  "dtype": "q8",
12
  "scoring": "gte-scalar",
13
  "fact_threshold": -1.6370911598205566,
14
+ "skill_threshold": -2.4,
15
+ "skill_example_threshold": -0.25,
16
  "sensitive_fact_margin": 2,
17
  "languages": [
18
  "cs",
 
22
  ],
23
  "files": {
24
  "config.json": {
25
+ "sha256": "01a3c997ee3f9ebdf250fa01086487f336ce573a46ad24cbb0a249e22eaf98b6",
26
+ "bytes": 1586
27
  },
28
  "tokenizer.json": {
29
  "sha256": "3a56def25aa40facc030ea8b0b87f3688e4b3c39eb8b45d5702b3a1300fe2a20",
 
38
  "bytes": 964
39
  },
40
  "onnx/model_quantized.onnx": {
41
+ "sha256": "1e88d816d1cefef69a7b43df3bb2dabdb92a21b0e3ce92cd6ceb0e753fcda74f",
42
  "bytes": 341032790
43
  }
44
  }