Hugging Face's logo

hkblockchaintechnology
/
tiny-agent-gguf

Text Generation
GGUF
English
multilingual
llama.cpp
cpu-inference
q4_k_m
qwen3
offline
local-first
low-end-hardware
document-generation
conversational
Model card Files
xet
 Community
tiny-agent-gguf / README.md
hkblockchaintechnology's picture
hkblockchaintechnology
Upload README.md with huggingface_hub
e568289 verified
preview code
|
Raw History Blame Contribute Delete
28.5 kB
---
license: apache-2.0
language:
- en
- multilingual
base_model:
- Qwen/Qwen3-1.7B
- Qwen/Qwen3-4B-Instruct-2507
base_model_relation: quantized
library_name: llama.cpp
pipeline_tag: text-generation
quantized_by: hkblockchaintechnology
tags:
- gguf
- llama.cpp
- cpu-inference
- q4_k_m
- qwen3
- offline
- local-first
- low-end-hardware
- document-generation
---
# tiny-agent-gguf
![Tiny Agent: one sentence in, a finished Office file out — offline, on your CPU](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/resolve/main/docs/art/hero.png)
Quantized language models for **Tiny Agent** — a Windows application that gives
you Word, Excel and PowerPoint with an AI assistant that runs on your own CPU,
with no GPU, no cloud API and no account.
**The app (1.35 GB):** [Download `TinyAgentSetup-1.1.0.exe`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/resolve/main/releases/TinyAgentSetup-1.1.0.exe)
([file page](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/releases/TinyAgentSetup-1.1.0.exe)).
It is in the **dataset** repository, not this model repository, under
`releases/`. The file browser does not preview `.exe` files, so use the direct
link if you cannot see it. SHA-256:
`26e571473a7717935097fda55342745187804d87398b43f59de394e194d86dff`.
The installer is not code-signed; Windows SmartScreen may ask you to choose
**More info -> Run anyway**. A [portable ZIP (1.44 GB)](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/resolve/main/releases/TinyAgent-1.1.0-portable.zip)
is also published, SHA-256
`d986db4d58f220ba760ca94d98304a3c24ac1677cb90b36a47176222378e0ccc`.
All files and checksums: [`releases/README.md`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/releases/README.md).
This repository supplies the model that installer uses — or that you can point any
local `llama-server` at.
**The signature idea:** one window with three editors — a page for `.docx`, a grid
for `.xlsx`, a deck for `.pptx` — and the local assistant in the panel beside them.
You type on the document, or describe a change in ordinary words; the model plans
it, plain Python applies it to a private copy, and the copy is verified before it
replaces what you see. `More tools` opens the workshop page for Create, Edit, Polish
and Batch, which run without a document open. Every result is an ordinary file that
real Office applications open.
**Release status, 1.1.0 (22 October 2026):** the editor ships. Verified on the
frozen build: both pages serve, a `.docx` is created through the assistant, a
hand-inserted table lands in the right place and is not duplicated by a second save,
a formula shows its result while the file keeps the formula, a picture survives an
AI edit, a deck prints every slide, and a whole-document job plus PDF export passes
`FROZEN_OK`. 56/56 file-level audit checks; the editor gate passes on Windows and on
a 20-core Linux box. The installer and the portable ZIP were both run end to end
(`INSTALLER_OK`: silent install, model loaded from the installed folder, slide and
table edited through the installed copy, clean quit, uninstall keeps the user's
document; the ZIP extracted and started with its own model). One-sentence edit to a
short document: median **4.0 s over 3
runs** (3.7–5.0 s) on a 4-core i5-4460, 16 GB, Windows 10, llama.cpp b11714, 4
threads, nothing else running. The 1.0.0 measurement record is kept in
[`docs/COMPARISON-REPORT.md`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/COMPARISON-REPORT.md).
**What changed on 21 October:** the runtime had been started with half the
machine's cores (`cpu_count() // 2`). Measured on the 20-core measurement box, the
same request took **4.85 s with 10 threads and 3.50 s with 20**; on the 4-core
reference desktop, 21.29 s against 19.35 s. The default now asks the kernel for the
physical core count, and the whole 73-task campaign was re-run on it: medians fell
from 1.7–2.5 s to **1.2–1.6 s per document**. Full record:
[`logbook/2026-10-21-speed.md`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/logbook/2026-10-21-speed.md).
These files are **quantizations of existing Apache-2.0 models**, not new models.
Nothing here is fine-tuned: we convert and quantize, so the knowledge, biases
and limitations are inherited unchanged from the base model.
## TL;DR
| | |
|---|---|
| What it is | Qwen3-1.7B converted to GGUF and quantized to 4-bit |
| What it's for | Filling short, schema-constrained JSON slots in the Tiny Agent pipeline |
| Licence | Apache-2.0 — free to redistribute with a product |
| Runs on | CPU only, no GPU. ~1.2 GB RAM for the 4-bit build |
| Speed | **1.2–1.6 s per document** end to end on the 20-core measurement box; 6.5 tok/s raw on a 2014-era 4-core desktop, **11.5 tok/s** with speculative decoding |
| Not for | Being asked to "write me a report" in a single prompt |
## Files in this repository
| File | Size | What it is |
|---|---|---|
| `qwen3-1.7b-Q4_K_M.gguf` | **1.28 GB** | the default model |
| `qwen3-4b-instruct-2507-Q4_K_M.gguf` | **2.33 GB** | optional 4B tier: newer Qwen3-4B-Instruct-2507, clears the same tasks as the 1.7B at 1.6x the time |
| `qwen3-0.6b-Q4_K_M.gguf` | **0.48 GB** | smallest and fastest (1.01 s vs 1.60 s on v3), but fails 5 of 54 strict-count samples |
| `qwen3-1.7b-eagle3.gguf` | **0.28 GB** | EAGLE3 speculative draft for the 1.7B model |
Sizes are the real blob sizes, not estimates. All four are Apache-2.0 and safe
to redistribute inside a product.
The 4B tier was added on 2026-10-20 and is **not** the default. On the original
16-task suite it tied the 1.7B: **16/16** with clean integrity at a median
**53.7 s**, against **48/48** at **16.2 s**, for 3.3x the time. On the harder
18-task corpus added the same day it first won — **17/18** against **14/18**, one
pass each. Three engine fixes then closed the gap (`logbook/2026-10-20-counts.md`),
and the full re-run of the final tree
(`logbook/2026-10-20-shipped-numbers.md`) measured **54/54 on both tiers**, 108 of
108 samples integrity-clean, at **14.3 s** and **25.4 s** median. A third corpus,
**v2 (21 new tasks, 55 in total)**, then measured **63/63 on both tiers**, 126 of
126 samples integrity-clean, at **13.4 s** and **35.7 s** median
(`logbook/2026-10-20-v2.md`); its long samples are all create tasks (worst 203 s on
the 4B against 44 s on the 1.7B). A fourth corpus, **v3 (18 strict-count tasks, 73
in total, where over-delivery fails)**, then measured **54/54 on both tiers** and
**49/54 on the 0.6B** on the measurement box: **1.60 s**, **2.56 s** and **1.01 s**
median, with v2 at **62/63** on the 1.7B — one sample added two slides where one was
asked for (`logbook/2026-10-20-v3.md`). All four corpora are cleared
by the two larger tiers, which is not a claim that they are equal: the 4B has more
capacity for instructions these corpora do not ask. The 1.7B stays the default
because it is half the download, clears every measured task, and costs 1.6x less
time on the newest corpus. The tier
experiment still did its job — it exercised discovery with a real second file, and
it found defects that the default tier alone did not. Raw runs:
`docs/bench/bench-v1-qwen3-1.7b-Q4_K_M.json`,
`docs/bench/bench-v2-qwen3-1.7b-Q4_K_M.json`,
`docs/bench/bench-v3-qwen3-1.7b-Q4_K_M.json`,
`docs/bench/bench-v3-qwen3-4b-instruct-2507-Q4_K_M.json`,
`docs/bench/bench-v3-qwen3-0.6b-Q4_K_M.json`, with the earlier revisions beside
them as `-r3` and `-final`, and the single-pass originals unchanged.
![Pass rate by corpus and tier: 48/48, 54/54, 62/63 and 54/54, with the 0.6B at 49/54 on v3](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/resolve/main/docs/art/pass-rate.png)
*Every published sample, drawn from the run files linked above. Bars are passes
out of samples; the dashed line is 100%. A bar below the line is a task the
pipeline did not complete — shown, not hidden.*
![Latency per document: median, p95 and worst, log scale, 1.0 to 2.6 s medians](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/resolve/main/docs/art/latency.png)
*One sentence to a verified file, warm model, on the 20-core measurement box.
The dot is the median, the bar reaches p95, the tick is the worst sample of that
run. The 4B's long tail is why it is not the default.*
![v3 per format: Word 21/21, Excel 18/18, PowerPoint 15/15 for both larger tiers](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/resolve/main/docs/art/formats.png)
*v3 asks for exact counts, so over-delivery is a failure. Both larger tiers clear
all three formats; the 0.6B loses five samples, all of them count-sensitive.*
SHA-256, for verifying a download:
| File | SHA-256 |
|---|---|
| `qwen3-1.7b-Q4_K_M.gguf` | `245362960ec00bfe277c0e06ba9811a5f1a580f6238d3bc50b6fd3a82f031051` |
| `qwen3-4b-instruct-2507-Q4_K_M.gguf` | `3605803b982cb64aead44f6c1b2ae36e3acdb41d8e46c8a94c6533bc4c67e597` |
| `qwen3-0.6b-Q4_K_M.gguf` | `e4c97785c11e2d8547a54fe104f4bedcdf3358d7e95d3f36876ed6caa96ab851` |
| `qwen3-1.7b-eagle3.gguf` | `7fd1b0338f7790fa1f8b372ee72060162c4db74e4efbd89f5b28ebb9a01b4c0b` |
## What it can do
Measured by actually running it — these are observed behaviours, not claims:
| Task | Verdict | Notes |
|---|---|---|
| Return strictly valid JSON matching a schema | ✅ reliable | Enforced by a llama.cpp grammar, so it is structurally impossible to emit malformed JSON |
| Plan a document outline (3–6 sections) | ✅ reliable | Produces sensible, ordered headings |
| Write 1–3 paragraphs of grammatical prose on a topic | ✅ good | Coherent and readable; occasionally generic |
| Condense source material into 3–5 short bullets | ✅ good | Best when given explicit research notes |
| Follow small constraints ("under 12 words", "no markdown") | ✅ usually | Occasionally needs the constraint repeated |
| Write a *whole* multi-page document in one pass | ❌ no | It drifts, repeats and loses structure — see the design note below |
| Be factually accurate from memory alone | ❌ no | It will invent plausible specifics. Grounding in retrieved text is what makes output trustworthy |
| Do arithmetic, analysis or comparisons | ❌ no | Use Python for that |
| Cite its sources unprompted | ❌ no | Citations are attached by the pipeline, not the model |
| Keep a consistent voice across many sections | ⚠️ partial | Sections are generated independently, so tone can drift |
## Where the assistant sits: every workflow, not a chat box
The model is one component of an Office-style application, and the interface is
built so the assistant is available at each step rather than in a separate tab:
| Surface | The user does | The model does | Plain Python does |
|---|---|---|---|
| **Create** | writes a paragraph brief | plans the outline and short text slots | styles, tables, charts, pagination, file writing |
| **Edit** | opens an existing file and says what changes | returns only the allowed replacement/row/slide groups | applies them to a copy and fingerprints the result |
| **Polish** | asks for a review | offers editorial suggestions | finds empty slides, blank cells and other mechanical defects |
| **Batch** | writes one instruction and selects files | plans each file independently | applies, verifies and reports per file |
That is the signature of the product: **an Office workspace with AI in every
workflow**, not an assistant that produces text for a human to paste. The local
model is deliberately small because the surrounding code owns structure, file
format and verification.
## Why this model, and not something bigger
Tested rather than assumed. Six CPU-sized models were run through the real
pipeline against identical frozen research notes, and scored on how faithfully
they filled the JSON slots:
| model | grounding | repetition | verdict |
|---|---|---|---|
| **qwen3:1.7b** | **0.460** | 0.065 | **best of the six** |
| phi4-mini:3.8b | 0.375 | 0.062 | slower, grounded less |
| granite4:micro | 0.353 | 0.050 | worst instruction-following on shape |
| qwen3:4b | 0.327 | **0.128** | 2.4x the parameters, worst repetition |
| qwen3:0.6b | 0.213 | 0.000 | thin, but ~2.4x faster |
| llama3.2:3b | 0.562* | 0.000 | failed: 2 sections, 118 words, stopped |
*\*tiny-sample artefact; it returned the wrong shape.*
`Qwen3-1.7B` grounds retrieved facts best while staying fastest of the
faithful options. On this hardware the bottleneck is **tokens per second, not
parameters of world knowledge**, so a larger model buys nothing measurable and
costs multiples of the time per document. Full method and raw data:
[`docs/MODEL-BENCH.md`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/MODEL-BENCH.md).
## How Tiny Agent uses it
The model is never asked to write a document, and it never touches a file. It is
asked one small question at a time, and deterministic Python assembles the result:
![Pipeline: sentence → local model → fixed-shape plan → Python applier → verified file](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/resolve/main/docs/art/flow.png)
```
1. route which KINDS of change could this instruction need?
(replacements | newHeadings | newParagraphs | rows | totals
| slides | charts | theme) <- deterministic
2. plan the model fills ONLY those slots, JSON enforced by a llama.cpp
grammar, so a malformed plan is structurally impossible
3. apply python-docx / openpyxl / python-pptx write a NEW file <- deterministic
4. verify fingerprint the file before and after, and report anything that
changed WITHOUT being asked <- deterministic
```
Step 1 is what makes a model this size dependable: rather than a flat form with
eight optional sections to wander around, it is handed only the two or three the
instruction actually needs. Steps 3 and 4 mean its output can be *checked* rather
than trusted, which is the difference between a demo and a tool you would point at
real work.
Measured with the real model in the product pipeline:
| Run | Result |
|---|---|
| Competitive benchmark, i5-7200U (2 cores / 4 threads, 8 GB, no GPU) | **4 of 4** tasks verified; **13.8–34.5 s**, mean **19.8 s** |
| Real Microsoft Office opened the four outputs | **4 of 4**, no repair step |
| Real Microsoft Office exported them to PDF | **4 of 4**, content intact |
| Repeatability harness, reference hardware | **12 of 12** plans produced a usable document (3 runs each of a Word edit, a spreadsheet edit, a deck edit and a new Word table) |
| Warm plan | **2.5–5 s** typical; slowest measured 27 s (new Word table) |
The detailed head-to-head, including where Microsoft Office wins, is in
[`docs/COMPARISON-REPORT.md`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/COMPARISON-REPORT.md).
One measured failure worth publishing, because it is the kind of thing a card
usually leaves out: offered a single optional group, this model answered with prose
in `summary` and left the group empty — twice in a row, on every run. Naming the
group in the retry prompt did not fix it; making that one group **required** in the
retry grammar did, immediately. That is the whole design thesis in one incident:
the fix belongs in the shape of the output, not in the politeness of the request.
Each step is a JSON-schema-constrained call. For example, a plan's contract is
handed to llama.cpp as a grammar, so the shape is enforced by the decoder rather
than requested in the prompt:
```json
{
"type": "object",
"properties": {
"newParagraphs": {
"type": "array",
"minItems": 1,
"items": {"type": "string"}
}
},
"required": ["newParagraphs"]
}
```
That is why a 1.7B model is usable for a real task: structure, formatting, file
writing and verification are handled by code, and the model only supplies short
strings where a small model is genuinely competent.
## Hardware and measured speed
Reference machine — deliberately weak, and the actual development target:
| | |
|---|---|
| CPU | Intel Core i5-4460 (Haswell, 4 cores / 4 threads, AVX2) |
| RAM | 16 GB DDR3-1333 |
| GPU | none used |
| Storage | 7200 rpm HDD |
The 73-task campaign was also run on a rented Linux box with more cores, so that a
full sweep could be repeated without waiting on the desktop:
| | 20-core measurement box |
|---|---|
| CPU | 20 vCPU, x86-64, one thread per core |
| RAM | 160 GB |
| GPU | none used |
| Build | llama.cpp **b11714**, the same release the Windows installer ships |
| Model | File size | Cold load | **Generation** |
|---|---|---|---|
| Qwen3-1.7B Q4_K_M | 1.28 GB | ~13 s | **6.50 tok/s** |
| Qwen3-1.7B Q4_K_M **+ n-gram speculation** | 1.28 GB | ~13 s | **11.46 tok/s** (1.76x) |
| Qwen3-1.7B Q4_K_M + EAGLE3 draft | +0.28 GB | ~13 s | **4.00 tok/s — slower**, see below |
| Qwen3-0.6B Q4_K_M | 0.48 GB | ~6 s | ~24 tok/s (measured via Ollama) |
The generation figures were measured with the exact CPU-only `llama.cpp` build
that ships in the product, 4 threads, a 4096-token context and a 220-token
budget. The n-gram row is the mean of repeat runs on the same request.
### One thread per physical core, not half of them
Until 21 October the runtime asked for `cpu_count() // 2` threads, which is right
on a hyper-threaded laptop and wrong everywhere else. On the measurement box that
halved the work to 10 threads of 20; on the 4-core desktop it used 2 of 4. The
default now reads the physical core count from the kernel (Windows
`GetLogicalProcessorInformationEx`, Linux `/proc/cpuinfo`), overridable with
`TINYAGENT_THREADS`.
![Thread sweep on both machines: 4.85 to 3.50 s on the box, 21.29 to 19.35 s on the desktop](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/resolve/main/docs/art/threads.png)
*Same 220-token request, same flags, only the thread count changed. Raw runs:
[`docs/bench/thread-sweep-box.json`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/bench/thread-sweep-box.json)
and
[`docs/bench/thread-sweep-laptop.json`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/bench/thread-sweep-laptop.json).
The container exports no CPU identifier, so the box file records
`"cpu": "unknown"` and `logical_cpus: 20`.*
### What that means in practice
Measured end to end on the shipped runtime, web research enabled, 11 sources
retrieved:
| Job | Wall clock |
|---|---|
| 3-section researched Word report | **202 s** |
| 3-slide deck | ~112 s |
The effective rate is well below the raw tokens/second because prompt
processing over injected research context dominates, and every job makes
several calls in sequence. That is why the interface streams progress: two to
four minutes is perfectly acceptable for a "generate document" button, but only
if the user can see which stage is running.
### Cold start
Double-click to a usable window, model included, on the reference desktop with
its 7200 rpm HDD:
| Build | Window answers | First document ready |
|---|---|---|
| 1.0.0, first build from the measured tree | 2.6 s | 5.1 s |
| 1.0.0, delivered build | 3.3 s | 5.7 s |
![Cold start: window ready in 2.6–3.3 s, model ready in 5.1–5.7 s](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/resolve/main/docs/art/startup.png)
*There is no preloading daemon and no service: the window appears first, then the
model loads behind it. Source:
[`docs/bench/cold-start.json`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/bench/cold-start.json).*
### Speculative decoding is not optional on a CPU
Speculative decoding drafts several tokens cheaply and verifies them in one
forward pass of the real model. On a CPU — where a forward pass is expensive and
a batch is comparatively cheap — this is the single largest speedup available,
and it costs nothing in output quality.
`llama-server -m qwen3-1.7b-Q4_K_M.gguf --spec-type ngram-mod` needs **no extra
download** and measured **6.50 → 11.46 tok/s**. It mines repeated token patterns
from the context, so it is a *warm-session* win: the first request of a session
gains almost nothing (7.33 tok/s) and the third reaches 11.46 tok/s as the
shared hash pool fills. That suits a document generator exactly, because a job
is several calls in a row sharing a long system prompt and emitting repetitive
JSON.
EAGLE3 is the theoretically stronger option: a one-layer draft that reads the
target model's hidden states and should therefore achieve a much higher
acceptance rate. This repo ships `qwen3-1.7b-eagle3.gguf` and it is used like
this:
```bash
llama-server -m qwen3-1.7b-Q4_K_M.gguf -md qwen3-1.7b-eagle3.gguf \
--spec-type draft-eagle3 --spec-draft-n-max 8
```
**It measured slower on a CPU, and this card will not pretend otherwise.**
| Configuration | Speed | Draft acceptance |
|---|---|---|
| No speculation | 6.50 tok/s | — |
| n-gram (`ngram-mod`) | **11.46 tok/s** | 0.53 (warm) |
| EAGLE3 draft | **4.00 tok/s** | 0.089 |
Acceptance was only 8.9%, and the draft shares the same 4 physical cores as the
target, so it steals more throughput than it saves. EAGLE3 is a large win on a
GPU — where the tiny draft runs nearly free beside the target — and a
regression on a 4-thread CPU.
This was re-checked on a much faster machine (a 20-core server CPU running
CPU-only, 4 threads) using our own converted GGUF: baseline 21.90 tok/s, n-gram
21.60 tok/s on a cold single run, EAGLE3 **16.30 tok/s (0.74x)**. So EAGLE3 is
slower regardless of target conversion, and n-gram's benefit genuinely depends
on a warm, repetitive session — which is the normal case for this tool.
**Prefer n-gram speculation on CPU; treat the EAGLE3 file as an experiment.**
Practical consequences:
* **Cold load costs ~13 s on a spinning disk.** Keep the model resident between
requests rather than reloading per call.
* Use one thread per *physical* core. More threads than physical cores is slower.
* Qwen3 is a reasoning model. Left alone it spends most of its budget on hidden
thinking tokens, so **disable thinking** (`think=false`, or
`chat_template_kwargs={"enable_thinking": false}`).
* A grounded 3-section report takes ~140 s end to end. That is acceptable for a
"generate document" button, but only with progress reporting.
## Quick start
**llama.cpp server** (OpenAI-compatible, supports JSON-schema constrained output):
```bash
llama-server -m qwen3-1.7b-Q4_K_M.gguf -c 4096 -t 4 --port 8080
```
**llama.cpp CLI:**
```bash
llama-cli -m qwen3-1.7b-Q4_K_M.gguf -c 4096 -t 4 \
-p "Return only JSON: {\"heading\": str, \"paragraphs\": [str]} for a section about heat pumps."
```
**Ollama** (optional — the product does not need it):
```bash
ollama pull hf.co/hkblockchaintechnology/tiny-agent-gguf
```
The official CPU-only `llama.cpp` Windows release is an **18.8 MB** zip (a 46 MB
folder unpacked) and needs no GPU, CUDA toolkit or Python environment — that is
the intended way to ship this.
## Prompting notes for a model this size
1. **One question per call.** Asking for several things at once is where small
models fall apart.
2. **Constrain the output shape.** Use a JSON schema or GBNF grammar. It removes
an entire class of parsing failures.
3. **Disable thinking.** Qwen3 will otherwise spend its whole budget reasoning.
4. **Give it the facts.** Put retrieved text in the prompt; it cannot supply
accurate specifics from memory.
5. **Ask for less text.** 2–3 short paragraphs per section works; "write a
detailed 2000-word report" does not.
6. **Expect and clean up artefacts.** It adds list numbering to headings and
sometimes leaks markdown into prose fields. Strip these deterministically
during assembly rather than fighting them with prompt wording.
## Limitations
* **Hallucination.** Without grounding, specifics (figures, dates, product
names) are unreliable and often invented. Tiny Agent mitigates this by injecting
retrieved web text and attaching a source list, but it does not eliminate it.
* **Inherited knowledge cutoff and biases.** Because this is a quantization, the
base model's training data, cutoff and biases apply unchanged.
* **Language.** The base model is multilingual, but the pipeline prompts and all
validation were done in English.
* **Repetition.** Sections written in separate calls sometimes restate the same
point. Human editing is expected.
* **Not a general chat model.** It is tuned here for structured slot-filling, not
for open-ended conversation.
* **Quantization degrades quality slightly** versus the full-precision base
model. 4-bit is the point where quality and speed balance best on this class
of hardware.
## Training data
No training or fine-tuning was performed. This repository contains quantized
conversions of:
* [`Qwen/Qwen3-1.7B`](https://huggingface.co/Qwen/Qwen3-1.7B)
* [`Qwen/Qwen3-4B-Instruct-2507`](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507)
* [`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B)
and a converted speculative draft:
* [`AngelSlim/Qwen3-1.7B_eagle3`](https://huggingface.co/AngelSlim/Qwen3-1.7B_eagle3)
(Apache-2.0, Tencent AngelSlim) — converted to GGUF with
`convert_hf_to_gguf.py --target-model-dir Qwen/Qwen3-1.7B`. It is a *draft*
model: it proposes tokens that the 1.7B model verifies, and it never changes
the output, only the speed. It cannot be used on its own.
Refer to the base model cards for training data, evaluation results and
limitations.
Why these models: both are **Apache-2.0**, so they can be redistributed with a
commercial product without additional conditions. Llama, Gemma and several other
otherwise-capable small models are deliberately **not** bundled, because their
licences carry redistribution terms that are unsuitable for shipping to end
users.
## License and attribution
Apache-2.0, inherited from the base models. Copyright in the base weights
belongs to the Qwen team; these are derivative quantizations redistributed under
the same licence.
## Related
* Project source, benchmarks and generated documents: [`hkblockchaintechnology/tiny-agent`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent)
* Windows installer (1.35 GB, in the dataset repository): [`TinyAgentSetup-1.1.0.exe`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/resolve/main/releases/TinyAgentSetup-1.1.0.exe)
* Every speed number, with its raw run files and the thread-sweep data: [`docs/SPEED.md`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/SPEED.md)
* User guide: [`TinyAgent-UserGuide.pdf`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/TinyAgent-UserGuide.pdf)
* The 1.0.0 measurement record: [`COMPARISON-REPORT.md`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/COMPARISON-REPORT.md)
* Development logbook, with the commands, failures and artifact hashes behind each claim: [`LOGBOOK.md`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/LOGBOOK.md)
* Problems and questions: [discussions on the dataset repository](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/discussions)
* Licence: the application's own terms and third-party notices are in [`docs/LICENSE.txt`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/LICENSE.txt); the Apache-2.0 text covering these weights is in [`docs/LICENSE-APACHE-2.0.txt`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/docs/LICENSE-APACHE-2.0.txt), with the required statement of changes in [`NOTICE`](https://huggingface.co/datasets/hkblockchaintechnology/tiny-agent/blob/main/NOTICE).