README.md · BryceTRussell/tinycron at main

tinycron / README.md

Release 1.3.0

219ab3e verified 1 day ago

15.5 kB

---
license: apache-2.0
language:
- en
library_name: onnx
pipeline_tag: text-classification
datasets:
- AmazonScience/massive
metrics:
- exact_match
tags:
- cron
- crontab
- text-to-cron
- scheduling
- natural-language
- time-expressions
- structured-prediction
- linux
- devops
- gru
- character-level
- int8
- cpu
- offline
model-index:
- name: tinycron
  results:
  - task:
      type: text-classification
      name: Text to cron
    dataset:
      name: Template requests (held out)
      type: tinycron-template
    metrics:
    - type: exact_match
      value: 0.984
      name: Exact match
  - task:
      type: text-classification
      name: Text to cron
    dataset:
      name: Requests written by an AI model (not trained on; errors guided training data)
      type: tinycron-ai
    metrics:
    - type: exact_match
      value: 0.952
      name: Exact match
  - task:
      type: text-classification
      name: Text to cron
    dataset:
      name: TOPv2 alarm and reminder requests (test)
      type: topv2
    metrics:
    - type: exact_match
      value: 0.964
      name: Exact match
  - task:
      type: text-classification
      name: Text to cron
    dataset:
      name: Real assistant requests from five datasets (held out)
      type: tinycron-assistant
    metrics:
    - type: exact_match
      value: 0.942
      name: Exact match
  - task:
      type: text-classification
      name: Text to cron
    dataset:
      name: Fresh requests written by an AI model (held out)
      type: tinycron-fresh
    metrics:
    - type: exact_match
      value: 0.939
      name: Exact match
  - task:
      type: text-classification
      name: Text to cron
    dataset:
      name: Hand-written tricky requests (not trained on; errors guided training data)
      type: tinycron-probes
    metrics:
    - type: exact_match
      value: 0.974
      name: Exact match
  - task:
      type: text-classification
      name: Text to cron
    dataset:
      name: Real text from code, forums and Q&A sites (held out)
      type: tinycron-real-text
    metrics:
    - type: exact_match
      value: 0.879
      name: Exact match
  - task:
      type: text-classification
      name: Text to cron
    dataset:
      name: GitHub issue sentences (never trained on; errors guided training data)
      type: tinycron-github
    metrics:
    - type: exact_match
      value: 0.776
      name: Exact match
---
# tinycron

Turns an English schedule into a Linux cron line.

| Input | Output |
|---|---|
| every weekday at 9am | `0 9 * * 1-5` |
| Every Monday At 5:30 PM | `30 17 * * 1` |
| every 15 minutes between 9 and 5 | `*/15 9-17 * * *` |
| twice a month | `0 0 1,15 * *` |
| nightly, or twice a day | `0 */12 * * *` |
| every other tuesday | `UNSUPPORTED` |
| every few hours | `INCOMPLETE` |

If cron can't express a request, the answer is a class instead:

- `UNSUPPORTED`: cron can't express it (seconds, "every other week", "the last day of the month").
- `INCOMPLETE`: a time or count is missing.
- `INVALID`: not a schedule.

Each answer has a confidence from 0 to 1. The model is 1.6 MB (1.9 MB with its tokenizer) and takes 2 to 4 ms on a desktop CPU.

## Use

```bash
pip install -r requirements.txt
python to_cron.py "every weekday at 9am"
python to_cron.py --json < requests.txt
```

```python
from to_cron import CronModel
model = CronModel()
model.to_cron("every weekday at 9am")   # "0 9 * * 1-5"
model.read("at 7am on weekdays and 9am on weekends")
# ("schedule", ["0 7 * * 1-5", "0 9 * * 0,6"], 0.99...)
```

The command marks an answer `(unsure, check it)` when its confidence is under the `95%` threshold in `config.json`. Answers above that threshold were right 95% of the time on the dev set, counting `UNSUPPORTED` and `INVALID` as one answer.

Tested with Python 3.12 and onnxruntime 1.19, and with onnxruntime-web 1.20.1 (WebAssembly), which gives the same answers. Needs onnxruntime 1.19 or newer. Loading the model takes about 0.5 seconds, once.

## Files

| File | What it is |
|---|---|
| `model.onnx` | The model |
| `tokenizer.json` | The tokenizer |
| `config.json` | Version, file hashes, field layout, confidence thresholds |
| `to_cron.py` | Python runner, as a command or a module |
| `rules.py` | Exact parser for text that is already a cron line (optional) |
| `requirements.txt` | Python packages for `to_cron.py` |
| `examples.jsonl` | Test vectors for checking a port |

## Output

- One to three standard 5-field cron lines (Vixie cron and cronie; Sunday is 0).
- Or `UNSUPPORTED`, `INCOMPLETE` or `INVALID`.

Never seconds, years, `L`, `W`, `#` or `@` shortcuts.

Defaults for vague requests:

- No time means midnight: "every Monday" is `0 0 * * 1`.
- "Morning" is 9:00. "Every night at 7" is 19:00.
- A bare hour from 1 to 6 is afternoon: "at 3" is 15:00.
- "Weekly" is Sunday at midnight. "Monthly" is the 1st at midnight.
- "Between 9 and 5" includes the last hour: `9-17`.
- Time zones are ignored.

## Accuracy

Exact match. None of these rows are in the training data.

| Text | Rows | Accuracy | Wrong per 1,000 |
|---|---|---|---|
| Template requests | 2,875 | 98.4% | 16 |
| Real alarm and reminder requests (TOPv2) | 1,510 | 96.4% | 36 |
| Real assistant requests from five more datasets | 745 | 94.2% | 58 |
| Fresh requests written by an AI model | 949 | 93.9% | 61 |
| Real text from code, forums and Q&A sites | 2,978 | 87.9% | 121 |

Errors on these sets guided the training data, so they flatter this model:

| Text | Rows | Accuracy | Wrong per 1,000 |
|---|---|---|---|
| Requests written by an AI model | 1,762 | 95.2% | 48 |
| Hand-written tricky requests | 155 | 97.4% | 26 |
| GitHub issue sentences, never trained on | 2,699 | 77.6% | 224 |

Other tools on the same rows, counting only rows whose answer is a schedule, scored leniently:

| System | Template | AI-written | Real requests (TOPv2) | Real text |
|---|---|---|---|---|
| english-to-cron 0.1.7 (Rust rules) | 23.3% | 29.3% | 49.5% | 55.8% |
| natural-cron-expression 1.0.2 (PHP rules) | 16.3% | 21.7% | 56.8% | 61.7% |
| [cronformer](https://huggingface.co/uncommonstash/cronformer) 0.1.0 (6.3M parameters, 25 MB) | 43.2% | 41.6% | 70.3% | 64.8% |
| [cronlm](https://huggingface.co/pavstev/cronlm) (16M parameters, 24 MB) | 43.0% | 45.0% | 71.0% | 45.9% |
| [cron-mini](https://huggingface.co/Shpigford/cron-mini) (Qwen2.5-1.5B fine-tune, 3.1 GB)\* | 43.7% | 44.6% | 66.7% | 61.0% |
| [leom21/cron](https://huggingface.co/leom21/cron) (SmolLM2-360M fine-tune, 724 MB)\* | 11.1% | 9.1% | 36.5% | 24.4% |
| Claude Haiku (October 2026), prompted\*† | 80.2% | 76.9% | 46.9% | 71.5% |
| Claude Sonnet (October 2026), prompted\*† | 78.6% | 78.5% | 66.7% | 79.7% |
| Claude Opus (October 2026), prompted\*† | 81.7% | 81.8% | 66.7% | 78.9% |
| this model (1.9M parameters, 1.6 MB) | 98.6% | 94.8% | 96.7% | 90.9% |
\* The language models are too slow for whole splits, so they ran on a seeded sample of 150 rows per column (96 to 126 of them schedules). On the same sample this model scores 98.4%, 93.4%, 95.8% and 91.9%. leom21/cron has no documented prompt; it got "Convert to cron: " before the text, the best of the prompts tried.

† The Claude models got the same one-paragraph prompt: Vixie cron, up to three lines, or `UNSUPPORTED`, `INCOMPLETE` or `INVALID`. Most of their TOPv2 misses are refusals of requests this model's labeling rules read as schedules. A second Haiku run with the same prompt scored 64.6% on TOPv2 and 61.8% on real text, so treat these scores as rough to 10 points or more. Claude also wrote the AI-written requests and helped label the data, which may favor them.

This model's errors on the AI-written column guided its training data, so its lead there is flattered.

## Limits

- English only.
- Accuracy depends on the kind of text:

  | Text | Wrong per 100 |
  |---|---|
  | Spoken requests to an assistant ("wake me at 7 on weekdays") | 4 to 6 |
  | Sentences from code, forums and Q&A sites | 12 |
  | Short entries typed into an app ("daily except Fridays", "every 4 hrs from 0200") | 14 |
  | Sentences from GitHub issues | 22 |

- The `95%` threshold is fitted on the dev set, and how far to trust it depends on the text. Above it, answers were right 97% to 98% of the time on spoken requests, 93% on sentences from code, forums and Q&A sites, 92% on short app entries and 85% on sentences from GitHub issues, counting `UNSUPPORTED` and `INVALID` as one answer.
- In sentences, the most common mistake is the class: a schedule answered with a class, or a class answered with a schedule. The next is one wrong field. In short requests, one wrong field comes first, then the class.
- Weakest on different times on different days in one request, requests with a detail missing (`INCOMPLETE`), several minutes past the hour, hour windows ("9 to 5"), lists of hours, days by exclusion and days of the month.
- Reads one request at a time, not a whole paragraph.

## Porting to another language

`to_cron.py` is the whole recipe. The steps:

0. Optional: if the text is already a cron line (`*/5 * * * *`, `@daily`), answer with it as `rules.py` does and skip the model.
1. Cut the text to 256 characters. If only whitespace is left, answer `INVALID`.
2. Tokenize with `tokenizer.json` (no special tokens). Give every character the id of the token that covers it, 0 for none. Offsets must count Unicode code points: convert Rust byte offsets and JavaScript UTF-16 offsets first.
3. Run `model.onnx` with two int64 arrays of shape (1, length): `chars` (code points) and `tokens` (ids from step 2).
4. Read the outputs: `class_index` (index into `classes` in `config.json`; 0 is a schedule), `line_count` (1 to 3), one boolean array per field (`minutes`, `hours`, `doms`, `months`, `dows`, shape (1, 3, field size); index i means the field's offset + i), `day_or`, `confidence` (shape (1,)), `pairs` (shape (1, 3, 24, 60): whether hour h fires at minute m) and `hour_days` (shape (1, 3, 24, 38): for hour h, the first 7 values are weekdays from Sunday and the last 31 are days of the month from the 1st).
5. For a schedule, read the first `line_count` slots. In each slot, for each hour that is on, take the minutes that are on and that `pairs` marks for it; an hour with none is dropped. Give the hour the weekdays `hour_days` marks for it, or the slot's `dows` if it marks none, and the days of the month it marks, or the slot's `doms` if it marks none. Hours with the same minutes and days share a line. If every hour is dropped, the slot is one line of all its hours and minutes. Drop repeated lines, then sort by first hour, then first minute, then the line's text. More than 3 lines is `UNSUPPORTED`. Write a field as `*` when every value is on, `*/n` when the values are every n-th from the field's lowest value and n is at most half the field's size, otherwise the values, with runs of 3 or more as ranges (`8,9` but `8-10`). In the two day fields write a step as `first-last/n`, never `*/n`. If both day fields are restricted and `day_or` is false, answer `UNSUPPORTED`.
6. Check the port against `examples.jsonl`.

Run one text at a time, as the scores above were. In a batch the texts share the model's quantization scale, which changed 1 answer in 1,019 when tested.

## Model

| | |
|---|---|
| Architecture | Character-level bidirectional GRU with pretrained word piece vectors from `cross-encoder/ms-marco-MiniLM-L-6-v2` |
| Parameters | 1.9 million, 0.5 million of them the word piece table |
| Size | 1.6 MB, and 0.3 MB of tokenizer |
| Weights | Word piece table 3-bit, GRU 5-bit, output layers 6-bit, character table 8-bit; unpacked when the model loads |
| Format | ONNX, opset 20 |
| Input | English text, up to 256 characters |
| Output | One to three cron lines or a class, with a confidence |
| Runs on | CPU, offline |

## Training data

About 200,000 labeled rows, plus about 56,000 texts labeled by earlier models of this design. Real text comes from:

- [Stack Exchange](https://archive.org/details/stackexchange): android, apple, askubuntu.com, dba, devops, drupal, magento, pm, raspberrypi, salesforce, serverfault.com, sharepoint, sqa, superuser.com, unix, webapps, webmasters, wordpress and Stack Overflow ([CC BY-SA 2.5](https://creativecommons.org/licenses/by-sa/2.5/), [3.0](https://creativecommons.org/licenses/by-sa/3.0/) or [4.0](https://creativecommons.org/licenses/by-sa/4.0/), by post date);
- [TOPv2](https://dl.fbaipublicfiles.com/topv2/TOPv2_Dataset.zip) (Chen et al. 2020, [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/));
- [Amazon MASSIVE 1.1](https://github.com/alexa/massive) ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/));
- [SMCalFlow 2.0](https://github.com/microsoft/task_oriented_dialogue_as_dataflow_synthesis) ([CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)) and [PRESTO](https://github.com/google-research-datasets/presto) ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/));
- [MTOP](https://fb.me/mtop_dataset) ([CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)), [Multilingual Task-Oriented Dialog](https://fb.me/multilingual_task_oriented_data) ([CC BY-SA](https://creativecommons.org/licenses/by-sa/4.0/)), [SLURP](https://github.com/pswietojanski/slurp) ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) and [HWU64](https://github.com/xliuhw/NLU-Evaluation-Data) ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/));
- code comments next to cron lines in public repositories, found with Sourcegraph.

The rest is generated: template requests, and requests and developer prose written by an AI model, checked by blind readings. `microsoft/deberta-v3-xsmall` (MIT) served as a training teacher and does not ship with the model. `cross-encoder/ms-marco-MiniLM-L-6-v2` (Apache-2.0) also served as a teacher, and its word piece vectors and a cut-down copy of its tokenizer ship in `model.onnx` and `tokenizer.json`. No dataset under a non-commercial license was used for this model's training rows.

## Changes

- 1.0.0: first release.
- 1.0.1: the runtime uses one thread, which is faster on a laptop CPU. Same model.
- 1.1.0: weights packed to 3, 5 and 6 bits and unused tokenizer pieces dropped: 3.2 MB to 1.5 MB. Same answers on the test vectors; scores move by at most 0.7 points.
- 1.2.0: retrained on app-style requests and prose that names times without asking for a schedule. Same size, speed and test vector answers. Held-out app-style requests 66.9% to 73.9% (counting `UNSUPPORTED` and `INVALID` as one answer); other scores move by at most 1.6 points.
- 1.3.0: new weights, trained on more short requests, with new heads for minute steps and for giving each hour its own days. A port must read the new `hour_days` output (steps 4 and 5). 1.6 MB, up from 1.5 MB, and 2.3 ms, up from 2.0 ms. `rules.py` now only reads text that is already a cron line: it no longer fills gaps in the model's hours. Scores against 1.2.0: template 96.0% to 98.4%, AI-written 90.1% to 95.2%, tricky 96.1% to 97.4%, real text 86.5% to 87.9%, GitHub 74.8% to 77.6% (1.2 points of it from corrected labels), TOPv2 level at 96.4%, assistant 94.5% to 94.2% (two rows). Held-out app-style requests 73.9% to 85.9%, counted as in 1.2.0. Answers above the `95%` threshold are a little less reliable on long prose and spoken requests, and more reliable on short app entries. The test vector "remind me sometime" is now `UNSUPPORTED`, was `INVALID`.

## License

Apache-2.0 for the model and the code in this folder. Training data credits are above.

	---
	license: apache-2.0
	language:
	- en
	library_name: onnx
	pipeline_tag: text-classification
	datasets:
	- AmazonScience/massive
	metrics:
	- exact_match
	tags:
	- cron
	- crontab
	- text-to-cron
	- scheduling
	- natural-language
	- time-expressions
	- structured-prediction
	- linux
	- devops
	- gru
	- character-level
	- int8
	- cpu
	- offline
	model-index:
	- name: tinycron
	results:
	- task:
	type: text-classification
	name: Text to cron
	dataset:
	name: Template requests (held out)
	type: tinycron-template
	metrics:
	- type: exact_match
	value: 0.984
	name: Exact match
	- task:
	type: text-classification
	name: Text to cron
	dataset:
	name: Requests written by an AI model (not trained on; errors guided training data)
	type: tinycron-ai
	metrics:
	- type: exact_match
	value: 0.952
	name: Exact match
	- task:
	type: text-classification
	name: Text to cron
	dataset:
	name: TOPv2 alarm and reminder requests (test)
	type: topv2
	metrics:
	- type: exact_match
	value: 0.964
	name: Exact match
	- task:
	type: text-classification
	name: Text to cron
	dataset:
	name: Real assistant requests from five datasets (held out)
	type: tinycron-assistant
	metrics:
	- type: exact_match
	value: 0.942
	name: Exact match
	- task:
	type: text-classification
	name: Text to cron
	dataset:
	name: Fresh requests written by an AI model (held out)
	type: tinycron-fresh
	metrics:
	- type: exact_match
	value: 0.939
	name: Exact match
	- task:
	type: text-classification
	name: Text to cron
	dataset:
	name: Hand-written tricky requests (not trained on; errors guided training data)
	type: tinycron-probes
	metrics:
	- type: exact_match
	value: 0.974
	name: Exact match
	- task:
	type: text-classification
	name: Text to cron
	dataset:
	name: Real text from code, forums and Q&A sites (held out)
	type: tinycron-real-text
	metrics:
	- type: exact_match
	value: 0.879
	name: Exact match
	- task:
	type: text-classification
	name: Text to cron
	dataset:
	name: GitHub issue sentences (never trained on; errors guided training data)
	type: tinycron-github
	metrics:
	- type: exact_match
	value: 0.776
	name: Exact match
	---

	# tinycron

	Turns an English schedule into a Linux cron line.

	\| Input \| Output \|
	\|---\|---\|
	\| every weekday at 9am \| `0 9 * * 1-5` \|
	\| Every Monday At 5:30 PM \| `30 17 * * 1` \|
	\| every 15 minutes between 9 and 5 \| `/15 9-17 * *` \|
	\| twice a month \| `0 0 1,15 * *` \|
	\| nightly, or twice a day \| `0 /12 * *` \|
	\| every other tuesday \| `UNSUPPORTED` \|
	\| every few hours \| `INCOMPLETE` \|

	If cron can't express a request, the answer is a class instead:

	- `UNSUPPORTED`: cron can't express it (seconds, "every other week", "the last day of the month").
	- `INCOMPLETE`: a time or count is missing.
	- `INVALID`: not a schedule.

	Each answer has a confidence from 0 to 1. The model is 1.6 MB (1.9 MB with its tokenizer) and takes 2 to 4 ms on a desktop CPU.

	## Use

	```bash
	pip install -r requirements.txt
	python to_cron.py "every weekday at 9am"
	python to_cron.py --json < requests.txt
	```

	```python
	from to_cron import CronModel
	model = CronModel()
	model.to_cron("every weekday at 9am") # "0 9 * * 1-5"
	model.read("at 7am on weekdays and 9am on weekends")
	# ("schedule", ["0 7 * * 1-5", "0 9 * * 0,6"], 0.99...)
	```

	The command marks an answer `(unsure, check it)` when its confidence is under the `95%` threshold in `config.json`. Answers above that threshold were right 95% of the time on the dev set, counting `UNSUPPORTED` and `INVALID` as one answer.

	Tested with Python 3.12 and onnxruntime 1.19, and with onnxruntime-web 1.20.1 (WebAssembly), which gives the same answers. Needs onnxruntime 1.19 or newer. Loading the model takes about 0.5 seconds, once.

	## Files

	\| File \| What it is \|
	\|---\|---\|
	\| `model.onnx` \| The model \|
	\| `tokenizer.json` \| The tokenizer \|
	\| `config.json` \| Version, file hashes, field layout, confidence thresholds \|
	\| `to_cron.py` \| Python runner, as a command or a module \|
	\| `rules.py` \| Exact parser for text that is already a cron line (optional) \|
	\| `requirements.txt` \| Python packages for `to_cron.py` \|
	\| `examples.jsonl` \| Test vectors for checking a port \|

	## Output

	- One to three standard 5-field cron lines (Vixie cron and cronie; Sunday is 0).
	- Or `UNSUPPORTED`, `INCOMPLETE` or `INVALID`.

	Never seconds, years, `L`, `W`, `#` or `@` shortcuts.

	Defaults for vague requests:

	- No time means midnight: "every Monday" is `0 0 * * 1`.
	- "Morning" is 9:00. "Every night at 7" is 19:00.
	- A bare hour from 1 to 6 is afternoon: "at 3" is 15:00.
	- "Weekly" is Sunday at midnight. "Monthly" is the 1st at midnight.
	- "Between 9 and 5" includes the last hour: `9-17`.
	- Time zones are ignored.

	## Accuracy

	Exact match. None of these rows are in the training data.

	\| Text \| Rows \| Accuracy \| Wrong per 1,000 \|
	\|---\|---\|---\|---\|
	\| Template requests \| 2,875 \| 98.4% \| 16 \|
	\| Real alarm and reminder requests (TOPv2) \| 1,510 \| 96.4% \| 36 \|
	\| Real assistant requests from five more datasets \| 745 \| 94.2% \| 58 \|
	\| Fresh requests written by an AI model \| 949 \| 93.9% \| 61 \|
	\| Real text from code, forums and Q&A sites \| 2,978 \| 87.9% \| 121 \|

	Errors on these sets guided the training data, so they flatter this model:

	\| Text \| Rows \| Accuracy \| Wrong per 1,000 \|
	\|---\|---\|---\|---\|
	\| Requests written by an AI model \| 1,762 \| 95.2% \| 48 \|
	\| Hand-written tricky requests \| 155 \| 97.4% \| 26 \|
	\| GitHub issue sentences, never trained on \| 2,699 \| 77.6% \| 224 \|

	Other tools on the same rows, counting only rows whose answer is a schedule, scored leniently:

	\| System \| Template \| AI-written \| Real requests (TOPv2) \| Real text \|
	\|---\|---\|---\|---\|---\|
	\| english-to-cron 0.1.7 (Rust rules) \| 23.3% \| 29.3% \| 49.5% \| 55.8% \|
	\| natural-cron-expression 1.0.2 (PHP rules) \| 16.3% \| 21.7% \| 56.8% \| 61.7% \|
	\| [cronformer](https://huggingface.co/uncommonstash/cronformer) 0.1.0 (6.3M parameters, 25 MB) \| 43.2% \| 41.6% \| 70.3% \| 64.8% \|
	\| [cronlm](https://huggingface.co/pavstev/cronlm) (16M parameters, 24 MB) \| 43.0% \| 45.0% \| 71.0% \| 45.9% \|
	\| [cron-mini](https://huggingface.co/Shpigford/cron-mini) (Qwen2.5-1.5B fine-tune, 3.1 GB)\* \| 43.7% \| 44.6% \| 66.7% \| 61.0% \|
	\| [leom21/cron](https://huggingface.co/leom21/cron) (SmolLM2-360M fine-tune, 724 MB)\* \| 11.1% \| 9.1% \| 36.5% \| 24.4% \|
	\| Claude Haiku (October 2026), prompted\*† \| 80.2% \| 76.9% \| 46.9% \| 71.5% \|
	\| Claude Sonnet (October 2026), prompted\*† \| 78.6% \| 78.5% \| 66.7% \| 79.7% \|
	\| Claude Opus (October 2026), prompted\*† \| 81.7% \| 81.8% \| 66.7% \| 78.9% \|
	\| this model (1.9M parameters, 1.6 MB) \| 98.6% \| 94.8% \| 96.7% \| 90.9% \|

	\* The language models are too slow for whole splits, so they ran on a seeded sample of 150 rows per column (96 to 126 of them schedules). On the same sample this model scores 98.4%, 93.4%, 95.8% and 91.9%. leom21/cron has no documented prompt; it got "Convert to cron: " before the text, the best of the prompts tried.

	† The Claude models got the same one-paragraph prompt: Vixie cron, up to three lines, or `UNSUPPORTED`, `INCOMPLETE` or `INVALID`. Most of their TOPv2 misses are refusals of requests this model's labeling rules read as schedules. A second Haiku run with the same prompt scored 64.6% on TOPv2 and 61.8% on real text, so treat these scores as rough to 10 points or more. Claude also wrote the AI-written requests and helped label the data, which may favor them.

	This model's errors on the AI-written column guided its training data, so its lead there is flattered.

	## Limits

	- English only.
	- Accuracy depends on the kind of text:

	\| Text \| Wrong per 100 \|
	\|---\|---\|
	\| Spoken requests to an assistant ("wake me at 7 on weekdays") \| 4 to 6 \|
	\| Sentences from code, forums and Q&A sites \| 12 \|
	\| Short entries typed into an app ("daily except Fridays", "every 4 hrs from 0200") \| 14 \|
	\| Sentences from GitHub issues \| 22 \|

	- The `95%` threshold is fitted on the dev set, and how far to trust it depends on the text. Above it, answers were right 97% to 98% of the time on spoken requests, 93% on sentences from code, forums and Q&A sites, 92% on short app entries and 85% on sentences from GitHub issues, counting `UNSUPPORTED` and `INVALID` as one answer.
	- In sentences, the most common mistake is the class: a schedule answered with a class, or a class answered with a schedule. The next is one wrong field. In short requests, one wrong field comes first, then the class.
	- Weakest on different times on different days in one request, requests with a detail missing (`INCOMPLETE`), several minutes past the hour, hour windows ("9 to 5"), lists of hours, days by exclusion and days of the month.
	- Reads one request at a time, not a whole paragraph.

	## Porting to another language

	`to_cron.py` is the whole recipe. The steps:

	0. Optional: if the text is already a cron line (`/5 * * *`, `@daily`), answer with it as `rules.py` does and skip the model.
	1. Cut the text to 256 characters. If only whitespace is left, answer `INVALID`.
	2. Tokenize with `tokenizer.json` (no special tokens). Give every character the id of the token that covers it, 0 for none. Offsets must count Unicode code points: convert Rust byte offsets and JavaScript UTF-16 offsets first.
	3. Run `model.onnx` with two int64 arrays of shape (1, length): `chars` (code points) and `tokens` (ids from step 2).
	4. Read the outputs: `class_index` (index into `classes` in `config.json`; 0 is a schedule), `line_count` (1 to 3), one boolean array per field (`minutes`, `hours`, `doms`, `months`, `dows`, shape (1, 3, field size); index i means the field's offset + i), `day_or`, `confidence` (shape (1,)), `pairs` (shape (1, 3, 24, 60): whether hour h fires at minute m) and `hour_days` (shape (1, 3, 24, 38): for hour h, the first 7 values are weekdays from Sunday and the last 31 are days of the month from the 1st).
	5. For a schedule, read the first `line_count` slots. In each slot, for each hour that is on, take the minutes that are on and that `pairs` marks for it; an hour with none is dropped. Give the hour the weekdays `hour_days` marks for it, or the slot's `dows` if it marks none, and the days of the month it marks, or the slot's `doms` if it marks none. Hours with the same minutes and days share a line. If every hour is dropped, the slot is one line of all its hours and minutes. Drop repeated lines, then sort by first hour, then first minute, then the line's text. More than 3 lines is `UNSUPPORTED`. Write a field as `` when every value is on, `/n` when the values are every n-th from the field's lowest value and n is at most half the field's size, otherwise the values, with runs of 3 or more as ranges (`8,9` but `8-10`). In the two day fields write a step as `first-last/n`, never `*/n`. If both day fields are restricted and `day_or` is false, answer `UNSUPPORTED`.
	6. Check the port against `examples.jsonl`.

	Run one text at a time, as the scores above were. In a batch the texts share the model's quantization scale, which changed 1 answer in 1,019 when tested.

	## Model

	\| \| \|
	\|---\|---\|
	\| Architecture \| Character-level bidirectional GRU with pretrained word piece vectors from `cross-encoder/ms-marco-MiniLM-L-6-v2` \|
	\| Parameters \| 1.9 million, 0.5 million of them the word piece table \|
	\| Size \| 1.6 MB, and 0.3 MB of tokenizer \|
	\| Weights \| Word piece table 3-bit, GRU 5-bit, output layers 6-bit, character table 8-bit; unpacked when the model loads \|
	\| Format \| ONNX, opset 20 \|
	\| Input \| English text, up to 256 characters \|
	\| Output \| One to three cron lines or a class, with a confidence \|
	\| Runs on \| CPU, offline \|

	## Training data

	About 200,000 labeled rows, plus about 56,000 texts labeled by earlier models of this design. Real text comes from:

	- [Stack Exchange](https://archive.org/details/stackexchange): android, apple, askubuntu.com, dba, devops, drupal, magento, pm, raspberrypi, salesforce, serverfault.com, sharepoint, sqa, superuser.com, unix, webapps, webmasters, wordpress and Stack Overflow ([CC BY-SA 2.5](https://creativecommons.org/licenses/by-sa/2.5/), [3.0](https://creativecommons.org/licenses/by-sa/3.0/) or [4.0](https://creativecommons.org/licenses/by-sa/4.0/), by post date);
	- [TOPv2](https://dl.fbaipublicfiles.com/topv2/TOPv2_Dataset.zip) (Chen et al. 2020, [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/));
	- [Amazon MASSIVE 1.1](https://github.com/alexa/massive) ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/));
	- [SMCalFlow 2.0](https://github.com/microsoft/task_oriented_dialogue_as_dataflow_synthesis) ([CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)) and [PRESTO](https://github.com/google-research-datasets/presto) ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/));
	- [MTOP](https://fb.me/mtop_dataset) ([CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)), [Multilingual Task-Oriented Dialog](https://fb.me/multilingual_task_oriented_data) ([CC BY-SA](https://creativecommons.org/licenses/by-sa/4.0/)), [SLURP](https://github.com/pswietojanski/slurp) ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) and [HWU64](https://github.com/xliuhw/NLU-Evaluation-Data) ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/));
	- code comments next to cron lines in public repositories, found with Sourcegraph.

	The rest is generated: template requests, and requests and developer prose written by an AI model, checked by blind readings. `microsoft/deberta-v3-xsmall` (MIT) served as a training teacher and does not ship with the model. `cross-encoder/ms-marco-MiniLM-L-6-v2` (Apache-2.0) also served as a teacher, and its word piece vectors and a cut-down copy of its tokenizer ship in `model.onnx` and `tokenizer.json`. No dataset under a non-commercial license was used for this model's training rows.

	## Changes

	- 1.0.0: first release.
	- 1.0.1: the runtime uses one thread, which is faster on a laptop CPU. Same model.
	- 1.1.0: weights packed to 3, 5 and 6 bits and unused tokenizer pieces dropped: 3.2 MB to 1.5 MB. Same answers on the test vectors; scores move by at most 0.7 points.
	- 1.2.0: retrained on app-style requests and prose that names times without asking for a schedule. Same size, speed and test vector answers. Held-out app-style requests 66.9% to 73.9% (counting `UNSUPPORTED` and `INVALID` as one answer); other scores move by at most 1.6 points.
	- 1.3.0: new weights, trained on more short requests, with new heads for minute steps and for giving each hour its own days. A port must read the new `hour_days` output (steps 4 and 5). 1.6 MB, up from 1.5 MB, and 2.3 ms, up from 2.0 ms. `rules.py` now only reads text that is already a cron line: it no longer fills gaps in the model's hours. Scores against 1.2.0: template 96.0% to 98.4%, AI-written 90.1% to 95.2%, tricky 96.1% to 97.4%, real text 86.5% to 87.9%, GitHub 74.8% to 77.6% (1.2 points of it from corrected labels), TOPv2 level at 96.4%, assistant 94.5% to 94.2% (two rows). Held-out app-style requests 73.9% to 85.9%, counted as in 1.2.0. Answers above the `95%` threshold are a little less reliable on long prose and spoken requests, and more reliable on short app entries. The test vector "remind me sometime" is now `UNSUPPORTED`, was `INVALID`.

	## License

	Apache-2.0 for the model and the code in this folder. Training data credits are above.