Spaces:
Running
Running
data-line commited on
Commit ·
975e4ee
1
Parent(s): bec7df7
portal: catalog page of repository commit ae5043d3f133
Browse filesFiles of catalog/portal/ (18), digest 5defae89560cdd5321dbce69717fc621ee175790ca7b87b2fa198a863ea3b875.
- README.md +59 -17
- app.js +73 -7
- data.js +6 -0
- first.js +6 -3
- index.html +22 -8
- llms.txt +1 -1
- locales/en.json +41 -16
- locales/zh.json +57 -33
- manual.js +59 -0
- manual/manual.en.md +142 -0
- manual/manual.zh.md +142 -0
- manual/shots.yml +42 -0
- portal.json +3 -3
- style.css +23 -2
README.md
CHANGED
|
@@ -16,7 +16,7 @@ datasets:
|
|
| 16 |
A static page into the public pointer index [Battam/agentic-env-index](https://huggingface.co/datasets/Battam/agentic-env-index)
|
| 17 |
for people who build tasks for a terminal agent, and for their agent: task material (material sets in their
|
| 18 |
collections) and corpora as two layers, one card per collection, the largest material sets, the most requested
|
| 19 |
-
|
| 20 |
filters the others and the list), the material sets or corpora with their fit and a preview of their rows, and one
|
| 21 |
command per layer to take a sample.
|
| 22 |
There is no backend; the page reads the dataset in the browser. Agents read [llms.txt](llms.txt).
|
|
@@ -47,6 +47,10 @@ The index's dataset card and `docs/sources.md` keep the two layers in separate t
|
|
| 47 |
|
| 48 |
One pair of names is used everywhere: "corpus" and "corpora", "material set" and "material sets"
|
| 49 |
(in Chinese the two names that `locales/zh.json` has for them, `kind.corpus` and `kind.material_set`).
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
No number and no heading counts "sources" over both layers.
|
| 51 |
|
| 52 |
Every number says which layer it counts: task material in collections, material sets, units and units with an answer file;
|
|
@@ -56,12 +60,12 @@ When `meta/catalog.parquet` has the columns `units` (distinct units; a corpus co
|
|
| 56 |
an answer file) and `files` (files of every role), the page reads them as they are.
|
| 57 |
Until then a material set's units are its answer files when it has any (answers kept apart), else its unit rows (tests
|
| 58 |
and worked examples hold their expected results inside).
|
| 59 |
-
A
|
| 60 |
-
rows the
|
| 61 |
the two are never added.
|
| 62 |
-
"
|
| 63 |
-
or conditional (`license_rows.redistribution_ok`), weighted by the units of material sets ("
|
| 64 |
-
or by the rows of corpora when a
|
| 65 |
A collection card shows the units of its material sets, its material sets, its units with an answer file and its main
|
| 66 |
domains;
|
| 67 |
a count of 0 is left out, and the card carries no collection code (#1197: codes stay where someone types them, in the
|
|
@@ -90,7 +94,10 @@ the recorded size (`bytes`, the sum of `size_bytes` over the rows that have one)
|
|
| 90 |
the deduplicated size (`distinct_bytes`, the recorded size minus the sizes of the rows that repeat another row's
|
| 91 |
content);
|
| 92 |
the check (`size_check`: of the rows read from `meta/sizes.parquet` that compared a size or hash, `compared`, the
|
| 93 |
-
share that match their recorded size, "Size check: 98.8% match (3,413 rows compared)", #1225
|
|
|
|
|
|
|
|
|
|
| 94 |
and, for the rows without a recorded size (`unsized_rows`), their estimate (`unsized_estimate`), marked as an estimate,
|
| 95 |
with its 95% range.
|
| 96 |
Above a source the estimate is the sum of its sources' estimates and its range the sum of their ranges, which is wider
|
|
@@ -132,7 +139,8 @@ The first screen (`first.js`) and the reader of the index (`data.js`) share that
|
|
| 132 |
|
| 133 |
| File | Read | Fields used |
|
| 134 |
|---|---|---|
|
| 135 |
-
| `meta/catalog.parquet` | at load; then loaded whole into DuckDB-WASM as the table `catalog` | `source`, `title`, `summary`, `description`, `status`, `status_reasons`, `kind` (when published), `collection` (when published), `domain`, `form` (when published), `license` (`spdx`, `tier`, `note`, `ship`), `contamination` (`level`, `benchmark`, `scope`; level `not scanned` reads as not known), `pii` (`level`, `evidence`), `files`, `license_counts` (when published: per material set `kind`, `upstream` or `index` for the files the index wrote under `campaign_files/`, `spdx`, `training_ok`, `redistribution_ok` and `files`, counted by the projection over all its rows), `bytes`, `distinct_bytes` (when published), `unsized_rows`, `unit`, `top_types`, `needs_auth` |
|
|
|
|
| 136 |
| `meta/size_figures.json` | at load, when the commit has it | per level (`index`, `layers`, `collections`, `sources`): `bytes`, `distinct_bytes`, `unsized_rows`, `check` (`rows`, `compared`, `match`, `unreachable`, `gone`, `needs_auth`, `no_length`, `error`), `estimate` (`bytes`, `low`, `high`, `readings`, `rows`) |
|
| 137 |
| `meta/sources.json` | at load | per source: `license_rows` (training and redistribution, rows per answer), `license` (`name`, `statement`, `attribution`, `url`), `by_ext`, `size_quantiles_upper_bytes` |
|
| 138 |
| `meta/coverage.json` | at load | `by_license.training_ok` (rows per answer) |
|
|
@@ -201,7 +209,7 @@ the other filters (leaf, form, status, training, redistribution, size, file type
|
|
| 201 |
`details` element closed by default and open when one of them is set (#1197).
|
| 202 |
The panel has a `max-height` with its own scroll, so it never runs past the viewport.
|
| 203 |
The filters are the clauses of one crossfilter selection (Mosaic, `linked.js`), whatever sets them: the filter panel, a
|
| 204 |
-
click on a bar of the domain or form chart, a click on a map cell (domain and form), a
|
| 205 |
or the layer switch.
|
| 206 |
Each chart is filtered by every clause but its own (a click on a domain bar filters the map, the form bars and the
|
| 207 |
source list, and the domain bars stay whole); the source list and "Get the data" read every clause.
|
|
@@ -217,6 +225,9 @@ there), most rows first, as CKAN's Formats facet and the Hub's Format filter do;
|
|
| 217 |
the keys in brackets (`(none)`, `(other)`) name no extension and are not listed.
|
| 218 |
Selecting types keeps the sources with rows of any of them; the command adds one `--ext <type>` per selected type, the
|
| 219 |
SQL tab `WHERE ext IN (...)`, and the CSV keeps only the rows of those types.
|
|
|
|
|
|
|
|
|
|
| 220 |
|
| 221 |
## Languages
|
| 222 |
|
|
@@ -248,7 +259,9 @@ first, `:lang(zh)` in `style.css`).
|
|
| 248 |
The detector reads the query string first (`?lng=zh`), then localStorage, and keeps the choice in localStorage
|
| 249 |
(`i18nextLng`), so the page opens in English unless the reader chose Chinese;
|
| 250 |
the switch drops a `lng` from the address, so the kept choice holds on the next visit.
|
| 251 |
-
`llms.txt` and this README stay in English
|
|
|
|
|
|
|
| 252 |
|
| 253 |
## Files
|
| 254 |
|
|
@@ -271,6 +284,15 @@ the switch drops a `lng` from the address, so the kept choice holds on the next
|
|
| 271 |
- `i18n.js`: the page's languages (i18next with its browser language detector, the words from `locales/en.json` and
|
| 272 |
`locales/zh.json`), the words of `index.html` (`data-i18n` sets an element's text, `data-i18n-attr` its attributes)
|
| 273 |
and the language switch; the other modules draw their text again on i18next's `languageChanged`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 274 |
- `linked.js`: the linked views of the browse section, by Mosaic vgplot: DuckDB-WASM through `wasmConnector()`, the
|
| 275 |
tables `catalog`, `page` and `src` and the view `dom`, the domain × form map and the bars by domain and by form, of
|
| 276 |
the layer the clause `kind` selects, in its color (material sets `#4F6D80`, corpora `#8C8C8C`): the bars with the
|
|
@@ -301,7 +323,7 @@ the switch drops a `lng` from the address, so the kept choice holds on the next
|
|
| 301 |
material sets and its units with an answer file side by side at one size, named as on the first screen, its units in
|
| 302 |
one small line under them; its main domains wrap to at most two lines, with an ellipsis and the full text as the
|
| 303 |
title), the largest material sets as
|
| 304 |
-
bars (the rest in one line under the chart and its table, never as a bar), the most requested
|
| 305 |
`MOST_REQUESTED`, ten: one card each, in two rows of five from 1,100 px; those with material by size, then the
|
| 306 |
wanted ones with a request; `index.html` has ten skeleton cards to match), the filters and their URL state (`F`), the
|
| 307 |
chips of the selection, the sections, the fit of a source, the command, the SQL and the CSV.
|
|
@@ -315,7 +337,11 @@ the switch drops a `lng` from the address, so the kept choice holds on the next
|
|
| 315 |
#1302: an address in `uri` is cut in its middle, its host at the start and its file name at the end, with the whole
|
| 316 |
address on hover, and the table's caption is in the page's language only;
|
| 317 |
each fit dot names its check and its state on hover (`title` and `aria-label`), and the header of the fit column
|
| 318 |
-
names the
|
|
|
|
|
|
|
|
|
|
|
|
|
| 319 |
the caret with the title, the kind mark (only when the list holds both layers) and the id start at the title's left
|
| 320 |
edge;
|
| 321 |
in a collection whose material sets all have one form, the form cells are left out (#1216).
|
|
@@ -340,8 +366,10 @@ the switch drops a `lng` from the address, so the kept choice holds on the next
|
|
| 340 |
a picture of what its output folder will hold (`commandArgs`, `layerArgs`, `selectionArgs` and `folderTree`).
|
| 341 |
Beside it, two downloads keep the selection: the selection file (`agentic-env-selection-<commit>.json`: the ids of
|
| 342 |
the sources, the filters with the layer and the same release tag as its revision), which `fetch.py --selection`
|
| 343 |
-
reads, with the command that reads it pinned to the same tag; and the attribution file (Markdown, in English: per source its title, license
|
| 344 |
-
attribution statement from `meta/sources.json`
|
|
|
|
|
|
|
| 345 |
#1286: each row of the list, in either layer, has a "Pick" toggle that reads "Picked" once on;
|
| 346 |
the picks are kept in the address as `pick=`, as the filters are.
|
| 347 |
While at least one source is picked, "Get the data" says how many and shows, for exactly them, the same "Look first"
|
|
@@ -394,8 +422,9 @@ Fonts: Inter (400 to 800) for text and numbers, Noto Sans SC for CJK, IBM Plex M
|
|
| 394 |
Motion (update highlights, sections fading in, cards lifting, view transitions) is off under `prefers-reduced-motion`.
|
| 395 |
|
| 396 |
Libraries come from cdn.jsdelivr.net, pinned: `hyparquet@1.31.5` and `hyparquet-compressors@1.1.2` (the meta files
|
| 397 |
-
are zstd-compressed), `i18next@26.4.5` and `i18next-browser-languagedetector@8.2.1`,
|
| 398 |
-
(Mosaic, an ES module, loaded after the first screen has painted; it brings DuckDB-WASM from jsDelivr)
|
|
|
|
| 399 |
There is no build step. To look at it locally: `python -m http.server` in this folder (without a
|
| 400 |
`first.json` the hero shows skeletons until the index is read; `03_data_line/steps/portal.py --write-first first.json`
|
| 401 |
writes one).
|
|
@@ -408,4 +437,17 @@ Space's `portal.json` records, after the push has passed its outbound checks. It
|
|
| 408 |
numbers, per layer and never added together (task material: collections, material sets, units, units with an answer file;
|
| 409 |
corpora: corpora and rows), and the size figures of the whole index and each layer (`sizes`, from
|
| 410 |
`meta/size_figures.json`, format `portal-first/3`), from the commit main of the index points to,
|
| 411 |
-
computed with the same rule as `data.js` (the catalog column `kind` when the commit has it).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
A static page into the public pointer index [Battam/agentic-env-index](https://huggingface.co/datasets/Battam/agentic-env-index)
|
| 17 |
for people who build tasks for a terminal agent, and for their agent: task material (material sets in their
|
| 18 |
collections) and corpora as two layers, one card per collection, the largest material sets, the most requested
|
| 19 |
+
domains, the domain × form map with the bars by domain and by form for one layer at a time (linked: a click on one
|
| 20 |
filters the others and the list), the material sets or corpora with their fit and a preview of their rows, and one
|
| 21 |
command per layer to take a sample.
|
| 22 |
There is no backend; the page reads the dataset in the browser. Agents read [llms.txt](llms.txt).
|
|
|
|
| 47 |
|
| 48 |
One pair of names is used everywhere: "corpus" and "corpora", "material set" and "material sets"
|
| 49 |
(in Chinese the two names that `locales/zh.json` has for them, `kind.corpus` and `kind.material_set`).
|
| 50 |
+
#1577: one word per concept in each language, the same as in the index's docs: a source is "source" (in Chinese the
|
| 51 |
+
word of `col.title`), its card the source card, a domain "domain" (in Chinese the word of `filter.domain`; never
|
| 52 |
+
"category"), and the redistribution permit "redistribution" (in Chinese the word of `chip.redistribution`; never
|
| 53 |
+
"shipping", which names the card field `license.ship`).
|
| 54 |
No number and no heading counts "sources" over both layers.
|
| 55 |
|
| 56 |
Every number says which layer it counts: task material in collections, material sets, units and units with an answer file;
|
|
|
|
| 60 |
an answer file) and `files` (files of every role), the page reads them as they are.
|
| 61 |
Until then a material set's units are its answer files when it has any (answers kept apart), else its unit rows (tests
|
| 62 |
and worked examples hold their expected results inside).
|
| 63 |
+
A domain card leads with the units of its material sets ("none yet" at 0) and says in a second line how many corpus
|
| 64 |
+
rows the domain also has to search;
|
| 65 |
the two are never added.
|
| 66 |
+
"Redistributable" is the share of rows whose redistribution is yes
|
| 67 |
+
or conditional (`license_rows.redistribution_ok`), weighted by the units of material sets ("Redistributable, material
|
| 68 |
+
sets"), or by the rows of corpora when a domain has no material set ("Redistributable, corpora").
|
| 69 |
A collection card shows the units of its material sets, its material sets, its units with an answer file and its main
|
| 70 |
domains;
|
| 71 |
a count of 0 is left out, and the card carries no collection code (#1197: codes stay where someone types them, in the
|
|
|
|
| 94 |
the deduplicated size (`distinct_bytes`, the recorded size minus the sizes of the rows that repeat another row's
|
| 95 |
content);
|
| 96 |
the check (`size_check`: of the rows read from `meta/sizes.parquet` that compared a size or hash, `compared`, the
|
| 97 |
+
share that match their recorded size, "Size check: 98.8% match (3,413 rows compared)", #1225; #1577: a level with
|
| 98 |
+
rows read but none compared shows its own reading, the rows read and what kept each from its file, "Size check: none
|
| 99 |
+
of the 9 rows read could be compared (address unreachable 9)", never "no row compared yet", which is left for a level
|
| 100 |
+
none of whose rows is read);
|
| 101 |
and, for the rows without a recorded size (`unsized_rows`), their estimate (`unsized_estimate`), marked as an estimate,
|
| 102 |
with its 95% range.
|
| 103 |
Above a source the estimate is the sum of its sources' estimates and its range the sum of their ranges, which is wider
|
|
|
|
| 139 |
|
| 140 |
| File | Read | Fields used |
|
| 141 |
|---|---|---|
|
| 142 |
+
| `meta/catalog.parquet` | at load; then loaded whole into DuckDB-WASM as the table `catalog` | `source`, `title`, `summary`, `description`, `status`, `status_reasons`, `kind` (when published), `collection` (when published), `domain`, `form` (when published), `license` (`spdx`, `tier`, `note`, `ship`), `contamination` (`level`, `benchmark`, `scope`; level `not scanned` reads as not known), `pii` (`level`, `evidence`), `files`, `license_counts` (when published: per material set `kind`, `upstream` or `index` for the files the index wrote under `campaign_files/`, `spdx`, `training_ok`, `redistribution_ok` and `files`, counted by the projection over all its rows), `bytes`, `distinct_bytes` (when published), `unsized_rows`, `fetch` (when published, #1577: `level`, `rows`, `fetched`, `read_at`), `unit`, `top_types`, `needs_auth` |
|
| 143 |
+
| `meta/attribution.parquet` | when the attribution file is made, when the commit has it (#1577) | `source`, `publisher`, `license`, `license_url`, `files` |
|
| 144 |
| `meta/size_figures.json` | at load, when the commit has it | per level (`index`, `layers`, `collections`, `sources`): `bytes`, `distinct_bytes`, `unsized_rows`, `check` (`rows`, `compared`, `match`, `unreachable`, `gone`, `needs_auth`, `no_length`, `error`), `estimate` (`bytes`, `low`, `high`, `readings`, `rows`) |
|
| 145 |
| `meta/sources.json` | at load | per source: `license_rows` (training and redistribution, rows per answer), `license` (`name`, `statement`, `attribution`, `url`), `by_ext`, `size_quantiles_upper_bytes` |
|
| 146 |
| `meta/coverage.json` | at load | `by_license.training_ok` (rows per answer) |
|
|
|
|
| 209 |
`details` element closed by default and open when one of them is set (#1197).
|
| 210 |
The panel has a `max-height` with its own scroll, so it never runs past the viewport.
|
| 211 |
The filters are the clauses of one crossfilter selection (Mosaic, `linked.js`), whatever sets them: the filter panel, a
|
| 212 |
+
click on a bar of the domain or form chart, a click on a map cell (domain and form), a domain card, a collection card
|
| 213 |
or the layer switch.
|
| 214 |
Each chart is filtered by every clause but its own (a click on a domain bar filters the map, the form bars and the
|
| 215 |
source list, and the domain bars stay whole); the source list and "Get the data" read every clause.
|
|
|
|
| 225 |
the keys in brackets (`(none)`, `(other)`) name no extension and are not listed.
|
| 226 |
Selecting types keeps the sources with rows of any of them; the command adds one `--ext <type>` per selected type, the
|
| 227 |
SQL tab `WHERE ext IN (...)`, and the CSV keeps only the rows of those types.
|
| 228 |
+
#1577: the SQL tab's query reads the rows as published and applies no rule of the default profile of `tools/fetch.py`
|
| 229 |
+
(no deduplication, and none of the benchmark, license, language, personal data and size rules);
|
| 230 |
+
a comment line in the query says so (`sql.noProfile`), so a copied query carries it.
|
| 231 |
|
| 232 |
## Languages
|
| 233 |
|
|
|
|
| 259 |
The detector reads the query string first (`?lng=zh`), then localStorage, and keeps the choice in localStorage
|
| 260 |
(`i18nextLng`), so the page opens in English unless the reader chose Chinese;
|
| 261 |
the switch drops a `lng` from the address, so the kept choice holds on the next visit.
|
| 262 |
+
`llms.txt` and this README stay in English;
|
| 263 |
+
the manual has a text of its own per language (`manual/manual.en.md`, `manual/manual.zh.md`), and the view shows the one of the
|
| 264 |
+
page's language.
|
| 265 |
|
| 266 |
## Files
|
| 267 |
|
|
|
|
| 284 |
- `i18n.js`: the page's languages (i18next with its browser language detector, the words from `locales/en.json` and
|
| 285 |
`locales/zh.json`), the words of `index.html` (`data-i18n` sets an element's text, `data-i18n-attr` its attributes)
|
| 286 |
and the language switch; the other modules draw their text again on i18next's `languageChanged`.
|
| 287 |
+
- `manual/`, `manual.js` (#1617): the user manual for people who build data from the index, a view of its own at the
|
| 288 |
+
address `#manual` (the links "Manual" in the header, under the first screen and in the footer; `app.js` `showView`
|
| 289 |
+
shows it in place of the page, and any other address shows the page again).
|
| 290 |
+
Its master copy is here: `manual/manual.en.md` and `manual/manual.zh.md`, one text per language in the four parts of Diátaxis
|
| 291 |
+
(tutorial, how-to guides, reference, explanation), and `manual/shots.yml`, its screenshots in shot-scraper's `multi`
|
| 292 |
+
format (per shot `url`, `output`, `width`, `selector`, `javascript`, `wait_for`).
|
| 293 |
+
`manual.js` renders the text of the page's language with markdown-it (CommonMark), again when the language changes;
|
| 294 |
+
an image names its shot's `output` and is read from `manual/shots/`, and a shot with no image there yet shows its
|
| 295 |
+
name in its place.
|
| 296 |
- `linked.js`: the linked views of the browse section, by Mosaic vgplot: DuckDB-WASM through `wasmConnector()`, the
|
| 297 |
tables `catalog`, `page` and `src` and the view `dom`, the domain × form map and the bars by domain and by form, of
|
| 298 |
the layer the clause `kind` selects, in its color (material sets `#4F6D80`, corpora `#8C8C8C`): the bars with the
|
|
|
|
| 323 |
material sets and its units with an answer file side by side at one size, named as on the first screen, its units in
|
| 324 |
one small line under them; its main domains wrap to at most two lines, with an ellipsis and the full text as the
|
| 325 |
title), the largest material sets as
|
| 326 |
+
bars (the rest in one line under the chart and its table, never as a bar), the most requested domains (the constant
|
| 327 |
`MOST_REQUESTED`, ten: one card each, in two rows of five from 1,100 px; those with material by size, then the
|
| 328 |
wanted ones with a request; `index.html` has ten skeleton cards to match), the filters and their URL state (`F`), the
|
| 329 |
chips of the selection, the sections, the fit of a source, the command, the SQL and the CSV.
|
|
|
|
| 337 |
#1302: an address in `uri` is cut in its middle, its host at the start and its file name at the end, with the whole
|
| 338 |
address on hover, and the table's caption is in the page's language only;
|
| 339 |
each fit dot names its check and its state on hover (`title` and `aria-label`), and the header of the fit column
|
| 340 |
+
names the eight checks with an icon and a short label (`fitShort.*`), each above its dot;
|
| 341 |
+
#1577: the eighth, "Upstream serves the files", reads the catalog column `fetch`, the projection's reading of the
|
| 342 |
+
source's sampled rows in `meta/sizes.parquet` (`all` fits, `some` partly, `none` does not, `not read` is not known),
|
| 343 |
+
and the detail says it in a line of its own, "Fetch check", with the rows read, those that fetched and matched their
|
| 344 |
+
recorded size and hash, and the day read;
|
| 345 |
the caret with the title, the kind mark (only when the list holds both layers) and the id start at the title's left
|
| 346 |
edge;
|
| 347 |
in a collection whose material sets all have one form, the form cells are left out (#1216).
|
|
|
|
| 366 |
a picture of what its output folder will hold (`commandArgs`, `layerArgs`, `selectionArgs` and `folderTree`).
|
| 367 |
Beside it, two downloads keep the selection: the selection file (`agentic-env-selection-<commit>.json`: the ids of
|
| 368 |
the sources, the filters with the layer and the same release tag as its revision), which `fetch.py --selection`
|
| 369 |
+
reads, with the command that reads it pinned to the same tag; and the attribution file (Markdown, in English: per source its title, license and
|
| 370 |
+
attribution statement from `meta/sources.json`; #1577: and its files by publisher, license and license URL from
|
| 371 |
+
`meta/attribution.parquet`, each file's own values, its source's license and license URL where its row names none;
|
| 372 |
+
a source without rows there, or a commit without the table, gives its source-level license URL).
|
| 373 |
#1286: each row of the list, in either layer, has a "Pick" toggle that reads "Picked" once on;
|
| 374 |
the picks are kept in the address as `pick=`, as the filters are.
|
| 375 |
While at least one source is picked, "Get the data" says how many and shows, for exactly them, the same "Look first"
|
|
|
|
| 422 |
Motion (update highlights, sections fading in, cards lifting, view transitions) is off under `prefers-reduced-motion`.
|
| 423 |
|
| 424 |
Libraries come from cdn.jsdelivr.net, pinned: `hyparquet@1.31.5` and `hyparquet-compressors@1.1.2` (the meta files
|
| 425 |
+
are zstd-compressed), `i18next@26.4.5` and `i18next-browser-languagedetector@8.2.1`, `@uwdata/vgplot@0.34.3`
|
| 426 |
+
(Mosaic, an ES module, loaded after the first screen has painted; it brings DuckDB-WASM from jsDelivr), and
|
| 427 |
+
`markdown-it@15.0.3` (the manual's Markdown, loaded with `manual.js` the first time the manual is shown).
|
| 428 |
There is no build step. To look at it locally: `python -m http.server` in this folder (without a
|
| 429 |
`first.json` the hero shows skeletons until the index is read; `03_data_line/steps/portal.py --write-first first.json`
|
| 430 |
writes one).
|
|
|
|
| 437 |
numbers, per layer and never added together (task material: collections, material sets, units, units with an answer file;
|
| 438 |
corpora: corpora and rows), and the size figures of the whole index and each layer (`sizes`, from
|
| 439 |
`meta/size_figures.json`, format `portal-first/3`), from the commit main of the index points to,
|
| 440 |
+
computed with the same rule as `data.js` (the catalog column `kind` when the commit has it).
|
| 441 |
+
|
| 442 |
+
#1617: the manual's texts go to the Space with their markers filled.
|
| 443 |
+
`{first:<key>}` is a number of `first.json` (the same computation; a count with thousands separators), and
|
| 444 |
+
`{first:index}` the index commit the deploy reads.
|
| 445 |
+
A line `{run:<command>}` becomes the command after `$ ` with its output below it: the deploy runs the command (with
|
| 446 |
+
`{first:index}` filled in) against that commit, within a time limit.
|
| 447 |
+
A command that fails or runs out of time makes the deploy step red, and the Space keeps its previous manual.
|
| 448 |
+
After the push, a step of the same publish run (`03_data_line/steps/shots.py`) waits until the live page serves the
|
| 449 |
+
deploy and takes every shot of `manual/shots.yml` on it with `shot-scraper multi` (Playwright, Chromium), each shot
|
| 450 |
+
waiting for its element with `wait_for`;
|
| 451 |
+
the images go to the Space under `manual/shots/` in a commit of their own after the deploy's, through the same
|
| 452 |
+
outbound gate.
|
| 453 |
+
A shot that fails keeps its previous image and is named in that step's summary, and the step is red.
|
app.js
CHANGED
|
@@ -10,7 +10,7 @@ import {
|
|
| 10 |
} from './data.js';
|
| 11 |
import {
|
| 12 |
clear, columns, compactParts, el, figureWithToggle, fmtAnd, fmtBytes, fmtCompact, fmtDate, fmtInt, fmtOr, fmtPct,
|
| 13 |
-
fmtShort, hbars, hideTip, meter, segmented, stack, table, tableShown, vt, withTip,
|
| 14 |
} from './charts.js';
|
| 15 |
import { decorate, drawHero, markFromIndex, setNumber, sizeLine, sizeParts, sizeShort } from './first.js';
|
| 16 |
import { lang, onLanguage, setUpSwitch, t, tEn, translatePage } from './i18n.js';
|
|
@@ -34,6 +34,8 @@ const MAX_FILE = 20e6;
|
|
| 34 |
const MAX_LISTED = 30;
|
| 35 |
// the phone layout of style.css
|
| 36 |
const PHONE = '(max-width: 720px)';
|
|
|
|
|
|
|
| 37 |
const $ = (id) => document.getElementById(id);
|
| 38 |
|
| 39 |
// ---- plain names (codes only inside the SQL and the commands); the words are in the locale files, by code:
|
|
@@ -173,9 +175,19 @@ const FIT = [
|
|
| 173 |
{ k: 'alone', icon: 'doc' },
|
| 174 |
{ k: 'answers', icon: 'check' },
|
| 175 |
{ k: 'login', icon: 'link' },
|
|
|
|
|
|
|
| 176 |
];
|
| 177 |
const fitLabel = (k) => t(`fit.${k}`);
|
| 178 |
const fitWord = (state) => t(`fitWord.${state}`);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 179 |
function fitOf(s) {
|
| 180 |
const q = s.info.size_quantiles_upper_bytes || {};
|
| 181 |
const c = s.contamination?.level;
|
|
@@ -190,6 +202,7 @@ function fitOf(s) {
|
|
| 190 |
alone: kinds.every((k) => ['unit', 'file', 'API response'].includes(k)) ? 'yes' : 'part',
|
| 191 |
answers: s.isTask ? 'yes' : 'no',
|
| 192 |
login: s.needsAuth === 'no' ? 'yes' : s.needsAuth === 'mixed' ? 'part' : 'no',
|
|
|
|
| 193 |
};
|
| 194 |
}
|
| 195 |
function fitDot(state, label) {
|
|
@@ -1142,6 +1155,9 @@ function detail(s) {
|
|
| 1142 |
dl.append(item('size', t('detail.size'), el('span', { class: 'lic' }, el('b', {}, recorded), el('span', { class: 'small' }, amounts),
|
| 1143 |
rest.length ? el('span', { class: 'small size-parts' }, rest.join(' \u00b7 ')) : null),
|
| 1144 |
q.p50 ? sizeStrip(q) : null));
|
|
|
|
|
|
|
|
|
|
| 1145 |
// 9 one example file (#1197): the first unit file of its preview rows (meta/preview.parquet, where a material set's
|
| 1146 |
// unit files come first); a source with none there has no example
|
| 1147 |
const pv = previewOf(s);
|
|
@@ -1209,6 +1225,8 @@ function selectionSql(sel) {
|
|
| 1209 |
return [
|
| 1210 |
`-- ${head}`,
|
| 1211 |
`-- ${chips.length ? chips.map(([k, v]) => t('sql.chip', { k, v })).join('; ') : t('sql.all')}`,
|
|
|
|
|
|
|
| 1212 |
`SELECT ${ROW_COLUMNS.map((c) => (c === 'sha256' ? 'lower(hex(sha256)) AS sha256' : c)).join(', ')}`,
|
| 1213 |
'FROM read_parquet([',
|
| 1214 |
paths.map((p) => ` ${sqlStr(p)}`).join(',\n'),
|
|
@@ -1274,24 +1292,43 @@ function download(name, text, type) {
|
|
| 1274 |
}
|
| 1275 |
// ---- the selection kept as files: the selection file (the ids, the filters, the release tag of the commit, which the
|
| 1276 |
// command beside it names too; tools/fetch.py --selection reads it, so the same file gives the same files later) and
|
| 1277 |
-
// the attribution file (per source its title, licence
|
|
|
|
| 1278 |
// in English, the language of the dataset)
|
| 1279 |
const selectionName = () => `agentic-env-selection-${IX.sha.slice(0, 7)}.json`;
|
|
|
|
|
|
|
| 1280 |
function selectionFile(sel, picks) {
|
| 1281 |
return `${JSON.stringify({
|
| 1282 |
format: 'agentic-env-selection/1', dataset: REPO, revision: IX.pin,
|
| 1283 |
filters: picks ? {} : Object.fromEntries([...activeFilters(), 'kind'].map((k) => [k, F[k]])), sources: sel.map((s) => s.id).sort(),
|
| 1284 |
}, null, 1)}\n`;
|
| 1285 |
}
|
| 1286 |
-
function attributionFile(sel) {
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1287 |
const lines = [`# ${tEn('attr.title', { count: sel.length, n: fmtInt(sel.length), dataset: REPO })}`, '',
|
| 1288 |
tEn('attr.commit', { sha: IX.sha, url: `${DATASET_URL}/tree/${IX.sha}` }), ''];
|
| 1289 |
for (const s of [...sel].sort((a, b) => a.title.localeCompare(b.title))) {
|
| 1290 |
const lic = s.info.license || {};
|
|
|
|
| 1291 |
lines.push(`## ${s.title}`, '', `- ${tEn('attr.source')}: \`${s.id}\``,
|
| 1292 |
-
`- ${tEn('attr.licence')}: ${lic.name || s.spdx || tEn('attr.notStated')}`
|
| 1293 |
-
|
| 1294 |
-
`- ${tEn('attr.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1295 |
}
|
| 1296 |
return lines.join('\n');
|
| 1297 |
}
|
|
@@ -1307,7 +1344,11 @@ function keepCard(sel, picks = false) {
|
|
| 1307 |
el('div', { class: 'keep-actions' },
|
| 1308 |
el('button', { type: 'button', class: 'btn solid', onclick: () => download(selectionName(), selectionFile(sel, picks), 'application/json') },
|
| 1309 |
icon('get', { size: 16 }), t('keep.selection')),
|
| 1310 |
-
el('button', { type: 'button', class: 'btn', onclick: () =>
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1311 |
icon('doc', { size: 16 }), t('keep.attribution'))),
|
| 1312 |
el('p', { class: 'small' }, el('code', {}, selectionName()), ' ', t('keep.selectionNote', { sources: plural(picks ? 'n.picked' : layerCount(), sel.length), sha: IX.sha.slice(0, 7) }))),
|
| 1313 |
codeCard('run', t('keep.again'), selectionCommand(picks), 3));
|
|
@@ -1581,6 +1622,29 @@ function reveal() {
|
|
| 1581 |
nodes.forEach((n) => io.observe(n));
|
| 1582 |
}
|
| 1583 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1584 |
/** Everything that shows the selection, drawn again when the selection answers with its ids. */
|
| 1585 |
function renderScoped() {
|
| 1586 |
hideTip();
|
|
@@ -1660,6 +1724,8 @@ async function main() {
|
|
| 1660 |
onLanguage(renderLanguage);
|
| 1661 |
decorate();
|
| 1662 |
reveal();
|
|
|
|
|
|
|
| 1663 |
let ticking = false;
|
| 1664 |
// the tooltip is placed for where its mark was; a scroll moves the mark away, so the tooltip goes
|
| 1665 |
const onScroll = () => { hideTip(); if (!ticking) { ticking = true; requestAnimationFrame(() => { ticking = false; progress(); }); } };
|
|
|
|
| 10 |
} from './data.js';
|
| 11 |
import {
|
| 12 |
clear, columns, compactParts, el, figureWithToggle, fmtAnd, fmtBytes, fmtCompact, fmtDate, fmtInt, fmtOr, fmtPct,
|
| 13 |
+
fmtShare, fmtShort, hbars, hideTip, meter, segmented, stack, table, tableShown, vt, withTip,
|
| 14 |
} from './charts.js';
|
| 15 |
import { decorate, drawHero, markFromIndex, setNumber, sizeLine, sizeParts, sizeShort } from './first.js';
|
| 16 |
import { lang, onLanguage, setUpSwitch, t, tEn, translatePage } from './i18n.js';
|
|
|
|
| 34 |
const MAX_LISTED = 30;
|
| 35 |
// the phone layout of style.css
|
| 36 |
const PHONE = '(max-width: 720px)';
|
| 37 |
+
// #1617: the address of the user manual's view (#manual; showView)
|
| 38 |
+
const MANUAL_VIEW = 'manual';
|
| 39 |
const $ = (id) => document.getElementById(id);
|
| 40 |
|
| 41 |
// ---- plain names (codes only inside the SQL and the commands); the words are in the locale files, by code:
|
|
|
|
| 175 |
{ k: 'alone', icon: 'doc' },
|
| 176 |
{ k: 'answers', icon: 'check' },
|
| 177 |
{ k: 'login', icon: 'link' },
|
| 178 |
+
// #1577: whether the upstream still serves the files (catalog column fetch)
|
| 179 |
+
{ k: 'fetch', icon: 'get' },
|
| 180 |
];
|
| 181 |
const fitLabel = (k) => t(`fit.${k}`);
|
| 182 |
const fitWord = (state) => t(`fitWord.${state}`);
|
| 183 |
+
// #1577: the fit of a fetch level (catalog column fetch, index_projection.FETCH_LEVELS)
|
| 184 |
+
const FETCH_FIT = { all: 'yes', some: 'part', none: 'no' };
|
| 185 |
+
/** #1577: a source's fetch check in words: how many of its sampled rows read fetched and matched their record, out of
|
| 186 |
+
* how many, and the day read; or that none is read yet. */
|
| 187 |
+
function fetchText(f) {
|
| 188 |
+
if (!f || !FETCH_FIT[f.level]) return t('fetch.notRead');
|
| 189 |
+
return t(`fetch.${f.level}`, { fetched: fmtInt(f.fetched), rows: fmtInt(f.rows), share: fmtShare(f.fetched, f.rows), date: f.readAt || t('common.unknown') });
|
| 190 |
+
}
|
| 191 |
function fitOf(s) {
|
| 192 |
const q = s.info.size_quantiles_upper_bytes || {};
|
| 193 |
const c = s.contamination?.level;
|
|
|
|
| 202 |
alone: kinds.every((k) => ['unit', 'file', 'API response'].includes(k)) ? 'yes' : 'part',
|
| 203 |
answers: s.isTask ? 'yes' : 'no',
|
| 204 |
login: s.needsAuth === 'no' ? 'yes' : s.needsAuth === 'mixed' ? 'part' : 'no',
|
| 205 |
+
fetch: FETCH_FIT[s.fetch?.level] || 'unknown',
|
| 206 |
};
|
| 207 |
}
|
| 208 |
function fitDot(state, label) {
|
|
|
|
| 1155 |
dl.append(item('size', t('detail.size'), el('span', { class: 'lic' }, el('b', {}, recorded), el('span', { class: 'small' }, amounts),
|
| 1156 |
rest.length ? el('span', { class: 'small size-parts' }, rest.join(' \u00b7 ')) : null),
|
| 1157 |
q.p50 ? sizeStrip(q) : null));
|
| 1158 |
+
// #1577: whether its upstream still serves the files, from the sampled rows read in meta/sizes.parquet (catalog
|
| 1159 |
+
// column fetch)
|
| 1160 |
+
dl.append(item('get', t('detail.fetch'), fetchText(s.fetch)));
|
| 1161 |
// 9 one example file (#1197): the first unit file of its preview rows (meta/preview.parquet, where a material set's
|
| 1162 |
// unit files come first); a source with none there has no example
|
| 1163 |
const pv = previewOf(s);
|
|
|
|
| 1225 |
return [
|
| 1226 |
`-- ${head}`,
|
| 1227 |
`-- ${chips.length ? chips.map(([k, v]) => t('sql.chip', { k, v })).join('; ') : t('sql.all')}`,
|
| 1228 |
+
// #1577: the query reads the rows as published; the default profile of the command is not applied, and it says so
|
| 1229 |
+
`-- ${t('sql.noProfile')}`,
|
| 1230 |
`SELECT ${ROW_COLUMNS.map((c) => (c === 'sha256' ? 'lower(hex(sha256)) AS sha256' : c)).join(', ')}`,
|
| 1231 |
'FROM read_parquet([',
|
| 1232 |
paths.map((p) => ` ${sqlStr(p)}`).join(',\n'),
|
|
|
|
| 1292 |
}
|
| 1293 |
// ---- the selection kept as files: the selection file (the ids, the filters, the release tag of the commit, which the
|
| 1294 |
// command beside it names too; tools/fetch.py --selection reads it, so the same file gives the same files later) and
|
| 1295 |
+
// the attribution file (per source its title, licence and attribution statement from meta/sources.json; #1577: and
|
| 1296 |
+
// its files by publisher, licence and licence URL from meta/attribution.parquet, each file's own, else its source's;
|
| 1297 |
// in English, the language of the dataset)
|
| 1298 |
const selectionName = () => `agentic-env-selection-${IX.sha.slice(0, 7)}.json`;
|
| 1299 |
+
const ATTRIBUTION = 'meta/attribution.parquet';
|
| 1300 |
+
const ATTRIBUTION_COLUMNS = ['source', 'publisher', 'license', 'license_url', 'files'];
|
| 1301 |
function selectionFile(sel, picks) {
|
| 1302 |
return `${JSON.stringify({
|
| 1303 |
format: 'agentic-env-selection/1', dataset: REPO, revision: IX.pin,
|
| 1304 |
filters: picks ? {} : Object.fromEntries([...activeFilters(), 'kind'].map((k) => [k, F[k]])), sources: sel.map((s) => s.id).sort(),
|
| 1305 |
}, null, 1)}\n`;
|
| 1306 |
}
|
| 1307 |
+
async function attributionFile(sel) {
|
| 1308 |
+
const ids = new Set(sel.map((s) => s.id));
|
| 1309 |
+
const files = new Map();
|
| 1310 |
+
if (IX.hasAttribution) {
|
| 1311 |
+
for (const r of (await readColumns(fileUrl(IX.sha, ATTRIBUTION), ATTRIBUTION_COLUMNS)).rows) {
|
| 1312 |
+
if (!ids.has(r.source)) continue;
|
| 1313 |
+
if (!files.has(r.source)) files.set(r.source, []);
|
| 1314 |
+
files.get(r.source).push(r);
|
| 1315 |
+
}
|
| 1316 |
+
}
|
| 1317 |
const lines = [`# ${tEn('attr.title', { count: sel.length, n: fmtInt(sel.length), dataset: REPO })}`, '',
|
| 1318 |
tEn('attr.commit', { sha: IX.sha, url: `${DATASET_URL}/tree/${IX.sha}` }), ''];
|
| 1319 |
for (const s of [...sel].sort((a, b) => a.title.localeCompare(b.title))) {
|
| 1320 |
const lic = s.info.license || {};
|
| 1321 |
+
const own = files.get(s.id) || [];
|
| 1322 |
lines.push(`## ${s.title}`, '', `- ${tEn('attr.source')}: \`${s.id}\``,
|
| 1323 |
+
`- ${tEn('attr.licence')}: ${lic.name || s.spdx || tEn('attr.notStated')}`);
|
| 1324 |
+
if (own.length) {
|
| 1325 |
+
lines.push(`- ${tEn('attr.files')}:`, ...own.map((r) => {
|
| 1326 |
+
const n = num(r.files);
|
| 1327 |
+
return ` - ${tEn('attr.fileLine', { count: n, n: fmtInt(n), publisher: r.publisher || tEn('attr.noPublisher'),
|
| 1328 |
+
licence: r.license || tEn('attr.notStated'), url: r.license_url || tEn('attr.notStated') })}`;
|
| 1329 |
+
}));
|
| 1330 |
+
} else lines.push(`- ${tEn('attr.licenceUrl')}: ${lic.url || tEn('attr.notStated')}`);
|
| 1331 |
+
lines.push(`- ${tEn('attr.attribution')}: ${lic.attribution || lic.statement || tEn('attr.notStated')}`, '');
|
| 1332 |
}
|
| 1333 |
return lines.join('\n');
|
| 1334 |
}
|
|
|
|
| 1344 |
el('div', { class: 'keep-actions' },
|
| 1345 |
el('button', { type: 'button', class: 'btn solid', onclick: () => download(selectionName(), selectionFile(sel, picks), 'application/json') },
|
| 1346 |
icon('get', { size: 16 }), t('keep.selection')),
|
| 1347 |
+
el('button', { type: 'button', class: 'btn', onclick: async (ev) => {
|
| 1348 |
+
const btn = ev.currentTarget;
|
| 1349 |
+
btn.disabled = true;
|
| 1350 |
+
try { download(`agentic-env-attribution-${IX.sha.slice(0, 7)}.md`, await attributionFile(sel), 'text/markdown'); } finally { btn.disabled = false; }
|
| 1351 |
+
} },
|
| 1352 |
icon('doc', { size: 16 }), t('keep.attribution'))),
|
| 1353 |
el('p', { class: 'small' }, el('code', {}, selectionName()), ' ', t('keep.selectionNote', { sources: plural(picks ? 'n.picked' : layerCount(), sel.length), sha: IX.sha.slice(0, 7) }))),
|
| 1354 |
codeCard('run', t('keep.again'), selectionCommand(picks), 3));
|
|
|
|
| 1622 |
nodes.forEach((n) => io.observe(n));
|
| 1623 |
}
|
| 1624 |
|
| 1625 |
+
/** #1617: the user manual is a view of its own: the address #manual shows it in place of the page (manual.js, loaded
|
| 1626 |
+
* the first time it is shown, draws it in the page's language); any other address shows the page, at the block it
|
| 1627 |
+
* names. */
|
| 1628 |
+
function showView() {
|
| 1629 |
+
const on = location.hash === `#${MANUAL_VIEW}`;
|
| 1630 |
+
const back = !on && $('page').hidden;
|
| 1631 |
+
$('manual').hidden = !on;
|
| 1632 |
+
$('page').hidden = on;
|
| 1633 |
+
document.body.classList.toggle('manual-on', on);
|
| 1634 |
+
if (on) {
|
| 1635 |
+
scrollTo(0, 0);
|
| 1636 |
+
import('./manual.js').then((m) => m.drawManual($('manual-body')), (e) => {
|
| 1637 |
+
clear($('manual-body')).append(el('p', { class: 'small problem-text' }, t('manual.missing', { error: e.message })));
|
| 1638 |
+
console.error(e);
|
| 1639 |
+
});
|
| 1640 |
+
} else if (back) {
|
| 1641 |
+
// the charts were drawn, if at all, while the page was hidden
|
| 1642 |
+
redrawOnResize();
|
| 1643 |
+
progress();
|
| 1644 |
+
if (location.hash) document.getElementById(decodeURIComponent(location.hash.slice(1)))?.scrollIntoView();
|
| 1645 |
+
}
|
| 1646 |
+
}
|
| 1647 |
+
|
| 1648 |
/** Everything that shows the selection, drawn again when the selection answers with its ids. */
|
| 1649 |
function renderScoped() {
|
| 1650 |
hideTip();
|
|
|
|
| 1724 |
onLanguage(renderLanguage);
|
| 1725 |
decorate();
|
| 1726 |
reveal();
|
| 1727 |
+
showView();
|
| 1728 |
+
addEventListener('hashchange', showView);
|
| 1729 |
let ticking = false;
|
| 1730 |
// the tooltip is placed for where its mark was; a scroll moves the mark away, so the tooltip goes
|
| 1731 |
const onScroll = () => { hideTip(); if (!ticking) { ticking = true; requestAnimationFrame(() => { ticking = false; progress(); }); } };
|
data.js
CHANGED
|
@@ -394,6 +394,9 @@ export async function loadIndex(params, onNote) {
|
|
| 394 |
distinct: has('distinct_bytes') ? num(r.distinct_bytes) : null,
|
| 395 |
// #1158: the source's size figures from meta/size_figures.json (recorded, deduplicated, check, estimate)
|
| 396 |
size: figures?.sources?.[r.source] || null,
|
|
|
|
|
|
|
|
|
|
| 397 |
topTypes: Array.isArray(r.top_types) ? r.top_types : r.top_types?.toArray?.() || [],
|
| 398 |
needsAuth: r.needs_auth || null,
|
| 399 |
train: permission(info.license_rows?.training_ok),
|
|
@@ -414,6 +417,9 @@ export async function loadIndex(params, onNote) {
|
|
| 414 |
date: rev.date,
|
| 415 |
overrides: { catalog: params.get('catalog'), vocabulary: params.get('vocabulary') },
|
| 416 |
hasVocabulary: Boolean(vocabJson),
|
|
|
|
|
|
|
|
|
|
| 417 |
hasKind,
|
| 418 |
vocab,
|
| 419 |
sources,
|
|
|
|
| 394 |
distinct: has('distinct_bytes') ? num(r.distinct_bytes) : null,
|
| 395 |
// #1158: the source's size figures from meta/size_figures.json (recorded, deduplicated, check, estimate)
|
| 396 |
size: figures?.sources?.[r.source] || null,
|
| 397 |
+
// #1577: whether its upstream still serves its files (catalog column fetch: level, rows, fetched, read_at); null
|
| 398 |
+
// for a catalog without the column
|
| 399 |
+
fetch: r.fetch && r.fetch.level ? { level: String(r.fetch.level), rows: num(r.fetch.rows) || 0, fetched: num(r.fetch.fetched) || 0, readAt: r.fetch.read_at || null } : null,
|
| 400 |
topTypes: Array.isArray(r.top_types) ? r.top_types : r.top_types?.toArray?.() || [],
|
| 401 |
needsAuth: r.needs_auth || null,
|
| 402 |
train: permission(info.license_rows?.training_ok),
|
|
|
|
| 417 |
date: rev.date,
|
| 418 |
overrides: { catalog: params.get('catalog'), vocabulary: params.get('vocabulary') },
|
| 419 |
hasVocabulary: Boolean(vocabJson),
|
| 420 |
+
// #1577: who published each file of a material set and under which license, read when the attribution list is
|
| 421 |
+
// made; a commit before it has none
|
| 422 |
+
hasAttribution: has('meta/attribution.parquet'),
|
| 423 |
hasKind,
|
| 424 |
vocab,
|
| 425 |
sources,
|
first.js
CHANGED
|
@@ -37,9 +37,12 @@ export function setNumber(node, key, value, fmt) {
|
|
| 37 |
}
|
| 38 |
|
| 39 |
/** #1225: the size check of a level (check of meta/size_figures.json): the share of the rows compared that match,
|
| 40 |
-
* with their number
|
|
|
|
|
|
|
|
|
|
| 41 |
const checkText = (c) => (c.compared ? t('size.check', { share: fmtShare(c.match, c.compared), n: fmtInt(c.compared) })
|
| 42 |
-
: t('size.checkNone'));
|
| 43 |
|
| 44 |
/** #1158: one level's size figures (a level of meta/size_figures.json, read as they are and never summed here), each
|
| 45 |
* saying what it is: the recorded size, the deduplicated size, the check (checkText) and, when the level
|
|
@@ -74,7 +77,7 @@ export function sizeShort(g, summed) {
|
|
| 74 |
const recorded = t('size.recordedShort', { size: fmtBytes(g.bytes) });
|
| 75 |
const rest = [t('size.distinctShort', { size: fmtBytes(g.distinct_bytes) })];
|
| 76 |
const tip = [c.compared ? t('size.checkSentence', { share: fmtShare(c.match, c.compared), n: fmtInt(c.compared) })
|
| 77 |
-
: t('size.checkNoneSentence')];
|
| 78 |
if (g.unsized_rows) {
|
| 79 |
rest.push(e ? t('size.unsizedEst', { size: fmtBytes(e.bytes) }) : t('size.unsizedNone', { rows: fmtCompact(g.unsized_rows) }));
|
| 80 |
tip.push(sizeParts(g, summed).at(-1));
|
|
|
|
| 37 |
}
|
| 38 |
|
| 39 |
/** #1225: the size check of a level (check of meta/size_figures.json): the share of the rows compared that match,
|
| 40 |
+
* with their number; #1577: a level with rows read but none compared says so with its own reading, the rows read and
|
| 41 |
+
* what kept each from its file (the same counts as the md card's size line), never that nothing was compared yet. */
|
| 42 |
+
const NOT_COMPARED = ['unreachable', 'gone', 'needs_auth', 'no_length', 'error'];
|
| 43 |
+
const whyNot = (c) => NOT_COMPARED.filter((k) => c[k]).map((k) => t(`size.status.${k}`, { n: fmtInt(c[k]) })).join(t('size.listSep'));
|
| 44 |
const checkText = (c) => (c.compared ? t('size.check', { share: fmtShare(c.match, c.compared), n: fmtInt(c.compared) })
|
| 45 |
+
: c.rows ? t('size.checkNotCompared', { read: fmtInt(c.rows), why: whyNot(c) }) : t('size.checkNone'));
|
| 46 |
|
| 47 |
/** #1158: one level's size figures (a level of meta/size_figures.json, read as they are and never summed here), each
|
| 48 |
* saying what it is: the recorded size, the deduplicated size, the check (checkText) and, when the level
|
|
|
|
| 77 |
const recorded = t('size.recordedShort', { size: fmtBytes(g.bytes) });
|
| 78 |
const rest = [t('size.distinctShort', { size: fmtBytes(g.distinct_bytes) })];
|
| 79 |
const tip = [c.compared ? t('size.checkSentence', { share: fmtShare(c.match, c.compared), n: fmtInt(c.compared) })
|
| 80 |
+
: c.rows ? t('size.checkNotComparedSentence', { read: fmtInt(c.rows), why: whyNot(c) }) : t('size.checkNoneSentence')];
|
| 81 |
if (g.unsized_rows) {
|
| 82 |
rest.push(e ? t('size.unsizedEst', { size: fmtBytes(e.bytes) }) : t('size.unsizedNone', { rows: fmtCompact(g.unsized_rows) }));
|
| 83 |
tip.push(sizeParts(g, summed).at(-1));
|
index.html
CHANGED
|
@@ -6,7 +6,7 @@
|
|
| 6 |
<title data-i18n="page.title">Agentic Env Index</title>
|
| 7 |
<link rel="icon" href="data:,">
|
| 8 |
<link rel="alternate" type="text/plain" href="llms.txt" title="For agents" data-i18n-attr="title:nav.agents">
|
| 9 |
-
<meta name="description" content="Real material for terminal-agent tasks from the public pointer index Battam/agentic-env-index: by
|
| 10 |
<meta name="theme-color" content="#112532">
|
| 11 |
<link rel="preconnect" href="https://fonts.googleapis.com">
|
| 12 |
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
|
@@ -31,6 +31,7 @@
|
|
| 31 |
<li><a href="#releases" data-part="releases" data-icon="releases"><span class="pl pl-long" data-i18n="nav.releases">Releases</span><span class="pl pl-short" data-i18n="nav.releasesShort">Releases</span><span class="track"><span class="fill"></span></span></a></li>
|
| 32 |
</ol>
|
| 33 |
<div class="nav-links">
|
|
|
|
| 34 |
<a href="llms.txt" data-icon="robot" data-size="16"><span data-i18n="nav.agents">For agents</span></a>
|
| 35 |
<a class="btn solid" href="https://huggingface.co/datasets/Battam/agentic-env-index/discussions/new" data-i18n="nav.request">Request data</a>
|
| 36 |
<!-- the language switch: the other language's name, in that language (i18n.js) -->
|
|
@@ -39,11 +40,11 @@
|
|
| 39 |
</div>
|
| 40 |
</nav>
|
| 41 |
|
| 42 |
-
<main>
|
| 43 |
<header class="hero" id="top" data-part="overview">
|
| 44 |
<div class="wrap hero-grid">
|
| 45 |
<div class="hero-main">
|
| 46 |
-
<p class="eyebrow" data-icon="sources" data-size="18"><span data-i18n="hero.eyebrow">The Agentic Env Index, by
|
| 47 |
<h1 data-i18n="hero.title">Real material for agent tasks</h1>
|
| 48 |
<!-- #1256: under the title, the sentence on how the two layers relate (meta/vocabulary.json, app.js renderLayers)
|
| 49 |
and how the sizes are made, in the index's docs/sources.md (the size note under the list links there too);
|
|
@@ -51,7 +52,7 @@
|
|
| 51 |
<p class="hero-relation" id="hero-relation"><span class="sk sk-line"></span></p>
|
| 52 |
<p class="hero-how"><a href="https://huggingface.co/datasets/Battam/agentic-env-index/blob/main/docs/sources.md" data-i18n-attr="href:url.docsSources"><span data-i18n="size.how">How sizes are measured</span></a></p>
|
| 53 |
<div class="hero-cta">
|
| 54 |
-
<a class="btn hero-btn" href="#material" data-icon="browse" data-size="18"><span data-i18n="hero.browse">Browse
|
| 55 |
<a class="btn hero-btn ghost" href="#get" data-icon="get" data-size="18"><span data-i18n="nav.get">Get the data</span></a>
|
| 56 |
</div>
|
| 57 |
</div>
|
|
@@ -86,6 +87,7 @@
|
|
| 86 |
<span class="chip chip-dark" id="preview-chip" hidden></span>
|
| 87 |
<a href="https://huggingface.co/datasets/Battam/agentic-env-index" data-i18n="link.dataset">Dataset</a>
|
| 88 |
<a href="https://huggingface.co/datasets/Battam/agentic-env-index/blob/main/docs/fields.md" data-i18n="link.fields" data-i18n-attr="href:url.docsFields">Catalog fields</a>
|
|
|
|
| 89 |
<span class="notice" id="notice" hidden></span>
|
| 90 |
</div>
|
| 91 |
</header>
|
|
@@ -103,8 +105,8 @@
|
|
| 103 |
<div class="where-grid one">
|
| 104 |
<figure class="card chart-card where-card" id="where-src"><div class="sk sk-block"></div></figure>
|
| 105 |
</div>
|
| 106 |
-
<div class="band-head sub-head reveal"><h3 id="cats-h" data-i18n="material.cats">Most requested
|
| 107 |
-
<!-- one card per most requested
|
| 108 |
<ul class="cat-grid" id="req-cards">
|
| 109 |
<li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li>
|
| 110 |
</ul>
|
|
@@ -139,7 +141,7 @@
|
|
| 139 |
<option value="yes" data-i18n="filter.permit.yes">Allowed, all rows</option>
|
| 140 |
<option value="cond" data-i18n="filter.permit.cond">Allowed or conditional</option>
|
| 141 |
</select></label>
|
| 142 |
-
<label><span data-i18n="filter.shipping">
|
| 143 |
<option value="" data-i18n="filter.any">Any</option>
|
| 144 |
<option value="yes" data-i18n="filter.permit.yes">Allowed, all rows</option>
|
| 145 |
<option value="cond" data-i18n="filter.permit.cond">Allowed or conditional</option>
|
|
@@ -238,13 +240,24 @@
|
|
| 238 |
</section>
|
| 239 |
</main>
|
| 240 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 241 |
<footer class="site-foot" id="request" data-part="releases">
|
| 242 |
<div class="wrap foot-grid">
|
| 243 |
<div class="foot-cta">
|
| 244 |
<span class="foot-ico" data-icon="request" data-size="32"></span>
|
| 245 |
<div>
|
| 246 |
<h2 data-i18n="foot.missing">Missing material?</h2>
|
| 247 |
-
<p data-i18n="foot.ask">Ask for a
|
| 248 |
</div>
|
| 249 |
<a class="btn on-dark-solid" href="https://huggingface.co/datasets/Battam/agentic-env-index/discussions/new" data-i18n="nav.request">Request data</a>
|
| 250 |
</div>
|
|
@@ -252,6 +265,7 @@
|
|
| 252 |
<li><a href="https://huggingface.co/datasets/Battam/agentic-env-index" data-i18n="link.dataset">Dataset</a></li>
|
| 253 |
<li><a href="https://huggingface.co/datasets/Battam/agentic-env-index/blob/main/docs/fields.md" data-i18n="link.fields" data-i18n-attr="href:url.docsFields">Catalog fields</a></li>
|
| 254 |
<li><a href="llms.txt" data-i18n="nav.agents">For agents</a></li>
|
|
|
|
| 255 |
<li><a href="#top" data-i18n="foot.top">Back to top</a></li>
|
| 256 |
</ul>
|
| 257 |
<p class="foot-pin mono" id="foot-pin"></p>
|
|
|
|
| 6 |
<title data-i18n="page.title">Agentic Env Index</title>
|
| 7 |
<link rel="icon" href="data:,">
|
| 8 |
<link rel="alternate" type="text/plain" href="llms.txt" title="For agents" data-i18n-attr="title:nav.agents">
|
| 9 |
+
<meta name="description" content="Real material for terminal-agent tasks from the public pointer index Battam/agentic-env-index: by domain, fit at a glance, one command to take it." data-i18n-attr="content:page.description">
|
| 10 |
<meta name="theme-color" content="#112532">
|
| 11 |
<link rel="preconnect" href="https://fonts.googleapis.com">
|
| 12 |
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
|
|
|
| 31 |
<li><a href="#releases" data-part="releases" data-icon="releases"><span class="pl pl-long" data-i18n="nav.releases">Releases</span><span class="pl pl-short" data-i18n="nav.releasesShort">Releases</span><span class="track"><span class="fill"></span></span></a></li>
|
| 32 |
</ol>
|
| 33 |
<div class="nav-links">
|
| 34 |
+
<a href="#manual" data-icon="doc" data-size="16"><span data-i18n="nav.manual">Manual</span></a>
|
| 35 |
<a href="llms.txt" data-icon="robot" data-size="16"><span data-i18n="nav.agents">For agents</span></a>
|
| 36 |
<a class="btn solid" href="https://huggingface.co/datasets/Battam/agentic-env-index/discussions/new" data-i18n="nav.request">Request data</a>
|
| 37 |
<!-- the language switch: the other language's name, in that language (i18n.js) -->
|
|
|
|
| 40 |
</div>
|
| 41 |
</nav>
|
| 42 |
|
| 43 |
+
<main id="page">
|
| 44 |
<header class="hero" id="top" data-part="overview">
|
| 45 |
<div class="wrap hero-grid">
|
| 46 |
<div class="hero-main">
|
| 47 |
+
<p class="eyebrow" data-icon="sources" data-size="18"><span data-i18n="hero.eyebrow">The Agentic Env Index, by domain</span></p>
|
| 48 |
<h1 data-i18n="hero.title">Real material for agent tasks</h1>
|
| 49 |
<!-- #1256: under the title, the sentence on how the two layers relate (meta/vocabulary.json, app.js renderLayers)
|
| 50 |
and how the sizes are made, in the index's docs/sources.md (the size note under the list links there too);
|
|
|
|
| 52 |
<p class="hero-relation" id="hero-relation"><span class="sk sk-line"></span></p>
|
| 53 |
<p class="hero-how"><a href="https://huggingface.co/datasets/Battam/agentic-env-index/blob/main/docs/sources.md" data-i18n-attr="href:url.docsSources"><span data-i18n="size.how">How sizes are measured</span></a></p>
|
| 54 |
<div class="hero-cta">
|
| 55 |
+
<a class="btn hero-btn" href="#material" data-icon="browse" data-size="18"><span data-i18n="hero.browse">Browse domains</span></a>
|
| 56 |
<a class="btn hero-btn ghost" href="#get" data-icon="get" data-size="18"><span data-i18n="nav.get">Get the data</span></a>
|
| 57 |
</div>
|
| 58 |
</div>
|
|
|
|
| 87 |
<span class="chip chip-dark" id="preview-chip" hidden></span>
|
| 88 |
<a href="https://huggingface.co/datasets/Battam/agentic-env-index" data-i18n="link.dataset">Dataset</a>
|
| 89 |
<a href="https://huggingface.co/datasets/Battam/agentic-env-index/blob/main/docs/fields.md" data-i18n="link.fields" data-i18n-attr="href:url.docsFields">Catalog fields</a>
|
| 90 |
+
<a href="#manual" data-i18n="nav.manual">Manual</a>
|
| 91 |
<span class="notice" id="notice" hidden></span>
|
| 92 |
</div>
|
| 93 |
</header>
|
|
|
|
| 105 |
<div class="where-grid one">
|
| 106 |
<figure class="card chart-card where-card" id="where-src"><div class="sk sk-block"></div></figure>
|
| 107 |
</div>
|
| 108 |
+
<div class="band-head sub-head reveal"><h3 id="cats-h" data-i18n="material.cats">Most requested domains</h3></div>
|
| 109 |
+
<!-- one card per most requested domain (MOST_REQUESTED in app.js, ten): two rows of five from 1,100 px -->
|
| 110 |
<ul class="cat-grid" id="req-cards">
|
| 111 |
<li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li>
|
| 112 |
</ul>
|
|
|
|
| 141 |
<option value="yes" data-i18n="filter.permit.yes">Allowed, all rows</option>
|
| 142 |
<option value="cond" data-i18n="filter.permit.cond">Allowed or conditional</option>
|
| 143 |
</select></label>
|
| 144 |
+
<label><span data-i18n="filter.shipping">Redistribution</span><select id="f-redist">
|
| 145 |
<option value="" data-i18n="filter.any">Any</option>
|
| 146 |
<option value="yes" data-i18n="filter.permit.yes">Allowed, all rows</option>
|
| 147 |
<option value="cond" data-i18n="filter.permit.cond">Allowed or conditional</option>
|
|
|
|
| 240 |
</section>
|
| 241 |
</main>
|
| 242 |
|
| 243 |
+
<!-- #1617: the user manual, a view of its own at the address #manual (app.js showView, manual.js), in place of the page
|
| 244 |
+
above: manual/manual.<language>.md of this folder, its numbers and command outputs filled at deploy
|
| 245 |
+
(03_data_line/steps/portal.py), its images manual/shots/<output>, taken on this page after each deploy
|
| 246 |
+
(03_data_line/steps/shots.py) -->
|
| 247 |
+
<main class="manual-view" id="manual" hidden aria-label="User manual" data-i18n-attr="aria-label:manual.title">
|
| 248 |
+
<div class="wrap manual-wrap">
|
| 249 |
+
<p class="manual-back"><a href="#top" data-i18n="manual.back">Back to the page</a></p>
|
| 250 |
+
<article class="card manual-body" id="manual-body"><div class="sk sk-block tall"></div></article>
|
| 251 |
+
</div>
|
| 252 |
+
</main>
|
| 253 |
+
|
| 254 |
<footer class="site-foot" id="request" data-part="releases">
|
| 255 |
<div class="wrap foot-grid">
|
| 256 |
<div class="foot-cta">
|
| 257 |
<span class="foot-ico" data-icon="request" data-size="32"></span>
|
| 258 |
<div>
|
| 259 |
<h2 data-i18n="foot.missing">Missing material?</h2>
|
| 260 |
+
<p data-i18n="foot.ask">Ask for a domain, a form or a source.</p>
|
| 261 |
</div>
|
| 262 |
<a class="btn on-dark-solid" href="https://huggingface.co/datasets/Battam/agentic-env-index/discussions/new" data-i18n="nav.request">Request data</a>
|
| 263 |
</div>
|
|
|
|
| 265 |
<li><a href="https://huggingface.co/datasets/Battam/agentic-env-index" data-i18n="link.dataset">Dataset</a></li>
|
| 266 |
<li><a href="https://huggingface.co/datasets/Battam/agentic-env-index/blob/main/docs/fields.md" data-i18n="link.fields" data-i18n-attr="href:url.docsFields">Catalog fields</a></li>
|
| 267 |
<li><a href="llms.txt" data-i18n="nav.agents">For agents</a></li>
|
| 268 |
+
<li><a href="#manual" data-i18n="nav.manual">Manual</a></li>
|
| 269 |
<li><a href="#top" data-i18n="foot.top">Back to top</a></li>
|
| 270 |
</ul>
|
| 271 |
<p class="foot-pin mono" id="foot-pin"></p>
|
llms.txt
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
# Agentic Env Index: for agents
|
| 2 |
|
| 3 |
- The index holds two layers over the same public web, kept apart and never added together. A material set is a batch of files picked from one source (a site, a repository or a dataset), for building tasks; material sets are grouped into collections by task family; corpora are whole upstreams read in bulk; material sets are not taken from the corpora. Task material comes first: material sets, grouped into collections, one per task family. A material set (task-ready) is a batch of files picked from one source (a site, a repository or a dataset), for building tasks: its files are grouped into units, its rows carry the roles unit, companion and answer (the catalog column `kind` is `material_set`, and `collection` names its collection), and each unit comes with its expected results. Corpora come second, to search when no material set fits: a corpus (raw; plural corpora) is one whole upstream that a collector keeps reading, such as Common Crawl's office documents, PyPI's release files, Mendeley Data or NVD: its rows are pointers to files or API answers, with no units and no answers, and it is mined one unit per row (`kind` is `corpus`).
|
| 4 |
-
- Pick material sets or corpora from `meta/catalog.parquet` (one row per material set or corpus, about 50 KB; do not download `meta/` whole). Columns: `source`; `kind`, `material_set` or `corpus` (select one layer: units and rows are never added together); `collection`, the code of a material set's collection (A, B, C, G, GT, HW, M, MD, ML, OP, S, SC, SW; empty for a corpus); `domain`, a list of codes D1 to D10 or leaves such as D3.5, or `multi`, `unclassified`; `form`, such as `test-suites` or `spreadsheets-with-answers`; `units` (a unit is one test case or one workbook; a corpus has one unit per row), `answers` (units with an answer file, the expected results kept apart) and `files` (files of every role); `publishers` (a material set's distinct publisher names over all its rows; empty when its rows name none); `language` (the language codes of the rows' labels, what the default profile filters on; `not recorded` when a row has none); `contamination.level` (by the rule of the default profile of `tools/fetch.py` over `meta/bench_overlap.parquet` and `meta/bench_checked.parquet`: `none` when every data file is compared and kept and no row is removed, the hit class with the most removed rows when some are removed, `not scanned` otherwise); `license.ship`; `pii.level`; `distinct_bytes` (deduplicated size) and `bytes` (recorded size, the sum over rows); `size_check` (the rows read in `meta/sizes.parquet`, `compared` those that compared a size or hash, and how many have each status, `match` when the measured size agrees; the check's share is `match` over `compared`), `unsized_rows` (rows without a recorded size) and `unsized_estimate` (their estimated size, with its 95% range); the same figures for the whole index, each layer and each collection are in `meta/size_figures.json`. The vocabulary is `meta/vocabulary.json`: every domain code with its name (and `zh`, its Chinese name), what it covers and its parent, every form with its Chinese name and what it means, every category of `meta/not_included.parquet` with its requirement in English and in Chinese, the two layers with their relation, and every collection with its name, what it holds and its campaigns (take a collection with `--tag campaign:<name>` for each of its campaigns).
|
| 5 |
- Size of this release: Whole index: Recorded size 4.50 PB; deduplicated size 4.09 PB; size check: 99.6% match (48,396 of the 49,407 rows read compared); not compared: address unreachable 477, file gone 131, login needed 331, no size from the server 68, other error 4; estimated size of 309,522,457 of the 309,787,645 rows without a recorded size: 657 TB (an estimate; 95% range 512 TB to 1.05 PB, the sum of its sources' ranges, wider than a joint interval; from 17,054 rows read), the other 265,188 with too few rows read for an estimate. Task material: Recorded size 2.90 GB; deduplicated size 2.82 GB; size check: 99.7% match (9,111 of the 9,135 rows read compared); not compared: address unreachable 11, file gone 3, login needed 8, other error 2. Corpora: Recorded size 4.50 PB; deduplicated size 4.09 PB; size check: 99.6% match (39,285 of the 40,272 rows read compared); not compared: address unreachable 466, file gone 128, login needed 323, no size from the server 68, other error 2; estimated size of 309,522,457 of the 309,787,645 rows without a recorded size: 657 TB (an estimate; 95% range 512 TB to 1.05 PB, the sum of its sources' ranges, wider than a joint interval; from 17,054 rows read), the other 265,188 with too few rows read for an estimate. Distinct content is the sum of `size_bytes` over the rows minus the sizes of the rows that repeat another row's content (the same sha256, else sha1, else md5 and size, within the same hash scope; one copy of each content is kept, in the first source by registry order), counted at every release, per source in the catalog column `distinct_bytes`; archive members whose archive the same source also lists whole have other hashes, so they are not subtracted. `tools/fetch.py` removes duplicates over your own selection when it reads it. The recorded size is the sum of `size_bytes` over the rows that have one (catalog column `bytes`). The check reads a sample of rows of every data file (`meta/sizes.parquet`): 20 of its rows with a recorded size, and of its rows without one its share of its source's sample, sized for their mean size within 25% at 95% confidence by the spread of the sizes read so far (at most 400 rows per source per round, 20 for a source with none read yet); its share is the rows that match their recorded size out of the rows it compared (status `match`, `size_mismatch` or `hash_mismatch`), and the rows it could not compare, which never reached their file, are counted beside it by status (`unreachable`, `gone`, `needs_auth`, `no_length`, `error`; catalog column `size_check`). The estimate for a source's rows without a recorded size (catalog columns `unsized_rows` and `unsized_estimate`) is their number times the mean measured size of such rows read, with a 95% BCa bootstrap range (`scipy.stats.bootstrap`, its default resample count); a source with fewer than 10 such rows read has none. Above one source, the estimate is the sum of its sources' estimates and its range the sum of their ranges, which is wider than a joint interval. The figures of every level are in `meta/size_figures.json`.
|
| 6 |
- Fetch with one command: `uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/main/tools/fetch.py --domain D3 --form spreadsheets-with-answers --sample 3 --out ./data`. Without uv: `pip install duckdb`, download `tools/fetch.py` and run it with `python`. Select with `--source`, `--kind` (`material_set` or `corpus`: one layer), `--domain`, `--form`, `--tag`, `--ext`, or `--selection <file>` (a selection file the data page saves: the ids of its sources and its commit); `--sample N` takes N units per source, `--dry-run` only counts and writes `_meta/fetch_manifest.jsonl` with one line per file of the selection (status selected, removed with the criterion in removed_by, or not_taken), as many lines as the files the summary counts.
|
| 7 |
- It pins the commit it reads and prints it; pass `--revision <commit>` to get the same files later. A name that does not exist stops with exit code 2 and the closest names.
|
|
|
|
| 1 |
# Agentic Env Index: for agents
|
| 2 |
|
| 3 |
- The index holds two layers over the same public web, kept apart and never added together. A material set is a batch of files picked from one source (a site, a repository or a dataset), for building tasks; material sets are grouped into collections by task family; corpora are whole upstreams read in bulk; material sets are not taken from the corpora. Task material comes first: material sets, grouped into collections, one per task family. A material set (task-ready) is a batch of files picked from one source (a site, a repository or a dataset), for building tasks: its files are grouped into units, its rows carry the roles unit, companion and answer (the catalog column `kind` is `material_set`, and `collection` names its collection), and each unit comes with its expected results. Corpora come second, to search when no material set fits: a corpus (raw; plural corpora) is one whole upstream that a collector keeps reading, such as Common Crawl's office documents, PyPI's release files, Mendeley Data or NVD: its rows are pointers to files or API answers, with no units and no answers, and it is mined one unit per row (`kind` is `corpus`).
|
| 4 |
+
- Pick material sets or corpora from `meta/catalog.parquet` (one row per material set or corpus, about 50 KB; do not download `meta/` whole). Columns: `source`; `kind`, `material_set` or `corpus` (select one layer: units and rows are never added together); `collection`, the code of a material set's collection (A, B, C, G, GT, HW, M, MD, ML, OP, S, SC, SW; empty for a corpus); `domain`, a list of codes D1 to D10 or leaves such as D3.5, or `multi`, `unclassified`; `form`, such as `test-suites` or `spreadsheets-with-answers`; `units` (a unit is one test case or one workbook; a corpus has one unit per row), `answers` (units with an answer file, the expected results kept apart) and `files` (files of every role); `publishers` (a material set's distinct publisher names over all its rows; empty when its rows name none); `language` (the language codes of the rows' labels, what the default profile filters on; `not recorded` when a row has none); `contamination.level` (by the rule of the default profile of `tools/fetch.py` over `meta/bench_overlap.parquet` and `meta/bench_checked.parquet`: `none` when every data file is compared and kept and no row is removed, the hit class with the most removed rows when some are removed, `not scanned` otherwise); `license.ship`; `pii.level`; `distinct_bytes` (deduplicated size) and `bytes` (recorded size, the sum over rows); `size_check` (the rows read in `meta/sizes.parquet`, `compared` those that compared a size or hash, and how many have each status, `match` when the measured size agrees; the check's share is `match` over `compared`), `unsized_rows` (rows without a recorded size) and `unsized_estimate` (their estimated size, with its 95% range); `fetch` (whether the upstream still serves the files: `rows`, the sampled rows read in `meta/sizes.parquet`; `fetched`, those that fetched and matched their recorded size and hash; `read_at`, the day read; `level`, `all`, `some`, `none` or `not read`); the same figures for the whole index, each layer and each collection are in `meta/size_figures.json`. Who published each file of a material set, and under which license and license URL, is in `meta/attribution.parquet` (one row per source, publisher, license and license URL, with its number of files). The vocabulary is `meta/vocabulary.json`: every domain code with its name (and `zh`, its Chinese name), what it covers and its parent, every form with its Chinese name and what it means, every category of `meta/not_included.parquet` with its requirement in English and in Chinese, the two layers with their relation, and every collection with its name, what it holds and its campaigns (take a collection with `--tag campaign:<name>` for each of its campaigns).
|
| 5 |
- Size of this release: Whole index: Recorded size 4.50 PB; deduplicated size 4.09 PB; size check: 99.6% match (48,396 of the 49,407 rows read compared); not compared: address unreachable 477, file gone 131, login needed 331, no size from the server 68, other error 4; estimated size of 309,522,457 of the 309,787,645 rows without a recorded size: 657 TB (an estimate; 95% range 512 TB to 1.05 PB, the sum of its sources' ranges, wider than a joint interval; from 17,054 rows read), the other 265,188 with too few rows read for an estimate. Task material: Recorded size 2.90 GB; deduplicated size 2.82 GB; size check: 99.7% match (9,111 of the 9,135 rows read compared); not compared: address unreachable 11, file gone 3, login needed 8, other error 2. Corpora: Recorded size 4.50 PB; deduplicated size 4.09 PB; size check: 99.6% match (39,285 of the 40,272 rows read compared); not compared: address unreachable 466, file gone 128, login needed 323, no size from the server 68, other error 2; estimated size of 309,522,457 of the 309,787,645 rows without a recorded size: 657 TB (an estimate; 95% range 512 TB to 1.05 PB, the sum of its sources' ranges, wider than a joint interval; from 17,054 rows read), the other 265,188 with too few rows read for an estimate. Distinct content is the sum of `size_bytes` over the rows minus the sizes of the rows that repeat another row's content (the same sha256, else sha1, else md5 and size, within the same hash scope; one copy of each content is kept, in the first source by registry order), counted at every release, per source in the catalog column `distinct_bytes`; archive members whose archive the same source also lists whole have other hashes, so they are not subtracted. `tools/fetch.py` removes duplicates over your own selection when it reads it. The recorded size is the sum of `size_bytes` over the rows that have one (catalog column `bytes`). The check reads a sample of rows of every data file (`meta/sizes.parquet`): 20 of its rows with a recorded size, and of its rows without one its share of its source's sample, sized for their mean size within 25% at 95% confidence by the spread of the sizes read so far (at most 400 rows per source per round, 20 for a source with none read yet); its share is the rows that match their recorded size out of the rows it compared (status `match`, `size_mismatch` or `hash_mismatch`), and the rows it could not compare, which never reached their file, are counted beside it by status (`unreachable`, `gone`, `needs_auth`, `no_length`, `error`; catalog column `size_check`). The estimate for a source's rows without a recorded size (catalog columns `unsized_rows` and `unsized_estimate`) is their number times the mean measured size of such rows read, with a 95% BCa bootstrap range (`scipy.stats.bootstrap`, its default resample count); a source with fewer than 10 such rows read has none. Above one source, the estimate is the sum of its sources' estimates and its range the sum of their ranges, which is wider than a joint interval. The figures of every level are in `meta/size_figures.json`.
|
| 6 |
- Fetch with one command: `uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/main/tools/fetch.py --domain D3 --form spreadsheets-with-answers --sample 3 --out ./data`. Without uv: `pip install duckdb`, download `tools/fetch.py` and run it with `python`. Select with `--source`, `--kind` (`material_set` or `corpus`: one layer), `--domain`, `--form`, `--tag`, `--ext`, or `--selection <file>` (a selection file the data page saves: the ids of its sources and its commit); `--sample N` takes N units per source, `--dry-run` only counts and writes `_meta/fetch_manifest.jsonl` with one line per file of the selection (status selected, removed with the criterion in removed_by, or not_taken), as many lines as the files the summary counts.
|
| 7 |
- It pins the commit it reads and prints it; pass `--revision <commit>` to get the same files later. A name that does not exist stops with exit code 2 and the closest names.
|
locales/en.json
CHANGED
|
@@ -3,7 +3,7 @@
|
|
| 3 |
"lang.switch": "Show this page in Chinese",
|
| 4 |
|
| 5 |
"page.title": "Agentic Env Index",
|
| 6 |
-
"page.description": "Real material for terminal-agent tasks from the public pointer index Battam/agentic-env-index: by
|
| 7 |
|
| 8 |
"nav.sections": "Page sections",
|
| 9 |
"nav.overview": "Overview",
|
|
@@ -15,11 +15,12 @@
|
|
| 15 |
"nav.releases": "Releases",
|
| 16 |
"nav.releasesShort": "Releases",
|
| 17 |
"nav.agents": "For agents",
|
|
|
|
| 18 |
"nav.request": "Request data",
|
| 19 |
|
| 20 |
-
"hero.eyebrow": "The Agentic Env Index, by
|
| 21 |
"hero.title": "Real material for agent tasks",
|
| 22 |
-
"hero.browse": "Browse
|
| 23 |
"hero.facts": "Facts",
|
| 24 |
"hero.indexOf": "Index of",
|
| 25 |
"hero.commit": "Commit {{sha}}",
|
|
@@ -38,7 +39,7 @@
|
|
| 38 |
"url.docsSources": "https://huggingface.co/datasets/Battam/agentic-env-index/blob/main/docs/sources.md",
|
| 39 |
|
| 40 |
"foot.missing": "Missing material?",
|
| 41 |
-
"foot.ask": "Ask for a
|
| 42 |
"foot.top": "Back to top",
|
| 43 |
"foot.index": "Index {{sha}} · {{date}}",
|
| 44 |
|
|
@@ -126,7 +127,7 @@
|
|
| 126 |
"permit.yes": "Allowed",
|
| 127 |
"permit.no": "Not allowed",
|
| 128 |
"permit.conditional": "Conditional",
|
| 129 |
-
"permit.conditional.sameLicence": "Conditional (
|
| 130 |
"permit.conditional.nonCommercial": "Conditional (non-commercial or research use only)",
|
| 131 |
"permit.unknown": "Not stated",
|
| 132 |
"unitKind.file": "a file",
|
|
@@ -152,21 +153,23 @@
|
|
| 152 |
"batch.collecting": "Collecting",
|
| 153 |
"batch.complete": "Complete",
|
| 154 |
|
| 155 |
-
"fit.ship": "
|
| 156 |
"fit.bench": "No benchmark overlap",
|
| 157 |
"fit.pii": "No personal data",
|
| 158 |
"fit.size": "Files up to 20 MB",
|
| 159 |
"fit.alone": "Unit stands alone",
|
| 160 |
"fit.answers": "Answers or expected results",
|
| 161 |
"fit.login": "No login",
|
|
|
|
| 162 |
"fit.dot": "{{label}}: {{state}}",
|
| 163 |
-
"fitShort.ship": "
|
| 164 |
"fitShort.bench": "Bench",
|
| 165 |
"fitShort.pii": "PII",
|
| 166 |
"fitShort.size": "Size",
|
| 167 |
"fitShort.alone": "Alone",
|
| 168 |
"fitShort.answers": "Answer",
|
| 169 |
"fitShort.login": "Login",
|
|
|
|
| 170 |
"fitWord.yes": "Fits",
|
| 171 |
"fitWord.part": "Partly",
|
| 172 |
"fitWord.no": "Does not fit",
|
|
@@ -174,7 +177,7 @@
|
|
| 174 |
|
| 175 |
"material.before": "Where the",
|
| 176 |
"material.after": "units are",
|
| 177 |
-
"material.cats": "Most requested
|
| 178 |
"kind.material_set": "Material sets",
|
| 179 |
"kind.corpus": "Corpora",
|
| 180 |
"kindOne.material_set": "Material set",
|
|
@@ -192,10 +195,10 @@
|
|
| 192 |
"coll.unitsAll_other": "{{n}} units in all",
|
| 193 |
"coll.open": "Open its material sets",
|
| 194 |
"coll.showing": "Showing its material sets",
|
| 195 |
-
"cat.shippable": "
|
| 196 |
-
"cat.shippableRaw": "
|
| 197 |
-
"cat.shipTipReady": "of the units may be
|
| 198 |
-
"cat.shipTipRaw": "of the rows in corpora may be
|
| 199 |
"cat.sample": "Take a sample",
|
| 200 |
|
| 201 |
"stack.otherForms_one": "Other {{n}} form",
|
|
@@ -233,7 +236,7 @@
|
|
| 233 |
"filter.status": "Status",
|
| 234 |
"filter.status.flagged": "Flagged",
|
| 235 |
"filter.training": "Training",
|
| 236 |
-
"filter.shipping": "
|
| 237 |
"filter.permit.yes": "Allowed, all rows",
|
| 238 |
"filter.permit.cond": "Allowed or conditional",
|
| 239 |
"filter.tier": "License tier",
|
|
@@ -277,7 +280,7 @@
|
|
| 277 |
"map.title": "{{metric}}: domain × form",
|
| 278 |
|
| 279 |
"col.title": "Source",
|
| 280 |
-
"col.domain": "
|
| 281 |
"col.form": "Form",
|
| 282 |
"col.units": "Units",
|
| 283 |
"col.rows": "Rows",
|
|
@@ -309,6 +312,13 @@
|
|
| 309 |
"size.distinct": "{{size}} deduplicated size",
|
| 310 |
"size.check": "Size check: {{share}} match ({{n}} rows compared)",
|
| 311 |
"size.checkNone": "Size check: no row compared yet",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 312 |
"size.estimate": "estimate for the {{rows}} rows without a size: {{size}} (95% range {{low}} to {{high}}{{summed}}; from {{readings}} rows read)",
|
| 313 |
"size.estimatePart": "estimate for {{rows}} of the {{all}} rows without a size: {{size}} (95% range {{low}} to {{high}}{{summed}}; from {{readings}} rows read; {{rest}} rows have too few rows read for one)",
|
| 314 |
"size.estimateNone": "{{rows}} rows without a size, too few read for an estimate",
|
|
@@ -319,6 +329,7 @@
|
|
| 319 |
"size.how": "How sizes are measured",
|
| 320 |
"size.checkSentence": "Size check: {{n}} rows were read again from their source and their size and hash compared with those the index records; {{share}} of them match, that is, the file is still the one the index describes.",
|
| 321 |
"size.checkNoneSentence": "Size check: no row has been read again from its source and compared with the size and hash the index records yet.",
|
|
|
|
| 322 |
"size.unsizedEst": "≈ {{size}} more without a recorded size (est.)",
|
| 323 |
"size.unsizedNone": "{{rows}} rows without a recorded size",
|
| 324 |
|
|
@@ -347,7 +358,7 @@
|
|
| 347 |
"detail.anyEditor": "Any editor",
|
| 348 |
"detail.licence": "License",
|
| 349 |
"detail.training": "Training",
|
| 350 |
-
"detail.shipping": "
|
| 351 |
"detail.shareOfSet": "of the files of this material set ({{licences}})",
|
| 352 |
"detail.shareOfSetOnly": "of the files of this material set",
|
| 353 |
"detail.shareOfCorpus": "of the rows of this corpus ({{licences}})",
|
|
@@ -364,6 +375,11 @@
|
|
| 364 |
"detail.size": "Size",
|
| 365 |
"detail.median": "median {{size}}",
|
| 366 |
"detail.stripAria": "median {{median}}, 99% under {{p99}}",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 367 |
"detail.example": "Example",
|
| 368 |
"detail.take": "Take it",
|
| 369 |
"detail.more": "Batches and first rows",
|
|
@@ -391,6 +407,7 @@
|
|
| 391 |
"sql.headSets": "Agentic Env Index @ {{sha}}: {{sets}} ({{units}})",
|
| 392 |
"sql.headCorpora": "Agentic Env Index @ {{sha}}: {{corpora}} ({{rows}})",
|
| 393 |
"sql.all": "no filter",
|
|
|
|
| 394 |
"sql.chip": "{{k}}: {{v}}",
|
| 395 |
"csv.progress_one": "{{done}}/{{n}} file",
|
| 396 |
"csv.progress_other": "{{done}}/{{n}} files",
|
|
@@ -413,6 +430,10 @@
|
|
| 413 |
"attr.source": "Source",
|
| 414 |
"attr.licence": "License",
|
| 415 |
"attr.licenceUrl": "License URL",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 416 |
"attr.attribution": "Attribution",
|
| 417 |
"attr.notStated": "not stated",
|
| 418 |
"tree.title": "What it writes",
|
|
@@ -424,7 +445,7 @@
|
|
| 424 |
"tree.leftOut": "Left out by default",
|
| 425 |
"profile.duplicates": "Duplicates",
|
| 426 |
"profile.benchmark": "Benchmark copies",
|
| 427 |
-
"profile.licence": "No
|
| 428 |
"profile.english": "Not English",
|
| 429 |
"profile.personal": "Personal data",
|
| 430 |
"profile.large": "Over 20 MB",
|
|
@@ -433,6 +454,10 @@
|
|
| 433 |
"video.title": "Usage example video",
|
| 434 |
"video.text": "A usage example video, in Chinese and in English with subtitles, shows the index in use from this page. It plays in the page's language; media/ of the dataset holds both videos and their subtitle files.",
|
| 435 |
"video.subtitles": "English",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 436 |
|
| 437 |
"history.title": "What each release added",
|
| 438 |
"history.start": "start",
|
|
|
|
| 3 |
"lang.switch": "Show this page in Chinese",
|
| 4 |
|
| 5 |
"page.title": "Agentic Env Index",
|
| 6 |
+
"page.description": "Real material for terminal-agent tasks from the public pointer index Battam/agentic-env-index: by domain, fit at a glance, one command to take it.",
|
| 7 |
|
| 8 |
"nav.sections": "Page sections",
|
| 9 |
"nav.overview": "Overview",
|
|
|
|
| 15 |
"nav.releases": "Releases",
|
| 16 |
"nav.releasesShort": "Releases",
|
| 17 |
"nav.agents": "For agents",
|
| 18 |
+
"nav.manual": "Manual",
|
| 19 |
"nav.request": "Request data",
|
| 20 |
|
| 21 |
+
"hero.eyebrow": "The Agentic Env Index, by domain",
|
| 22 |
"hero.title": "Real material for agent tasks",
|
| 23 |
+
"hero.browse": "Browse domains",
|
| 24 |
"hero.facts": "Facts",
|
| 25 |
"hero.indexOf": "Index of",
|
| 26 |
"hero.commit": "Commit {{sha}}",
|
|
|
|
| 39 |
"url.docsSources": "https://huggingface.co/datasets/Battam/agentic-env-index/blob/main/docs/sources.md",
|
| 40 |
|
| 41 |
"foot.missing": "Missing material?",
|
| 42 |
+
"foot.ask": "Ask for a domain, a form or a source.",
|
| 43 |
"foot.top": "Back to top",
|
| 44 |
"foot.index": "Index {{sha}} · {{date}}",
|
| 45 |
|
|
|
|
| 127 |
"permit.yes": "Allowed",
|
| 128 |
"permit.no": "Not allowed",
|
| 129 |
"permit.conditional": "Conditional",
|
| 130 |
+
"permit.conditional.sameLicence": "Conditional (redistribute under the same license)",
|
| 131 |
"permit.conditional.nonCommercial": "Conditional (non-commercial or research use only)",
|
| 132 |
"permit.unknown": "Not stated",
|
| 133 |
"unitKind.file": "a file",
|
|
|
|
| 153 |
"batch.collecting": "Collecting",
|
| 154 |
"batch.complete": "Complete",
|
| 155 |
|
| 156 |
+
"fit.ship": "Redistributable license",
|
| 157 |
"fit.bench": "No benchmark overlap",
|
| 158 |
"fit.pii": "No personal data",
|
| 159 |
"fit.size": "Files up to 20 MB",
|
| 160 |
"fit.alone": "Unit stands alone",
|
| 161 |
"fit.answers": "Answers or expected results",
|
| 162 |
"fit.login": "No login",
|
| 163 |
+
"fit.fetch": "Upstream serves the files",
|
| 164 |
"fit.dot": "{{label}}: {{state}}",
|
| 165 |
+
"fitShort.ship": "Redist.",
|
| 166 |
"fitShort.bench": "Bench",
|
| 167 |
"fitShort.pii": "PII",
|
| 168 |
"fitShort.size": "Size",
|
| 169 |
"fitShort.alone": "Alone",
|
| 170 |
"fitShort.answers": "Answer",
|
| 171 |
"fitShort.login": "Login",
|
| 172 |
+
"fitShort.fetch": "Fetch",
|
| 173 |
"fitWord.yes": "Fits",
|
| 174 |
"fitWord.part": "Partly",
|
| 175 |
"fitWord.no": "Does not fit",
|
|
|
|
| 177 |
|
| 178 |
"material.before": "Where the",
|
| 179 |
"material.after": "units are",
|
| 180 |
+
"material.cats": "Most requested domains",
|
| 181 |
"kind.material_set": "Material sets",
|
| 182 |
"kind.corpus": "Corpora",
|
| 183 |
"kindOne.material_set": "Material set",
|
|
|
|
| 195 |
"coll.unitsAll_other": "{{n}} units in all",
|
| 196 |
"coll.open": "Open its material sets",
|
| 197 |
"coll.showing": "Showing its material sets",
|
| 198 |
+
"cat.shippable": "Redistributable, material sets",
|
| 199 |
+
"cat.shippableRaw": "Redistributable, corpora",
|
| 200 |
+
"cat.shipTipReady": "of the units may be redistributed",
|
| 201 |
+
"cat.shipTipRaw": "of the rows in corpora may be redistributed",
|
| 202 |
"cat.sample": "Take a sample",
|
| 203 |
|
| 204 |
"stack.otherForms_one": "Other {{n}} form",
|
|
|
|
| 236 |
"filter.status": "Status",
|
| 237 |
"filter.status.flagged": "Flagged",
|
| 238 |
"filter.training": "Training",
|
| 239 |
+
"filter.shipping": "Redistribution",
|
| 240 |
"filter.permit.yes": "Allowed, all rows",
|
| 241 |
"filter.permit.cond": "Allowed or conditional",
|
| 242 |
"filter.tier": "License tier",
|
|
|
|
| 280 |
"map.title": "{{metric}}: domain × form",
|
| 281 |
|
| 282 |
"col.title": "Source",
|
| 283 |
+
"col.domain": "Domain",
|
| 284 |
"col.form": "Form",
|
| 285 |
"col.units": "Units",
|
| 286 |
"col.rows": "Rows",
|
|
|
|
| 312 |
"size.distinct": "{{size}} deduplicated size",
|
| 313 |
"size.check": "Size check: {{share}} match ({{n}} rows compared)",
|
| 314 |
"size.checkNone": "Size check: no row compared yet",
|
| 315 |
+
"size.checkNotCompared": "Size check: none of the {{read}} rows read could be compared ({{why}})",
|
| 316 |
+
"size.listSep": ", ",
|
| 317 |
+
"size.status.unreachable": "address unreachable {{n}}",
|
| 318 |
+
"size.status.gone": "file gone {{n}}",
|
| 319 |
+
"size.status.needs_auth": "login needed {{n}}",
|
| 320 |
+
"size.status.no_length": "no size from the server {{n}}",
|
| 321 |
+
"size.status.error": "other error {{n}}",
|
| 322 |
"size.estimate": "estimate for the {{rows}} rows without a size: {{size}} (95% range {{low}} to {{high}}{{summed}}; from {{readings}} rows read)",
|
| 323 |
"size.estimatePart": "estimate for {{rows}} of the {{all}} rows without a size: {{size}} (95% range {{low}} to {{high}}{{summed}}; from {{readings}} rows read; {{rest}} rows have too few rows read for one)",
|
| 324 |
"size.estimateNone": "{{rows}} rows without a size, too few read for an estimate",
|
|
|
|
| 329 |
"size.how": "How sizes are measured",
|
| 330 |
"size.checkSentence": "Size check: {{n}} rows were read again from their source and their size and hash compared with those the index records; {{share}} of them match, that is, the file is still the one the index describes.",
|
| 331 |
"size.checkNoneSentence": "Size check: no row has been read again from its source and compared with the size and hash the index records yet.",
|
| 332 |
+
"size.checkNotComparedSentence": "Size check: {{read}} rows were read again from their source, but none could be compared with the size and hash the index records ({{why}}).",
|
| 333 |
"size.unsizedEst": "≈ {{size}} more without a recorded size (est.)",
|
| 334 |
"size.unsizedNone": "{{rows}} rows without a recorded size",
|
| 335 |
|
|
|
|
| 358 |
"detail.anyEditor": "Any editor",
|
| 359 |
"detail.licence": "License",
|
| 360 |
"detail.training": "Training",
|
| 361 |
+
"detail.shipping": "Redistribution",
|
| 362 |
"detail.shareOfSet": "of the files of this material set ({{licences}})",
|
| 363 |
"detail.shareOfSetOnly": "of the files of this material set",
|
| 364 |
"detail.shareOfCorpus": "of the rows of this corpus ({{licences}})",
|
|
|
|
| 375 |
"detail.size": "Size",
|
| 376 |
"detail.median": "median {{size}}",
|
| 377 |
"detail.stripAria": "median {{median}}, 99% under {{p99}}",
|
| 378 |
+
"detail.fetch": "Fetch check",
|
| 379 |
+
"fetch.all": "All {{rows}} sampled rows read fetched and matched their recorded size and hash (read {{date}}).",
|
| 380 |
+
"fetch.some": "{{fetched}} of the {{rows}} sampled rows read ({{share}}) fetched and matched their recorded size and hash (read {{date}}); the others did not.",
|
| 381 |
+
"fetch.none": "None of the {{rows}} sampled rows read fetched and matched their recorded size and hash (read {{date}}): the upstream does not serve these files now.",
|
| 382 |
+
"fetch.notRead": "No sampled row read yet.",
|
| 383 |
"detail.example": "Example",
|
| 384 |
"detail.take": "Take it",
|
| 385 |
"detail.more": "Batches and first rows",
|
|
|
|
| 407 |
"sql.headSets": "Agentic Env Index @ {{sha}}: {{sets}} ({{units}})",
|
| 408 |
"sql.headCorpora": "Agentic Env Index @ {{sha}}: {{corpora}} ({{rows}})",
|
| 409 |
"sql.all": "no filter",
|
| 410 |
+
"sql.noProfile": "Rows as published: this query applies no rule of the default profile of tools/fetch.py (no deduplication; benchmark copies, rows not compared with the benchmark list, licenses that do not allow redistribution, languages other than English, personal data and files over 20 MB all stay in); the command applies them.",
|
| 411 |
"sql.chip": "{{k}}: {{v}}",
|
| 412 |
"csv.progress_one": "{{done}}/{{n}} file",
|
| 413 |
"csv.progress_other": "{{done}}/{{n}} files",
|
|
|
|
| 430 |
"attr.source": "Source",
|
| 431 |
"attr.licence": "License",
|
| 432 |
"attr.licenceUrl": "License URL",
|
| 433 |
+
"attr.files": "Files by publisher, license and license URL (each file's own; its source's where the file names none)",
|
| 434 |
+
"attr.fileLine_one": "{{publisher}}: {{licence}}, {{url}} ({{n}} file)",
|
| 435 |
+
"attr.fileLine_other": "{{publisher}}: {{licence}}, {{url}} ({{n}} files)",
|
| 436 |
+
"attr.noPublisher": "publisher not stated",
|
| 437 |
"attr.attribution": "Attribution",
|
| 438 |
"attr.notStated": "not stated",
|
| 439 |
"tree.title": "What it writes",
|
|
|
|
| 445 |
"tree.leftOut": "Left out by default",
|
| 446 |
"profile.duplicates": "Duplicates",
|
| 447 |
"profile.benchmark": "Benchmark copies",
|
| 448 |
+
"profile.licence": "No redistribution license",
|
| 449 |
"profile.english": "Not English",
|
| 450 |
"profile.personal": "Personal data",
|
| 451 |
"profile.large": "Over 20 MB",
|
|
|
|
| 454 |
"video.title": "Usage example video",
|
| 455 |
"video.text": "A usage example video, in Chinese and in English with subtitles, shows the index in use from this page. It plays in the page's language; media/ of the dataset holds both videos and their subtitle files.",
|
| 456 |
"video.subtitles": "English",
|
| 457 |
+
"manual.title": "User manual",
|
| 458 |
+
"manual.back": "Back to the page",
|
| 459 |
+
"manual.loading": "Loading the manual…",
|
| 460 |
+
"manual.missing": "The manual is not on this page yet: {{error}}",
|
| 461 |
|
| 462 |
"history.title": "What each release added",
|
| 463 |
"history.start": "start",
|
locales/zh.json
CHANGED
|
@@ -3,7 +3,7 @@
|
|
| 3 |
"lang.switch": "以英文显示本页",
|
| 4 |
|
| 5 |
"page.title": "Agentic Env 索引",
|
| 6 |
-
"page.description": "来自公开指针索引 Battam/agentic-env-index 的终端智能体任务真实素材:按
|
| 7 |
|
| 8 |
"nav.sections": "页面分区",
|
| 9 |
"nav.overview": "概览",
|
|
@@ -15,11 +15,12 @@
|
|
| 15 |
"nav.releases": "发布",
|
| 16 |
"nav.releasesShort": "发布",
|
| 17 |
"nav.agents": "给智能体",
|
|
|
|
| 18 |
"nav.request": "申请数据",
|
| 19 |
|
| 20 |
-
"hero.eyebrow": "Agentic Env 索引,按
|
| 21 |
"hero.title": "面向智能体任务的真实素材",
|
| 22 |
-
"hero.browse": "浏览
|
| 23 |
"hero.facts": "概况",
|
| 24 |
"hero.indexOf": "索引版本",
|
| 25 |
"hero.commit": "提交 {{sha}}",
|
|
@@ -38,13 +39,13 @@
|
|
| 38 |
"url.docsSources": "https://huggingface.co/datasets/Battam/agentic-env-index/blob/main/docs/sources.zh.md",
|
| 39 |
|
| 40 |
"foot.missing": "缺少素材?",
|
| 41 |
-
"foot.ask": "可以申请一个
|
| 42 |
"foot.top": "回到顶部",
|
| 43 |
"foot.index": "索引 {{sha}} · {{date}}",
|
| 44 |
|
| 45 |
"notice.vocabulary": "无法读取词表({{error}});代码按原样显示。",
|
| 46 |
"notice.failed": "无法读取索引:{{error}}",
|
| 47 |
-
"notice.linked": "联动图表和
|
| 48 |
"error.noCommit": "Hub 未返回提交 ID",
|
| 49 |
|
| 50 |
"num.unknown": "未知",
|
|
@@ -65,7 +66,7 @@
|
|
| 65 |
|
| 66 |
"n.materialSet_other": "{{n}} 个材料集",
|
| 67 |
"n.corpus_other": "{{n}} 个语料库",
|
| 68 |
-
"n.picked_other": "{{n}} 个已挑选的
|
| 69 |
"n.domain_other": "{{n}} 个领域",
|
| 70 |
"n.unit_other": "{{n}} 个单元",
|
| 71 |
"n.unitFile_other": "{{n}} 个单元文件",
|
|
@@ -113,7 +114,7 @@
|
|
| 113 |
"permit.yes": "允许",
|
| 114 |
"permit.no": "不允许",
|
| 115 |
"permit.conditional": "有条件",
|
| 116 |
-
"permit.conditional.sameLicence": "有条件(
|
| 117 |
"permit.conditional.nonCommercial": "有条件(仅限非商业或研究用途)",
|
| 118 |
"permit.unknown": "未注明",
|
| 119 |
"unitKind.file": "一个文件",
|
|
@@ -133,21 +134,23 @@
|
|
| 133 |
"batch.collecting": "收集中",
|
| 134 |
"batch.complete": "已完成",
|
| 135 |
|
| 136 |
-
"fit.ship": "许可证允许分发",
|
| 137 |
"fit.bench": "与基准无重合",
|
| 138 |
"fit.pii": "无个人数据",
|
| 139 |
"fit.size": "文件不超过 20 MB",
|
| 140 |
"fit.alone": "单元可独立使用",
|
| 141 |
"fit.answers": "有答案或预期结果",
|
| 142 |
"fit.login": "无需登录",
|
|
|
|
| 143 |
"fit.dot": "{{label}}:{{state}}",
|
| 144 |
-
"fitShort.ship": "分发",
|
| 145 |
"fitShort.bench": "基准",
|
| 146 |
"fitShort.pii": "隐私",
|
| 147 |
"fitShort.size": "大小",
|
| 148 |
"fitShort.alone": "独立",
|
| 149 |
"fitShort.answers": "答案",
|
| 150 |
"fitShort.login": "登录",
|
|
|
|
| 151 |
"fitWord.yes": "适配",
|
| 152 |
"fitWord.part": "部分适配",
|
| 153 |
"fitWord.no": "不适配",
|
|
@@ -155,7 +158,7 @@
|
|
| 155 |
|
| 156 |
"material.before": "这",
|
| 157 |
"material.after": "个单元的分布",
|
| 158 |
-
"material.cats": "需求最多的
|
| 159 |
"kind.material_set": "材料集",
|
| 160 |
"kind.corpus": "语料库",
|
| 161 |
"kindOne.material_set": "材料集",
|
|
@@ -169,10 +172,10 @@
|
|
| 169 |
"coll.unitsAll_other": "共 {{n}} 个单元",
|
| 170 |
"coll.open": "打开它的材料集",
|
| 171 |
"coll.showing": "正在显示它的材料集",
|
| 172 |
-
"cat.shippable": "可分发(材料集)",
|
| 173 |
-
"cat.shippableRaw": "可分发(语料库)",
|
| 174 |
-
"cat.shipTipReady": "的单元可以分发",
|
| 175 |
-
"cat.shipTipRaw": "的语料库数据行可以分发",
|
| 176 |
"cat.sample": "取样",
|
| 177 |
|
| 178 |
"stack.otherForms_other": "其他 {{n}} 种形态",
|
|
@@ -206,11 +209,11 @@
|
|
| 206 |
"filter.status": "状态",
|
| 207 |
"filter.status.flagged": "已标记",
|
| 208 |
"filter.training": "训练",
|
| 209 |
-
"filter.shipping": "分发",
|
| 210 |
"filter.permit.yes": "允许,所有行",
|
| 211 |
"filter.permit.cond": "允许或有条件",
|
| 212 |
"filter.tier": "许可证等级",
|
| 213 |
-
"filter.size": "
|
| 214 |
"filter.size.lt1g": "小于 1 GB",
|
| 215 |
"filter.size.1g1t": "1 GB 到 1 TB",
|
| 216 |
"filter.size.ge1t": "1 TB 及以上",
|
|
@@ -247,8 +250,8 @@
|
|
| 247 |
"map.head": "{{amount}},{{domains}}",
|
| 248 |
"map.title": "{{metric}}:领域 × 形态",
|
| 249 |
|
| 250 |
-
"col.title": "
|
| 251 |
-
"col.domain": "
|
| 252 |
"col.form": "形态",
|
| 253 |
"col.units": "单元数",
|
| 254 |
"col.rows": "行数",
|
|
@@ -257,7 +260,7 @@
|
|
| 257 |
"col.fit": "适配",
|
| 258 |
"sources.headSets": "{{sets}}",
|
| 259 |
"sources.headCorpora": "{{corpora}}",
|
| 260 |
-
"sources.none": "没有匹配的
|
| 261 |
"sources.unitsLabel": "单元",
|
| 262 |
"sources.rowsLabel": "语料库行数",
|
| 263 |
"sources.unitWord_other": " 个单元",
|
|
@@ -269,7 +272,7 @@
|
|
| 269 |
"preview.hide": "收起行",
|
| 270 |
"preview.title": "前 {{n}} 行,单元文件在前(meta/preview.parquet)",
|
| 271 |
"preview.loading": "正在读取前几行…",
|
| 272 |
-
"preview.none": "本次发布还没有这个
|
| 273 |
"preview.failed": "无法读取前几行:{{error}}",
|
| 274 |
"size.note": "{{layer}}:{{line}}。",
|
| 275 |
"size.index": "整个索引:",
|
|
@@ -278,16 +281,24 @@
|
|
| 278 |
"size.distinct": "去重后大小 {{size}}",
|
| 279 |
"size.check": "大小核对:{{share}} 一致(比对了 {{n}} 行)",
|
| 280 |
"size.checkNone": "大小核对:还没有比对任何行",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 281 |
"size.estimate": "没有大小的 {{rows}} 行的估计值:{{size}}(95% 范围 {{low}} 至 {{high}}{{summed}};依据读过的 {{readings}} 行)",
|
| 282 |
"size.estimatePart": "没有大小的 {{all}} 行中 {{rows}} 行的估计值:{{size}}(95% 范围 {{low}} 至 {{high}}{{summed}};依据读过的 {{readings}} 行;其余 {{rest}} 行读过的太少,没有估计值)",
|
| 283 |
"size.estimateNone": "{{rows}} 行没有大小,读过的太少,没有估计值",
|
| 284 |
-
"size.summed": ",为各
|
| 285 |
"size.none": "本次发布尚未发布大小数字",
|
| 286 |
"size.recordedShort": "记录 {{size}}",
|
| 287 |
"size.distinctShort": "(去重后 {{size}})",
|
| 288 |
"size.how": "大小如何测量",
|
| 289 |
-
"size.checkSentence": "大小核对:从
|
| 290 |
-
"size.checkNoneSentence": "大小核对:还没有任何行从
|
|
|
|
| 291 |
"size.unsizedEst": "另有约 {{size}} 没有记录大小(估)",
|
| 292 |
"size.unsizedNone": "{{rows}} 行没有记录大小",
|
| 293 |
|
|
@@ -298,7 +309,7 @@
|
|
| 298 |
"detail.countsByFirst": "{{what}};前几行由 {{publishers}} 发布。",
|
| 299 |
"detail.oneUnit": "一个单元",
|
| 300 |
"detail.files": "文件",
|
| 301 |
-
"detail.from": "
|
| 302 |
"detail.publishers_other": "{{n}} 个发布方;主要网站",
|
| 303 |
"detail.notRecorded": "未记录",
|
| 304 |
"detail.alone": "可独立使用",
|
|
@@ -315,7 +326,7 @@
|
|
| 315 |
"detail.anyEditor": "任意编辑器",
|
| 316 |
"detail.licence": "许可证",
|
| 317 |
"detail.training": "训练",
|
| 318 |
-
"detail.shipping": "分发",
|
| 319 |
"detail.shareOfSet": "(占本材料集的文件;{{licences}})",
|
| 320 |
"detail.shareOfSetOnly": "(占本材料集的文件)",
|
| 321 |
"detail.shareOfCorpus": "(占本语料库的行;{{licences}})",
|
|
@@ -330,6 +341,11 @@
|
|
| 330 |
"detail.size": "大小",
|
| 331 |
"detail.median": "中位数 {{size}}",
|
| 332 |
"detail.stripAria": "中位数 {{median}},99% 小于 {{p99}}",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 333 |
"detail.example": "示例",
|
| 334 |
"detail.take": "获取",
|
| 335 |
"detail.more": "批次与前几行",
|
|
@@ -353,6 +369,7 @@
|
|
| 353 |
"sql.headSets": "Agentic Env 索引 @ {{sha}}:{{sets}}({{units}})",
|
| 354 |
"sql.headCorpora": "Agentic Env 索引 @ {{sha}}:{{corpora}}({{rows}})",
|
| 355 |
"sql.all": "无筛选",
|
|
|
|
| 356 |
"sql.chip": "{{k}}:{{v}}",
|
| 357 |
"csv.progress_other": "{{done}}/{{n}} 个文件",
|
| 358 |
"csv.button": "CSV · {{rows}}",
|
|
@@ -364,26 +381,29 @@
|
|
| 364 |
"keep.again": "按文件再取一次",
|
| 365 |
"pick.add": "挑选",
|
| 366 |
"pick.picked": "已挑选",
|
| 367 |
-
"pick.word_other": "个
|
| 368 |
"pick.clear": "清除挑选",
|
| 369 |
-
"pick.note": "这条命令只取已挑选的
|
| 370 |
-
"attr.title_other": "署名:{{dataset}} 的 {{n}} 个
|
| 371 |
"attr.commit": "数据集提交 {{sha}}:{{url}}",
|
| 372 |
-
"attr.source": "
|
| 373 |
-
"attr.licence": "许可",
|
| 374 |
-
"attr.licenceUrl": "许可链接",
|
|
|
|
|
|
|
|
|
|
| 375 |
"attr.attribution": "署名",
|
| 376 |
"attr.notStated": "未注明",
|
| 377 |
"tree.title": "写入内容",
|
| 378 |
"tree.units": "单元、README、LICENSE",
|
| 379 |
"tree.answers": "答案,单独存放",
|
| 380 |
"tree.manifest": "每个文件一行,已校验 sha256",
|
| 381 |
-
"tree.sizes": "每个
|
| 382 |
"tree.domainNote": ":例如 D3_data 或 D3",
|
| 383 |
"tree.leftOut": "默认排除",
|
| 384 |
"profile.duplicates": "重复内容",
|
| 385 |
"profile.benchmark": "基准副本",
|
| 386 |
-
"profile.licence": "许可证不允许分发",
|
| 387 |
"profile.english": "非英文",
|
| 388 |
"profile.personal": "个人数据",
|
| 389 |
"profile.large": "超过 20 MB",
|
|
@@ -392,6 +412,10 @@
|
|
| 392 |
"video.title": "使用示例视频",
|
| 393 |
"video.text": "一段使用示例视频有中文版和英文版,各带字幕,演示了如何从本页使用这份索引。这里按页面语言播放;两版视频和字幕文件都在数据集的 media/ 下。",
|
| 394 |
"video.subtitles": "中文",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 395 |
|
| 396 |
"history.title": "每次发布新增了什么",
|
| 397 |
"history.start": "起点",
|
|
|
|
| 3 |
"lang.switch": "以英文显示本页",
|
| 4 |
|
| 5 |
"page.title": "Agentic Env 索引",
|
| 6 |
+
"page.description": "来自公开指针索引 Battam/agentic-env-index 的终端智能体任务真实素材:按领域浏览,一眼看出是否适配,一条命令即可获取。",
|
| 7 |
|
| 8 |
"nav.sections": "页面分区",
|
| 9 |
"nav.overview": "概览",
|
|
|
|
| 15 |
"nav.releases": "发布",
|
| 16 |
"nav.releasesShort": "发布",
|
| 17 |
"nav.agents": "给智能体",
|
| 18 |
+
"nav.manual": "手册",
|
| 19 |
"nav.request": "申请数据",
|
| 20 |
|
| 21 |
+
"hero.eyebrow": "Agentic Env 索引,按领域",
|
| 22 |
"hero.title": "面向智能体任务的真实素材",
|
| 23 |
+
"hero.browse": "浏览领域",
|
| 24 |
"hero.facts": "概况",
|
| 25 |
"hero.indexOf": "索引版本",
|
| 26 |
"hero.commit": "提交 {{sha}}",
|
|
|
|
| 39 |
"url.docsSources": "https://huggingface.co/datasets/Battam/agentic-env-index/blob/main/docs/sources.zh.md",
|
| 40 |
|
| 41 |
"foot.missing": "缺少素材?",
|
| 42 |
+
"foot.ask": "可以申请一个领域、一种形态或一个来源。",
|
| 43 |
"foot.top": "回到顶部",
|
| 44 |
"foot.index": "索引 {{sha}} · {{date}}",
|
| 45 |
|
| 46 |
"notice.vocabulary": "无法读取词表({{error}});代码按原样显示。",
|
| 47 |
"notice.failed": "无法读取索引:{{error}}",
|
| 48 |
+
"notice.linked": "联动图表和来源列表无法启动:{{error}}",
|
| 49 |
"error.noCommit": "Hub 未返回提交 ID",
|
| 50 |
|
| 51 |
"num.unknown": "未知",
|
|
|
|
| 66 |
|
| 67 |
"n.materialSet_other": "{{n}} 个材料集",
|
| 68 |
"n.corpus_other": "{{n}} 个语料库",
|
| 69 |
+
"n.picked_other": "{{n}} 个已挑选的来源",
|
| 70 |
"n.domain_other": "{{n}} 个领域",
|
| 71 |
"n.unit_other": "{{n}} 个单元",
|
| 72 |
"n.unitFile_other": "{{n}} 个单元文件",
|
|
|
|
| 114 |
"permit.yes": "允许",
|
| 115 |
"permit.no": "不允许",
|
| 116 |
"permit.conditional": "有条件",
|
| 117 |
+
"permit.conditional.sameLicence": "有条件(以同样的许可证再分发)",
|
| 118 |
"permit.conditional.nonCommercial": "有条件(仅限非商业或研究用途)",
|
| 119 |
"permit.unknown": "未注明",
|
| 120 |
"unitKind.file": "一个文件",
|
|
|
|
| 134 |
"batch.collecting": "收集中",
|
| 135 |
"batch.complete": "已完成",
|
| 136 |
|
| 137 |
+
"fit.ship": "许可证允许再分发",
|
| 138 |
"fit.bench": "与基准无重合",
|
| 139 |
"fit.pii": "无个人数据",
|
| 140 |
"fit.size": "文件不超过 20 MB",
|
| 141 |
"fit.alone": "单元可独立使用",
|
| 142 |
"fit.answers": "有答案或预期结果",
|
| 143 |
"fit.login": "无需登录",
|
| 144 |
+
"fit.fetch": "上游提供文件",
|
| 145 |
"fit.dot": "{{label}}:{{state}}",
|
| 146 |
+
"fitShort.ship": "再分发",
|
| 147 |
"fitShort.bench": "基准",
|
| 148 |
"fitShort.pii": "隐私",
|
| 149 |
"fitShort.size": "大小",
|
| 150 |
"fitShort.alone": "独立",
|
| 151 |
"fitShort.answers": "答案",
|
| 152 |
"fitShort.login": "登录",
|
| 153 |
+
"fitShort.fetch": "取回",
|
| 154 |
"fitWord.yes": "适配",
|
| 155 |
"fitWord.part": "部分适配",
|
| 156 |
"fitWord.no": "不适配",
|
|
|
|
| 158 |
|
| 159 |
"material.before": "这",
|
| 160 |
"material.after": "个单元的分布",
|
| 161 |
+
"material.cats": "需求最多的领域",
|
| 162 |
"kind.material_set": "材料集",
|
| 163 |
"kind.corpus": "语料库",
|
| 164 |
"kindOne.material_set": "材料集",
|
|
|
|
| 172 |
"coll.unitsAll_other": "共 {{n}} 个单元",
|
| 173 |
"coll.open": "打开它的材料集",
|
| 174 |
"coll.showing": "正在显示它的材料集",
|
| 175 |
+
"cat.shippable": "可再分发(材料集)",
|
| 176 |
+
"cat.shippableRaw": "可再分发(语料库)",
|
| 177 |
+
"cat.shipTipReady": "的单元可以再分发",
|
| 178 |
+
"cat.shipTipRaw": "的语料库数据行可以再分发",
|
| 179 |
"cat.sample": "取样",
|
| 180 |
|
| 181 |
"stack.otherForms_other": "其他 {{n}} 种形态",
|
|
|
|
| 209 |
"filter.status": "状态",
|
| 210 |
"filter.status.flagged": "已标记",
|
| 211 |
"filter.training": "训练",
|
| 212 |
+
"filter.shipping": "再分发",
|
| 213 |
"filter.permit.yes": "允许,所有行",
|
| 214 |
"filter.permit.cond": "允许或有条件",
|
| 215 |
"filter.tier": "许可证等级",
|
| 216 |
+
"filter.size": "来源大小",
|
| 217 |
"filter.size.lt1g": "小于 1 GB",
|
| 218 |
"filter.size.1g1t": "1 GB 到 1 TB",
|
| 219 |
"filter.size.ge1t": "1 TB 及以上",
|
|
|
|
| 250 |
"map.head": "{{amount}},{{domains}}",
|
| 251 |
"map.title": "{{metric}}:领域 × 形态",
|
| 252 |
|
| 253 |
+
"col.title": "来源",
|
| 254 |
+
"col.domain": "领域",
|
| 255 |
"col.form": "形态",
|
| 256 |
"col.units": "单元数",
|
| 257 |
"col.rows": "行数",
|
|
|
|
| 260 |
"col.fit": "适配",
|
| 261 |
"sources.headSets": "{{sets}}",
|
| 262 |
"sources.headCorpora": "{{corpora}}",
|
| 263 |
+
"sources.none": "没有匹配的来源",
|
| 264 |
"sources.unitsLabel": "单元",
|
| 265 |
"sources.rowsLabel": "语料库行数",
|
| 266 |
"sources.unitWord_other": " 个单元",
|
|
|
|
| 272 |
"preview.hide": "收起行",
|
| 273 |
"preview.title": "前 {{n}} 行,单元文件在前(meta/preview.parquet)",
|
| 274 |
"preview.loading": "正在读取前几行…",
|
| 275 |
+
"preview.none": "本次发布还没有这个来源的预览行。",
|
| 276 |
"preview.failed": "无法读取前几行:{{error}}",
|
| 277 |
"size.note": "{{layer}}:{{line}}。",
|
| 278 |
"size.index": "整个索引:",
|
|
|
|
| 281 |
"size.distinct": "去重后大小 {{size}}",
|
| 282 |
"size.check": "大小核对:{{share}} 一致(比对了 {{n}} 行)",
|
| 283 |
"size.checkNone": "大小核对:还没有比对任何行",
|
| 284 |
+
"size.checkNotCompared": "大小核对:读过的 {{read}} 行都没能比对({{why}})",
|
| 285 |
+
"size.listSep": "、",
|
| 286 |
+
"size.status.unreachable": "地址无法访问 {{n}} 行",
|
| 287 |
+
"size.status.gone": "文件已不存在 {{n}} 行",
|
| 288 |
+
"size.status.needs_auth": "需要登录 {{n}} 行",
|
| 289 |
+
"size.status.no_length": "服务器未给出大小 {{n}} 行",
|
| 290 |
+
"size.status.error": "其他错误 {{n}} 行",
|
| 291 |
"size.estimate": "没有大小的 {{rows}} 行的估计值:{{size}}(95% 范围 {{low}} 至 {{high}}{{summed}};依据读过的 {{readings}} 行)",
|
| 292 |
"size.estimatePart": "没有大小的 {{all}} 行中 {{rows}} 行的估计值:{{size}}(95% 范围 {{low}} 至 {{high}}{{summed}};依据读过的 {{readings}} 行;其余 {{rest}} 行读过的太少,没有估计值)",
|
| 293 |
"size.estimateNone": "{{rows}} 行没有大小,读过的太少,没有估计值",
|
| 294 |
+
"size.summed": ",为各来源范围之和,比联合区间更宽",
|
| 295 |
"size.none": "本次发布尚未发布大小数字",
|
| 296 |
"size.recordedShort": "记录 {{size}}",
|
| 297 |
"size.distinctShort": "(去重后 {{size}})",
|
| 298 |
"size.how": "大小如何测量",
|
| 299 |
+
"size.checkSentence": "大小核对:从上游重新读取了 {{n}} 行,将其大小和哈希与索引记录的比对;其中 {{share}} 一致,即文件仍是索引所描述的那个文件。",
|
| 300 |
+
"size.checkNoneSentence": "大小核对:还没有任何行从上游重新读取并与索引记录的大小和哈希比对。",
|
| 301 |
+
"size.checkNotComparedSentence": "大小核对:从上游重新读取了 {{read}} 行,但都没能与索引记录的大小和哈希比对({{why}})。",
|
| 302 |
"size.unsizedEst": "另有约 {{size}} 没有记录大小(估)",
|
| 303 |
"size.unsizedNone": "{{rows}} 行没有记录大小",
|
| 304 |
|
|
|
|
| 309 |
"detail.countsByFirst": "{{what}};前几行由 {{publishers}} 发布。",
|
| 310 |
"detail.oneUnit": "一个单元",
|
| 311 |
"detail.files": "文件",
|
| 312 |
+
"detail.from": "出自",
|
| 313 |
"detail.publishers_other": "{{n}} 个发布方;主要网站",
|
| 314 |
"detail.notRecorded": "未记录",
|
| 315 |
"detail.alone": "可独立使用",
|
|
|
|
| 326 |
"detail.anyEditor": "任意编辑器",
|
| 327 |
"detail.licence": "许可证",
|
| 328 |
"detail.training": "训练",
|
| 329 |
+
"detail.shipping": "再分发",
|
| 330 |
"detail.shareOfSet": "(占本材料集的文件;{{licences}})",
|
| 331 |
"detail.shareOfSetOnly": "(占本材料集的文件)",
|
| 332 |
"detail.shareOfCorpus": "(占本语料库的行;{{licences}})",
|
|
|
|
| 341 |
"detail.size": "大小",
|
| 342 |
"detail.median": "中位数 {{size}}",
|
| 343 |
"detail.stripAria": "中位数 {{median}},99% 小于 {{p99}}",
|
| 344 |
+
"detail.fetch": "取回核对",
|
| 345 |
+
"fetch.all": "读过的 {{rows}} 个抽样行全部取到,大小和哈希与记录一致(读取于 {{date}})。",
|
| 346 |
+
"fetch.some": "读过的 {{rows}} 个抽样行中有 {{fetched}} 个({{share}})取到,大小和哈希与记录一致(读取于 {{date}});其余没有。",
|
| 347 |
+
"fetch.none": "读过的 {{rows}} 个抽样行没有一个取到大小和哈希与记录一致的文件(读取于 {{date}}):上游现在不提供这些文件。",
|
| 348 |
+
"fetch.notRead": "还没有读过抽样行。",
|
| 349 |
"detail.example": "示例",
|
| 350 |
"detail.take": "获取",
|
| 351 |
"detail.more": "批次与前几行",
|
|
|
|
| 369 |
"sql.headSets": "Agentic Env 索引 @ {{sha}}:{{sets}}({{units}})",
|
| 370 |
"sql.headCorpora": "Agentic Env 索引 @ {{sha}}:{{corpora}}({{rows}})",
|
| 371 |
"sql.all": "无筛选",
|
| 372 |
+
"sql.noProfile": "按发布原样读取各行:这条查询不应用 tools/fetch.py 默认配置的任何规则(不去重;基准副本、没有与基准测试清单比对过的行、不允许再分发的许可证、非英文、个人数据和超过 20 MB 的文件都保留);命令会应用这些规则。",
|
| 373 |
"sql.chip": "{{k}}:{{v}}",
|
| 374 |
"csv.progress_other": "{{done}}/{{n}} 个文件",
|
| 375 |
"csv.button": "CSV · {{rows}}",
|
|
|
|
| 381 |
"keep.again": "按文件再取一次",
|
| 382 |
"pick.add": "挑选",
|
| 383 |
"pick.picked": "已挑选",
|
| 384 |
+
"pick.word_other": "个来源已挑选",
|
| 385 |
"pick.clear": "清除挑选",
|
| 386 |
+
"pick.note": "这条命令只取已挑选的来源(两层都可挑选),不受筛选影响;清除挑选后改取筛选所选的内容。",
|
| 387 |
+
"attr.title_other": "署名:{{dataset}} 的 {{n}} 个来源",
|
| 388 |
"attr.commit": "数据集提交 {{sha}}:{{url}}",
|
| 389 |
+
"attr.source": "来源",
|
| 390 |
+
"attr.licence": "许可证",
|
| 391 |
+
"attr.licenceUrl": "许可证链接",
|
| 392 |
+
"attr.files": "按发布方、许可证和许可证链接分的文件(各文件自己的;文件没有写明时取其来源的)",
|
| 393 |
+
"attr.fileLine_other": "{{publisher}}:{{licence}},{{url}}({{n}} 个文件)",
|
| 394 |
+
"attr.noPublisher": "未注明发布方",
|
| 395 |
"attr.attribution": "署名",
|
| 396 |
"attr.notStated": "未注明",
|
| 397 |
"tree.title": "写入内容",
|
| 398 |
"tree.units": "单元、README、LICENSE",
|
| 399 |
"tree.answers": "答案,单独存放",
|
| 400 |
"tree.manifest": "每个文件一行,已校验 sha256",
|
| 401 |
+
"tree.sizes": "每个来源一行",
|
| 402 |
"tree.domainNote": ":例如 D3_data 或 D3",
|
| 403 |
"tree.leftOut": "默认排除",
|
| 404 |
"profile.duplicates": "重复内容",
|
| 405 |
"profile.benchmark": "基准副本",
|
| 406 |
+
"profile.licence": "许可证不允许再分发",
|
| 407 |
"profile.english": "非英文",
|
| 408 |
"profile.personal": "个人数据",
|
| 409 |
"profile.large": "超过 20 MB",
|
|
|
|
| 412 |
"video.title": "使用示例视频",
|
| 413 |
"video.text": "一段使用示例视频有中文版和英文版,各带字幕,演示了如何从本页使用这份索引。这里按页面语言播放;两版视频和字幕文件都在数据集的 media/ 下。",
|
| 414 |
"video.subtitles": "中文",
|
| 415 |
+
"manual.title": "使用手册",
|
| 416 |
+
"manual.back": "返回页面",
|
| 417 |
+
"manual.loading": "正在加载手册…",
|
| 418 |
+
"manual.missing": "本页还没有手册:{{error}}",
|
| 419 |
|
| 420 |
"history.title": "每次发布新增了什么",
|
| 421 |
"history.start": "起点",
|
manual.js
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
// #1617: the user manual (manual/manual.en.md and manual/manual.zh.md of this folder) in the page's language, as the
|
| 2 |
+
// view #manual shows it (app.js showView). Its numbers and command outputs were filled at deploy (03_data_line/steps/portal.py).
|
| 3 |
+
// An image names its shot's output in manual/shots.yml and is read from manual/shots/, where the screenshots taken on
|
| 4 |
+
// this page after each deploy are (03_data_line/steps/shots.py); a shot with no image there shows its name in its place.
|
| 5 |
+
// Standard tool: Markdown rendered to HTML by markdown-it (CommonMark).
|
| 6 |
+
import MarkdownIt from 'https://cdn.jsdelivr.net/npm/markdown-it@15.0.3/+esm';
|
| 7 |
+
import { clear, el } from './charts.js';
|
| 8 |
+
import { lang, onLanguage, t } from './i18n.js';
|
| 9 |
+
|
| 10 |
+
const SHOTS = 'manual/shots/';
|
| 11 |
+
const md = new MarkdownIt('commonmark');
|
| 12 |
+
const image = md.renderer.rules.image;
|
| 13 |
+
md.renderer.rules.image = (tokens, i, options, env, self) => {
|
| 14 |
+
const shot = tokens[i].attrGet('src');
|
| 15 |
+
tokens[i].attrSet('src', SHOTS + shot);
|
| 16 |
+
tokens[i].attrSet('data-shot', shot);
|
| 17 |
+
return image(tokens, i, options, env, self);
|
| 18 |
+
};
|
| 19 |
+
|
| 20 |
+
// per language, the text once read
|
| 21 |
+
const texts = new Map();
|
| 22 |
+
// the view's article, once drawn: the language switch draws it again while it is shown
|
| 23 |
+
let target = null;
|
| 24 |
+
|
| 25 |
+
function read(l) {
|
| 26 |
+
if (!texts.has(l)) {
|
| 27 |
+
const p = fetch(new URL(`manual/manual.${l}.md`, document.baseURI)).then((r) => {
|
| 28 |
+
if (!r.ok) throw new Error(`manual/manual.${l}.md: HTTP ${r.status}`);
|
| 29 |
+
return r.text();
|
| 30 |
+
});
|
| 31 |
+
p.catch(() => texts.delete(l));
|
| 32 |
+
texts.set(l, p);
|
| 33 |
+
}
|
| 34 |
+
return texts.get(l);
|
| 35 |
+
}
|
| 36 |
+
|
| 37 |
+
/** The manual of the page's language into node (the view's article). */
|
| 38 |
+
export async function drawManual(node) {
|
| 39 |
+
target = node;
|
| 40 |
+
const l = lang();
|
| 41 |
+
if (!node.querySelector('.manual-text')) clear(node).append(el('p', { class: 'small' }, t('manual.loading')));
|
| 42 |
+
let text;
|
| 43 |
+
try {
|
| 44 |
+
text = await read(l);
|
| 45 |
+
} catch (e) {
|
| 46 |
+
clear(node).append(el('p', { class: 'small problem-text' }, t('manual.missing', { error: e.message })));
|
| 47 |
+
return;
|
| 48 |
+
}
|
| 49 |
+
if (l !== lang()) return;
|
| 50 |
+
const body = el('div', { class: 'manual-text', lang: l === 'zh' ? 'zh-CN' : 'en' });
|
| 51 |
+
// the manual is this folder's own Markdown, rendered by markdown-it; nothing read from the dataset goes in
|
| 52 |
+
body.innerHTML = md.render(text);
|
| 53 |
+
for (const img of body.querySelectorAll('img[data-shot]')) {
|
| 54 |
+
img.addEventListener('error', () => img.replaceWith(el('span', { class: 'shot-missing' }, img.dataset.shot)), { once: true });
|
| 55 |
+
}
|
| 56 |
+
clear(node).append(body);
|
| 57 |
+
}
|
| 58 |
+
|
| 59 |
+
onLanguage(() => { if (target && !target.closest('[hidden]')) drawManual(target); });
|
manual/manual.en.md
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# User manual
|
| 2 |
+
|
| 3 |
+
This manual is for people who build data from the public index [Battam/agentic-env-index](https://huggingface.co/datasets/Battam/agentic-env-index).
|
| 4 |
+
This is a placeholder;
|
| 5 |
+
its texts come later, in the four parts of [Diátaxis](https://diataxis.fr/): a tutorial, how-to guides, a reference and an explanation.
|
| 6 |
+
The numbers below and the output of the command were filled in when this page was deployed, from the index commit `f4b175ed19709225382f0ae5a13fd4c18530b94b`.
|
| 7 |
+
|
| 8 |
+
## Tutorial
|
| 9 |
+
|
| 10 |
+
The index holds 1,799 material sets in 13 collections, with 4,183 units, 787 of them with an answer file.
|
| 11 |
+
Beside them it holds 133 corpora with 1,227,714,425 rows.
|
| 12 |
+
|
| 13 |
+

|
| 14 |
+
|
| 15 |
+
## How-to guides
|
| 16 |
+
|
| 17 |
+
The index's command `tools/fetch.py` takes a selection of the index into a folder;
|
| 18 |
+
these are its options:
|
| 19 |
+
|
| 20 |
+
```console
|
| 21 |
+
$ uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/f4b175ed19709225382f0ae5a13fd4c18530b94b/tools/fetch.py --help
|
| 22 |
+
usage: fetchHygMeO.py [-h] [--repo REPO] [--data DATA] [--revision REVISION]
|
| 23 |
+
[--tag TAG] [--domain DOMAIN] [--form FORM]
|
| 24 |
+
[--kind KIND] [--ext EXT] [--source SOURCE]
|
| 25 |
+
[--selection FILE] [--tier TIER] [--min-size MIN_SIZE]
|
| 26 |
+
[--max-size MAX_SIZE]
|
| 27 |
+
[--max-size-filter MAX_SIZE_FILTER] [--where WHERE]
|
| 28 |
+
[--limit LIMIT] [--sample SAMPLE] [--all]
|
| 29 |
+
[--keep-duplicates] [--allow-benchmark]
|
| 30 |
+
[--allow-not-compared] [--any-license] [--any-language]
|
| 31 |
+
[--allow-personal-data] [--any-size] [--out OUT]
|
| 32 |
+
[--dry-run] [--local-root LOCAL_ROOT]
|
| 33 |
+
[--staging STAGING] [--print-sql]
|
| 34 |
+
[--memory-limit MEMORY_LIMIT] [--threads THREADS]
|
| 35 |
+
[--temp-dir TEMP_DIR] [--decompress] [--check-content]
|
| 36 |
+
[--proxy PROXY] [--contact CONTACT] [--quiet] [--json]
|
| 37 |
+
[--timeout TIMEOUT]
|
| 38 |
+
|
| 39 |
+
Select rows of the Agentic Env Index, download the files they point to and check them against their rows.
|
| 40 |
+
|
| 41 |
+
options:
|
| 42 |
+
-h, --help show this help message and exit
|
| 43 |
+
--repo REPO dataset repository (default Battam/agentic-env-index)
|
| 44 |
+
--data DATA read the tables from this local copy or URL prefix instead of the repository
|
| 45 |
+
--revision REVISION read the repository at this commit (40 hex characters) or tag (pinned to the commit the
|
| 46 |
+
tag points to); default: the release this script was published in (release-2026-11-05.3),
|
| 47 |
+
else the commit the main branch points to when the script starts.
|
| 48 |
+
Record it to repeat a selection later. With a local --data copy it only records the
|
| 49 |
+
commit that copy holds (not checked)
|
| 50 |
+
--tag TAG collection tag campaign:<name> (repeatable; see meta/tags.parquet)
|
| 51 |
+
--domain DOMAIN domain code from meta/catalog.parquet (repeatable, any of them): D1 to D10 or a leaf
|
| 52 |
+
such as D1.6; a top-level code also matches its leaves (D3 matches D3.5), a leaf only
|
| 53 |
+
itself; multi and unclassified match only themselves. Codes: catalog/domains.yaml of
|
| 54 |
+
the pipeline repository, or the README of the index
|
| 55 |
+
--form FORM form id from meta/catalog.parquet, such as test-suites (repeatable, any of them);
|
| 56 |
+
with --domain a source must match both
|
| 57 |
+
--kind KIND layer from the column kind of meta/catalog.parquet: material_set or corpus (repeatable,
|
| 58 |
+
any of them); with --domain or --form a source must match those too
|
| 59 |
+
--ext EXT file extension without the dot (repeatable, any of them): takes the units with a unit
|
| 60 |
+
file of that extension, each with its answers and its source's companion files
|
| 61 |
+
--source SOURCE source name (repeatable)
|
| 62 |
+
--selection FILE a selection file saved by the data page (JSON: sources, revision, filters): its sources
|
| 63 |
+
are read as --source and its revision as --revision
|
| 64 |
+
--tier TIER training license tiers, comma separated (default: all tiers)
|
| 65 |
+
--min-size MIN_SIZE smallest size in bytes
|
| 66 |
+
--max-size MAX_SIZE size cap in bytes used when fetching, for every row (replaces the overall cap); it
|
| 67 |
+
does not remove rows from the selection
|
| 68 |
+
--max-size-filter MAX_SIZE_FILTER
|
| 69 |
+
leave out the rows whose published size is larger than this many bytes (rows of
|
| 70 |
+
unknown size stay; --max-size or the overall cap still bound what is downloaded)
|
| 71 |
+
--where WHERE extra SQL condition on the row table p (and the batch table b)
|
| 72 |
+
--limit LIMIT at most this many units over the whole selection (a fixed order)
|
| 73 |
+
--sample SAMPLE this many units per selected source, the same ones every run (a fixed order); with
|
| 74 |
+
their answers and the source's companion files
|
| 75 |
+
--all turn the default profile off: apply only the conditions given here
|
| 76 |
+
--keep-duplicates profile: keep duplicate copies
|
| 77 |
+
--allow-benchmark profile: keep rows that matched a benchmark (copies of benchmark data, collections
|
| 78 |
+
awaiting a ruling) and rows not compared with the benchmark list yet
|
| 79 |
+
--allow-not-compared profile: keep rows whose data file was not compared with the benchmark list yet (rows
|
| 80 |
+
that matched a benchmark stay out)
|
| 81 |
+
--any-license, --any-licence
|
| 82 |
+
profile: keep rows whose license does not let the files be shipped
|
| 83 |
+
--any-language profile: keep rows recorded in a language other than English
|
| 84 |
+
--allow-personal-data
|
| 85 |
+
profile: keep rows of sources or rows marked as carrying personal data
|
| 86 |
+
--any-size profile: keep rows above 20,000,000 bytes (the overall cap still bounds the
|
| 87 |
+
download)
|
| 88 |
+
--out OUT directory for the files and _meta/ (default ./fetched)
|
| 89 |
+
--dry-run print the counts and one line per source of what would be fetched (units, answers,
|
| 90 |
+
companion files, bytes) and a total; write _meta/fetch_manifest.jsonl with one line per
|
| 91 |
+
file of the selection (status selected, removed or not_taken), download nothing
|
| 92 |
+
--local-root LOCAL_ROOT
|
| 93 |
+
reuse a file under this directory whose size and hashes match a row instead of downloading
|
| 94 |
+
it (repeatable; default: the ARTPOOL_LOCAL_ROOTS variable, directories separated by
|
| 95 |
+
':'); each manifest line records obtained: reused, staged or fetched
|
| 96 |
+
--staging STAGING directory where an outside fetch placed files; checked like a local root, recorded as
|
| 97 |
+
staged (default: the ARTPOOL_STAGING variable)
|
| 98 |
+
--print-sql print the query and exit
|
| 99 |
+
--memory-limit MEMORY_LIMIT
|
| 100 |
+
DuckDB memory limit, for example 8GB (default: the smaller of 4GB and a quarter of the
|
| 101 |
+
memory available, the cgroup limit when lower); what does not fit is written to the
|
| 102 |
+
spill directory
|
| 103 |
+
--threads THREADS DuckDB threads (default: one per GB of the memory limit, at most the number of CPUs)
|
| 104 |
+
--temp-dir TEMP_DIR DuckDB spill directory (default: .fetch_spill under --out, removed at the end)
|
| 105 |
+
--decompress after a verified download, decompress .gz, .bz2 and .zst files (within the row's cap)
|
| 106 |
+
and delete the compressed file; .zst needs pip install zstandard
|
| 107 |
+
--check-content after a verified download, flag scanned PDFs, error pages saved as files and extensions
|
| 108 |
+
that do not match the content (manifest field content_check); needs the file command
|
| 109 |
+
and poppler-utils (pdffonts, pdftotext), a missing tool skips its check
|
| 110 |
+
--proxy PROXY proxy for every request: http://host:port, or socks5h://host:port
|
| 111 |
+
(a proxy login goes before the host as user:password followed by an at sign).
|
| 112 |
+
(DuckDB and the byte-range reads of archive members cannot use a socks proxy,
|
| 113 |
+
see --data).
|
| 114 |
+
Default: HTTPS_PROXY, HTTP_PROXY or ALL_PROXY from the environment
|
| 115 |
+
--contact CONTACT your e-mail or URL, sent in the User-Agent so a site can reach you instead of blocking
|
| 116 |
+
--quiet no progress lines and no per-file or per-source lines on stderr
|
| 117 |
+
--json end with the summary as one JSON object on stdout (the default when stdout is not a
|
| 118 |
+
terminal); on a terminal it is otherwise a few lines in words
|
| 119 |
+
--timeout TIMEOUT seconds a download may wait to connect, or for its next bytes, before that address
|
| 120 |
+
fails (default 120); it does not bound a whole download. The reads of the index tables
|
| 121 |
+
have their own: 120 seconds a request, retried for up to 510 seconds
|
| 122 |
+
|
| 123 |
+
environment:
|
| 124 |
+
HF_TOKEN a Hugging Face access token; the index tables are read with it (a DuckDB huggingface
|
| 125 |
+
secret), so Hugging Face allows more requests per 5 minutes than to an anonymous address,
|
| 126 |
+
which runs into HTTP 429 sooner (each read then waits and retries). It overrides a token
|
| 127 |
+
stored on the machine. Without it the run says once that it reads anonymously. To set it,
|
| 128 |
+
create a read token at https://huggingface.co/settings/tokens, then
|
| 129 |
+
export HF_TOKEN=hf_... (PowerShell: $env:HF_TOKEN = 'hf_...')
|
| 130 |
+
```
|
| 131 |
+
|
| 132 |
+
## Reference
|
| 133 |
+
|
| 134 |
+
The first screen on a phone:
|
| 135 |
+
|
| 136 |
+

|
| 137 |
+
|
| 138 |
+
## Explanation
|
| 139 |
+
|
| 140 |
+
The whole page:
|
| 141 |
+
|
| 142 |
+

|
manual/manual.zh.md
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 使用手册
|
| 2 |
+
|
| 3 |
+
本手册写给从公开索引 [Battam/agentic-env-index](https://huggingface.co/datasets/Battam/agentic-env-index) 构建数据的人。
|
| 4 |
+
这是占位版本;
|
| 5 |
+
正文稍后补上,按 [Diátaxis](https://diataxis.fr/) 分为四部分:教程、操作指南、参考和解释。
|
| 6 |
+
下面的数字和命令输出是本页部署时填入的,取自索引提交 `f4b175ed19709225382f0ae5a13fd4c18530b94b`。
|
| 7 |
+
|
| 8 |
+
## 教程
|
| 9 |
+
|
| 10 |
+
索引有 13 个合集中的 1,799 个材料集,共 4,183 个单元,其中 787 个带答案文件。
|
| 11 |
+
此外还有 133 个语料库,共 1,227,714,425 行。
|
| 12 |
+
|
| 13 |
+

|
| 14 |
+
|
| 15 |
+
## 操作指南
|
| 16 |
+
|
| 17 |
+
索引的命令 `tools/fetch.py` 把索引中的一份选择取到一个文件夹;
|
| 18 |
+
它的选项如下:
|
| 19 |
+
|
| 20 |
+
```console
|
| 21 |
+
$ uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/f4b175ed19709225382f0ae5a13fd4c18530b94b/tools/fetch.py --help
|
| 22 |
+
usage: fetchtXYHj5.py [-h] [--repo REPO] [--data DATA] [--revision REVISION]
|
| 23 |
+
[--tag TAG] [--domain DOMAIN] [--form FORM]
|
| 24 |
+
[--kind KIND] [--ext EXT] [--source SOURCE]
|
| 25 |
+
[--selection FILE] [--tier TIER] [--min-size MIN_SIZE]
|
| 26 |
+
[--max-size MAX_SIZE]
|
| 27 |
+
[--max-size-filter MAX_SIZE_FILTER] [--where WHERE]
|
| 28 |
+
[--limit LIMIT] [--sample SAMPLE] [--all]
|
| 29 |
+
[--keep-duplicates] [--allow-benchmark]
|
| 30 |
+
[--allow-not-compared] [--any-license] [--any-language]
|
| 31 |
+
[--allow-personal-data] [--any-size] [--out OUT]
|
| 32 |
+
[--dry-run] [--local-root LOCAL_ROOT]
|
| 33 |
+
[--staging STAGING] [--print-sql]
|
| 34 |
+
[--memory-limit MEMORY_LIMIT] [--threads THREADS]
|
| 35 |
+
[--temp-dir TEMP_DIR] [--decompress] [--check-content]
|
| 36 |
+
[--proxy PROXY] [--contact CONTACT] [--quiet] [--json]
|
| 37 |
+
[--timeout TIMEOUT]
|
| 38 |
+
|
| 39 |
+
Select rows of the Agentic Env Index, download the files they point to and check them against their rows.
|
| 40 |
+
|
| 41 |
+
options:
|
| 42 |
+
-h, --help show this help message and exit
|
| 43 |
+
--repo REPO dataset repository (default Battam/agentic-env-index)
|
| 44 |
+
--data DATA read the tables from this local copy or URL prefix instead of the repository
|
| 45 |
+
--revision REVISION read the repository at this commit (40 hex characters) or tag (pinned to the commit the
|
| 46 |
+
tag points to); default: the release this script was published in (release-2026-11-05.3),
|
| 47 |
+
else the commit the main branch points to when the script starts.
|
| 48 |
+
Record it to repeat a selection later. With a local --data copy it only records the
|
| 49 |
+
commit that copy holds (not checked)
|
| 50 |
+
--tag TAG collection tag campaign:<name> (repeatable; see meta/tags.parquet)
|
| 51 |
+
--domain DOMAIN domain code from meta/catalog.parquet (repeatable, any of them): D1 to D10 or a leaf
|
| 52 |
+
such as D1.6; a top-level code also matches its leaves (D3 matches D3.5), a leaf only
|
| 53 |
+
itself; multi and unclassified match only themselves. Codes: catalog/domains.yaml of
|
| 54 |
+
the pipeline repository, or the README of the index
|
| 55 |
+
--form FORM form id from meta/catalog.parquet, such as test-suites (repeatable, any of them);
|
| 56 |
+
with --domain a source must match both
|
| 57 |
+
--kind KIND layer from the column kind of meta/catalog.parquet: material_set or corpus (repeatable,
|
| 58 |
+
any of them); with --domain or --form a source must match those too
|
| 59 |
+
--ext EXT file extension without the dot (repeatable, any of them): takes the units with a unit
|
| 60 |
+
file of that extension, each with its answers and its source's companion files
|
| 61 |
+
--source SOURCE source name (repeatable)
|
| 62 |
+
--selection FILE a selection file saved by the data page (JSON: sources, revision, filters): its sources
|
| 63 |
+
are read as --source and its revision as --revision
|
| 64 |
+
--tier TIER training license tiers, comma separated (default: all tiers)
|
| 65 |
+
--min-size MIN_SIZE smallest size in bytes
|
| 66 |
+
--max-size MAX_SIZE size cap in bytes used when fetching, for every row (replaces the overall cap); it
|
| 67 |
+
does not remove rows from the selection
|
| 68 |
+
--max-size-filter MAX_SIZE_FILTER
|
| 69 |
+
leave out the rows whose published size is larger than this many bytes (rows of
|
| 70 |
+
unknown size stay; --max-size or the overall cap still bound what is downloaded)
|
| 71 |
+
--where WHERE extra SQL condition on the row table p (and the batch table b)
|
| 72 |
+
--limit LIMIT at most this many units over the whole selection (a fixed order)
|
| 73 |
+
--sample SAMPLE this many units per selected source, the same ones every run (a fixed order); with
|
| 74 |
+
their answers and the source's companion files
|
| 75 |
+
--all turn the default profile off: apply only the conditions given here
|
| 76 |
+
--keep-duplicates profile: keep duplicate copies
|
| 77 |
+
--allow-benchmark profile: keep rows that matched a benchmark (copies of benchmark data, collections
|
| 78 |
+
awaiting a ruling) and rows not compared with the benchmark list yet
|
| 79 |
+
--allow-not-compared profile: keep rows whose data file was not compared with the benchmark list yet (rows
|
| 80 |
+
that matched a benchmark stay out)
|
| 81 |
+
--any-license, --any-licence
|
| 82 |
+
profile: keep rows whose license does not let the files be shipped
|
| 83 |
+
--any-language profile: keep rows recorded in a language other than English
|
| 84 |
+
--allow-personal-data
|
| 85 |
+
profile: keep rows of sources or rows marked as carrying personal data
|
| 86 |
+
--any-size profile: keep rows above 20,000,000 bytes (the overall cap still bounds the
|
| 87 |
+
download)
|
| 88 |
+
--out OUT directory for the files and _meta/ (default ./fetched)
|
| 89 |
+
--dry-run print the counts and one line per source of what would be fetched (units, answers,
|
| 90 |
+
companion files, bytes) and a total; write _meta/fetch_manifest.jsonl with one line per
|
| 91 |
+
file of the selection (status selected, removed or not_taken), download nothing
|
| 92 |
+
--local-root LOCAL_ROOT
|
| 93 |
+
reuse a file under this directory whose size and hashes match a row instead of downloading
|
| 94 |
+
it (repeatable; default: the ARTPOOL_LOCAL_ROOTS variable, directories separated by
|
| 95 |
+
':'); each manifest line records obtained: reused, staged or fetched
|
| 96 |
+
--staging STAGING directory where an outside fetch placed files; checked like a local root, recorded as
|
| 97 |
+
staged (default: the ARTPOOL_STAGING variable)
|
| 98 |
+
--print-sql print the query and exit
|
| 99 |
+
--memory-limit MEMORY_LIMIT
|
| 100 |
+
DuckDB memory limit, for example 8GB (default: the smaller of 4GB and a quarter of the
|
| 101 |
+
memory available, the cgroup limit when lower); what does not fit is written to the
|
| 102 |
+
spill directory
|
| 103 |
+
--threads THREADS DuckDB threads (default: one per GB of the memory limit, at most the number of CPUs)
|
| 104 |
+
--temp-dir TEMP_DIR DuckDB spill directory (default: .fetch_spill under --out, removed at the end)
|
| 105 |
+
--decompress after a verified download, decompress .gz, .bz2 and .zst files (within the row's cap)
|
| 106 |
+
and delete the compressed file; .zst needs pip install zstandard
|
| 107 |
+
--check-content after a verified download, flag scanned PDFs, error pages saved as files and extensions
|
| 108 |
+
that do not match the content (manifest field content_check); needs the file command
|
| 109 |
+
and poppler-utils (pdffonts, pdftotext), a missing tool skips its check
|
| 110 |
+
--proxy PROXY proxy for every request: http://host:port, or socks5h://host:port
|
| 111 |
+
(a proxy login goes before the host as user:password followed by an at sign).
|
| 112 |
+
(DuckDB and the byte-range reads of archive members cannot use a socks proxy,
|
| 113 |
+
see --data).
|
| 114 |
+
Default: HTTPS_PROXY, HTTP_PROXY or ALL_PROXY from the environment
|
| 115 |
+
--contact CONTACT your e-mail or URL, sent in the User-Agent so a site can reach you instead of blocking
|
| 116 |
+
--quiet no progress lines and no per-file or per-source lines on stderr
|
| 117 |
+
--json end with the summary as one JSON object on stdout (the default when stdout is not a
|
| 118 |
+
terminal); on a terminal it is otherwise a few lines in words
|
| 119 |
+
--timeout TIMEOUT seconds a download may wait to connect, or for its next bytes, before that address
|
| 120 |
+
fails (default 120); it does not bound a whole download. The reads of the index tables
|
| 121 |
+
have their own: 120 seconds a request, retried for up to 510 seconds
|
| 122 |
+
|
| 123 |
+
environment:
|
| 124 |
+
HF_TOKEN a Hugging Face access token; the index tables are read with it (a DuckDB huggingface
|
| 125 |
+
secret), so Hugging Face allows more requests per 5 minutes than to an anonymous address,
|
| 126 |
+
which runs into HTTP 429 sooner (each read then waits and retries). It overrides a token
|
| 127 |
+
stored on the machine. Without it the run says once that it reads anonymously. To set it,
|
| 128 |
+
create a read token at https://huggingface.co/settings/tokens, then
|
| 129 |
+
export HF_TOKEN=hf_... (PowerShell: $env:HF_TOKEN = 'hf_...')
|
| 130 |
+
```
|
| 131 |
+
|
| 132 |
+
## 参考
|
| 133 |
+
|
| 134 |
+
手机上的首屏:
|
| 135 |
+
|
| 136 |
+

|
| 137 |
+
|
| 138 |
+
## 解释
|
| 139 |
+
|
| 140 |
+
整页:
|
| 141 |
+
|
| 142 |
+

|
manual/shots.yml
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# #1617: the screenshots of the user manual, in shot-scraper's multi format
|
| 2 |
+
# (https://shot-scraper.datasette.io/en/stable/multi.html). 03_data_line/steps/shots.py takes every shot on the live
|
| 3 |
+
# Space after each deploy, each url resolved against the live page, and writes each image to the Space as
|
| 4 |
+
# manual/shots/<output>; the manual's Markdown (manual.en.md, manual.zh.md) references an image by its output.
|
| 5 |
+
# url: relative to the live page; ?lng=en or ?lng=zh sets the page's language
|
| 6 |
+
# width: 1920 or 390, the two widths the page is checked at
|
| 7 |
+
# selector: the element to shoot; without one, the whole page
|
| 8 |
+
# javascript: run once the page has loaded, before the wait; before a full-page shot it shows the scroll-reveal
|
| 9 |
+
# elements (style.css starts .js-reveal .reveal at opacity 0 until it scrolls into view, and a full-page shot
|
| 10 |
+
# does not scroll)
|
| 11 |
+
# wait_for: a JavaScript expression, true once the shot's element is drawn; the page keeps loading modules from the
|
| 12 |
+
# CDN long after it is usable, so no shot waits for the network to be idle
|
| 13 |
+
- output: first-screen-en-1920.png
|
| 14 |
+
url: "?lng=en"
|
| 15 |
+
width: 1920
|
| 16 |
+
selector: "#top"
|
| 17 |
+
wait_for: "document.body.classList.contains('ready') && !document.querySelector('#top .sk')"
|
| 18 |
+
- output: first-screen-en-390.png
|
| 19 |
+
url: "?lng=en"
|
| 20 |
+
width: 390
|
| 21 |
+
selector: "#top"
|
| 22 |
+
wait_for: "document.body.classList.contains('ready') && !document.querySelector('#top .sk')"
|
| 23 |
+
- output: page-en-1920.png
|
| 24 |
+
url: "?lng=en"
|
| 25 |
+
width: 1920
|
| 26 |
+
javascript: "document.querySelectorAll('.reveal').forEach((e) => e.classList.add('in'))"
|
| 27 |
+
wait_for: "document.querySelector('#sources-body table') !== null"
|
| 28 |
+
- output: first-screen-zh-1920.png
|
| 29 |
+
url: "?lng=zh"
|
| 30 |
+
width: 1920
|
| 31 |
+
selector: "#top"
|
| 32 |
+
wait_for: "document.body.classList.contains('ready') && !document.querySelector('#top .sk')"
|
| 33 |
+
- output: first-screen-zh-390.png
|
| 34 |
+
url: "?lng=zh"
|
| 35 |
+
width: 390
|
| 36 |
+
selector: "#top"
|
| 37 |
+
wait_for: "document.body.classList.contains('ready') && !document.querySelector('#top .sk')"
|
| 38 |
+
- output: page-zh-1920.png
|
| 39 |
+
url: "?lng=zh"
|
| 40 |
+
width: 1920
|
| 41 |
+
javascript: "document.querySelectorAll('.reveal').forEach((e) => e.classList.add('in'))"
|
| 42 |
+
wait_for: "document.querySelector('#sources-body table') !== null"
|
portal.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
{
|
| 2 |
-
"commit": "
|
| 3 |
-
"digest": "
|
| 4 |
-
"files":
|
| 5 |
"format": "portal-deploy/1",
|
| 6 |
"index": "f4b175ed19709225382f0ae5a13fd4c18530b94b"
|
| 7 |
}
|
|
|
|
| 1 |
{
|
| 2 |
+
"commit": "ae5043d3f1334f1876bec9b2bd9ea384279deb75",
|
| 3 |
+
"digest": "5defae89560cdd5321dbce69717fc621ee175790ca7b87b2fa198a863ea3b875",
|
| 4 |
+
"files": 18,
|
| 5 |
"format": "portal-deploy/1",
|
| 6 |
"index": "f4b175ed19709225382f0ae5a13fd4c18530b94b"
|
| 7 |
}
|
style.css
CHANGED
|
@@ -410,8 +410,9 @@ button.bar-row:hover, button.bar-row:focus-visible { background: var(--ours-wash
|
|
| 410 |
.fit-part { background: linear-gradient(90deg, var(--ours) 50%, #fff 50%); box-shadow: inset 0 0 0 1.5px var(--ours); }
|
| 411 |
.fit-no { background: var(--problem); }
|
| 412 |
.fit-unknown { background: #fff; box-shadow: inset 0 0 0 1.5px var(--other); }
|
| 413 |
-
/* #1302: the dots and their names over the column share one grid of
|
| 414 |
-
|
|
|
|
| 415 |
.fit-head { display: flex; flex-direction: column; align-items: center; gap: 2px; padding: 0 0 var(--s1); font-size: 10px; font-weight: 500; line-height: 1.1; color: var(--ink-2); text-align: center; white-space: nowrap; }
|
| 416 |
.src-table { border-collapse: collapse; width: 100%; font-size: 15px; }
|
| 417 |
.src-table th { font-size: var(--fs-small); font-weight: 600; color: var(--ink-2); text-align: left; padding: 0; border-bottom: 1px solid var(--line); }
|
|
@@ -584,6 +585,26 @@ details.long[open] > summary { margin-bottom: var(--s1); }
|
|
| 584 |
.tip .k { display: block; color: var(--ink-2); }
|
| 585 |
.tip .k::before { content: ""; display: inline-block; width: 10px; height: 2px; background: var(--ours); vertical-align: 3px; margin-right: 6px; }
|
| 586 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 587 |
/* view transitions: brief, and none without motion */
|
| 588 |
::view-transition-old(root), ::view-transition-new(root) { animation-duration: 180ms; }
|
| 589 |
@media (prefers-reduced-motion: reduce) {
|
|
|
|
| 410 |
.fit-part { background: linear-gradient(90deg, var(--ours) 50%, #fff 50%); box-shadow: inset 0 0 0 1.5px var(--ours); }
|
| 411 |
.fit-no { background: var(--problem); }
|
| 412 |
.fit-unknown { background: #fff; box-shadow: inset 0 0 0 1.5px var(--other); }
|
| 413 |
+
/* #1302: the dots and their names over the column share one grid of a slot per fit (#1577: eight), so each name stands
|
| 414 |
+
above its dot */
|
| 415 |
+
.fit-strip, .fit-heads { display: inline-grid; grid-template-columns: repeat(8, 2.25rem); justify-items: center; align-items: center; }
|
| 416 |
.fit-head { display: flex; flex-direction: column; align-items: center; gap: 2px; padding: 0 0 var(--s1); font-size: 10px; font-weight: 500; line-height: 1.1; color: var(--ink-2); text-align: center; white-space: nowrap; }
|
| 417 |
.src-table { border-collapse: collapse; width: 100%; font-size: 15px; }
|
| 418 |
.src-table th { font-size: var(--fs-small); font-weight: 600; color: var(--ink-2); text-align: left; padding: 0; border-bottom: 1px solid var(--line); }
|
|
|
|
| 585 |
.tip .k { display: block; color: var(--ink-2); }
|
| 586 |
.tip .k::before { content: ""; display: inline-block; width: 10px; height: 2px; background: var(--ours); vertical-align: 3px; margin-right: 6px; }
|
| 587 |
|
| 588 |
+
/* ---- #1617: the user manual, a view of its own (#manual): Markdown rendered by markdown-it in one card ---- */
|
| 589 |
+
.manual-on .progress { visibility: hidden; }
|
| 590 |
+
.manual-view { padding-block: var(--s5) var(--s10); min-height: calc(100vh - var(--nav-h)); }
|
| 591 |
+
.manual-wrap { max-width: calc(960px + 2 * var(--pad)); }
|
| 592 |
+
.manual-back { margin: 0 0 var(--s2); font-size: var(--fs-small); }
|
| 593 |
+
.manual-body { padding: var(--s4) var(--s5); line-height: 1.65; overflow-wrap: anywhere; }
|
| 594 |
+
.manual-body > :first-child, .manual-text > :first-child { margin-top: 0; }
|
| 595 |
+
.manual-body h1 { font-size: var(--fs-h1); line-height: 1.2; letter-spacing: -0.01em; }
|
| 596 |
+
.manual-body h2 { font-size: var(--fs-sub); margin-top: var(--s5); padding-top: var(--s3); border-top: 1px solid var(--line); }
|
| 597 |
+
.manual-body img { display: block; max-width: 100%; height: auto; margin: var(--s2) 0; border: 1px solid var(--line); border-radius: 10px; box-shadow: var(--shadow); }
|
| 598 |
+
.manual-body pre { background: var(--code); color: #dfe8ee; border-radius: 12px; padding: 16px 20px; font-size: 13.5px; line-height: 1.6;
|
| 599 |
+
overflow: auto; max-height: 28rem; }
|
| 600 |
+
.manual-body :not(pre) > code { background: var(--ours-wash); border-radius: 6px; padding: 1px 6px; font-size: 0.9em; }
|
| 601 |
+
.manual-body table { border-collapse: collapse; }
|
| 602 |
+
.manual-body th, .manual-body td { border: 1px solid var(--line); padding: 6px 10px; text-align: left; }
|
| 603 |
+
/* a shot with no image on the Space yet: its name in its place */
|
| 604 |
+
.shot-missing { display: block; margin: var(--s2) 0; padding: var(--s3); border: 1px dashed var(--other); border-radius: 10px;
|
| 605 |
+
font-family: var(--mono); font-size: var(--fs-small); color: var(--ink-muted); text-align: center; }
|
| 606 |
+
@media (max-width: 720px) { .manual-body { padding: var(--s3) var(--s2); } }
|
| 607 |
+
|
| 608 |
/* view transitions: brief, and none without motion */
|
| 609 |
::view-transition-old(root), ::view-transition-new(root) { animation-duration: 180ms; }
|
| 610 |
@media (prefers-reduced-motion: reduce) {
|