data-line commited on
Commit
2ee2aa2
ยท
1 Parent(s): d20fe4d

portal: catalog page of repository commit a015777686bf

Browse files

Files of catalog/portal/ (19), digest 63a115cc1a819d10fc6a572ea53d2b8ee4aa63e824ef957d062826b8892bf2ff.

Files changed (13) hide show
  1. README.md +13 -3
  2. app.js +43 -1
  3. data.js +8 -1
  4. first.json +1 -1
  5. funnel.json +116 -0
  6. index.html +9 -0
  7. llms.txt +1 -1
  8. locales/en.json +12 -0
  9. locales/zh.json +12 -0
  10. manual/manual.en.md +33 -31
  11. manual/manual.zh.md +33 -31
  12. portal.json +4 -3
  13. stages.json +1 -1
README.md CHANGED
@@ -16,7 +16,8 @@ datasets:
16
  A static page into the public pointer index [Battam/agentic-env-index](https://huggingface.co/datasets/Battam/agentic-env-index)
17
  for people who build tasks for a terminal agent, and for their agent: task material (material sets in their
18
  collections) and corpora as two layers, their counts and their sizes by layer and collection on the first screen, one
19
- card per collection, the largest material sets, the most requested domains, each training stage's funnel, the domain ร—
 
20
  form map with the bars by domain and by form for one layer at a time (linked: a click on one filters the others and the
21
  list), the material sets or corpora with their fit and a preview of their rows, and one command per layer to take a
22
  sample.
@@ -161,6 +162,7 @@ The first screen (`first.js`) and the reader of the index (`data.js`) share that
161
  | `meta/projection.json` | at load, when the commit has it | `tag`, the release tag the page's commands name |
162
  | `stages.json` (#2006: beside the page, written by the deploy) | at load, when the deploy wrote one | `index` (the index commit it was computed for; read only when it is the commit the page reads), `counted` (whether that commit has the stage fields), `version` (of the stage table), `stages` (`id`, `name`, `zh`, `status`, `about`, `counted_by`, `counts`; #2066: the funnels' stages, whatever commit it names), `sources` (per source the ids of the stages one of its units passes) |
163
  | `meta/stage_figures.json` (#2066) | at load, when the commit has it | `rules` (the rules every stage's material passes, in their order), per stage of `stages` (`id`, `counts`, each null before the release counted the stage fields) its `funnel` (#2065: per step `after`, null before the rules or the rule of `rules` it is counted after, and `counts`, by the measures the stage is counted by), the funnels |
 
164
  | `meta/translations.parquet` (#2066: the table of Chinese names #2065 writes with the release) | at load, when the commit has it | `en` (an English text the index shows: a collection's name, a source's title or description as the catalog holds them) and `zh` (its Chinese text) |
165
 
166
  `meta/tags.parquet` and `meta/not_included.parquet` are not read.
@@ -249,6 +251,10 @@ A selection by stage takes its sources by `--source`, under the default profile,
249
  Each step is named as the dataset card's funnel tables name it (#2065: "After the benchmark rule", down to "After deduplication: the stage's count").
250
  Until a release has counted its stage fields (no `meta/stage_figures.json`, or its counts empty), the section says so in full, as the filter's note does (`funnel.notCounted`);
251
  a release that counts them without what each rule leaves says that.
 
 
 
 
252
 
253
  ## Languages
254
 
@@ -384,7 +390,8 @@ page's language.
384
  title), the largest material sets as
385
  bars (#2066: `plots.js` `bars`, each named by its title, its id, form and sizes on hover, a click opening it in the list; the rest in one line under the chart and its table, never as a bar; the table with their recorded and deduplicated sizes), the most requested domains (the constant
386
  `MOST_REQUESTED`, ten: one card each, in two rows of five from 1,100 px; those with material by size, then the
387
- wanted ones with a request; `index.html` has ten skeleton cards to match), the training stages' funnels (#2066,
 
388
  `renderStages`), the filters and their URL state (`F`), the
389
  chips of the selection, the sections, the fit of a source, the command, the SQL and the CSV.
390
  A source is named by its title wherever it is read (the largest material sets, the source table, the
@@ -507,7 +514,8 @@ page starts, since the first screen holds one of its charts: the builds vgplot i
507
  `markdown-it@15.0.3` (the manual's Markdown, loaded with `manual.js` the first time the manual is shown).
508
  There is no build step. To look at it locally: `python -m http.server` in this folder (without a
509
  `first.json` the hero shows skeletons until the index is read; `03_data_line/steps/portal.py --write-first first.json`
510
- writes one; without a `stages.json` the filter by stage is off).
 
511
 
512
  ## Deploy
513
 
@@ -524,6 +532,8 @@ a commit whose figures have no counts gets them from its catalog by the same rul
524
  commit has it).
525
  #2066: the page draws the counts of `first.json` at once; the first screen's sizes are its figure by layer and collection, drawn from the index's `meta/size_figures.json` once the page has read it.
526
  #2006: it also adds `stages.json` (format `portal-stages/1`), the stage table of its code tree evaluated over the stage fields of the same index commit (section Query parameters), and records the sha256 of the table in `portal.json`, so a changed table is deployed even when the index has not changed.
 
 
527
 
528
  #1617, #1633, #2060: the manual's texts are Jinja2 templates, the engine of the index's card and docs, and go to the
529
  Space rendered, both with the same values of their markers, computed once per deploy from the index commit it reads.
 
16
  A static page into the public pointer index [Battam/agentic-env-index](https://huggingface.co/datasets/Battam/agentic-env-index)
17
  for people who build tasks for a terminal agent, and for their agent: task material (material sets in their
18
  collections) and corpora as two layers, their counts and their sizes by layer and collection on the first screen, one
19
+ card per collection, the largest material sets, the most requested domains, the funnel from the corpus to the training
20
+ stages, each training stage's funnel, the domain ร—
21
  form map with the bars by domain and by form for one layer at a time (linked: a click on one filters the others and the
22
  list), the material sets or corpora with their fit and a preview of their rows, and one command per layer to take a
23
  sample.
 
162
  | `meta/projection.json` | at load, when the commit has it | `tag`, the release tag the page's commands name |
163
  | `stages.json` (#2006: beside the page, written by the deploy) | at load, when the deploy wrote one | `index` (the index commit it was computed for; read only when it is the commit the page reads), `counted` (whether that commit has the stage fields), `version` (of the stage table), `stages` (`id`, `name`, `zh`, `status`, `about`, `counted_by`, `counts`; #2066: the funnels' stages, whatever commit it names), `sources` (per source the ids of the stages one of its units passes) |
164
  | `meta/stage_figures.json` (#2066) | at load, when the commit has it | `rules` (the rules every stage's material passes, in their order), per stage of `stages` (`id`, `counts`, each null before the release counted the stage fields) its `funnel` (#2065: per step `after`, null before the rules or the rule of `rules` it is counted after, and `counts`, by the measures the stage is counted by), the funnels |
165
+ | `funnel.json` (#2287: beside the page, written by the deploy) | at load, when the deploy wrote one | `release` (the release tag it counts) and `index` (its merge commit), and `funnel`, the funnel's index half as the release counted it (the metadata of the release's asset `release_published`, which the task dashboard reads too): `layers` (`corpus`, its `rows`; `chosen` and `admitted`, their `units`; each with how many sources are not counted yet and `by_domain`, its count under each main domain; `admitted` also `promoted` and `search`) and `stages` (`id`, `name`, `zh`, `status`, `measure`, `before`, `kept` and `by_domain`), the figure from the corpus to the training stages |
166
  | `meta/translations.parquet` (#2066: the table of Chinese names #2065 writes with the release) | at load, when the commit has it | `en` (an English text the index shows: a collection's name, a source's title or description as the catalog holds them) and `zh` (its Chinese text) |
167
 
168
  `meta/tags.parquet` and `meta/not_included.parquet` are not read.
 
251
  Each step is named as the dataset card's funnel tables name it (#2065: "After the benchmark rule", down to "After deduplication: the stage's count").
252
  Until a release has counted its stage fields (no `meta/stage_figures.json`, or its counts empty), the section says so in full, as the filter's note does (`funnel.notCounted`);
253
  a release that counts them without what each rule leaves says that.
254
+ #2287: before the stages' funnels, the funnel's index half is one figure (`app.js` `renderFlow`, `plots.js` `bars`), from `funnel.json` beside the page, whatever commit it names, its title naming the release it counts: the corpora's rows, the units selection files of corpus rows chose for admission (#1909), the units admitted into material sets (on hover, those promoted from the corpus and those from the search) and what each training stage keeps, in its units or rows (on hover, what is in its condition before the rules), one bar each on one log axis, in the color of the layer each counts.
255
+ Its Table view gives the same layers under each main domain: a source counted under the top-level domain of its first domain code.
256
+ These are the numbers the task dashboard joins its task half to (`06_tasks/orchestration/README.md`, "The funnel"), so the units admitted into material sets are one number in both;
257
+ without `funnel.json` the figure says that no release has counted it yet (`flow.none`).
258
 
259
  ## Languages
260
 
 
390
  title), the largest material sets as
391
  bars (#2066: `plots.js` `bars`, each named by its title, its id, form and sizes on hover, a click opening it in the list; the rest in one line under the chart and its table, never as a bar; the table with their recorded and deduplicated sizes), the most requested domains (the constant
392
  `MOST_REQUESTED`, ten: one card each, in two rows of five from 1,100 px; those with material by size, then the
393
+ wanted ones with a request; `index.html` has ten skeleton cards to match), the funnel from the corpus to the training
394
+ stages (#2287, `renderFlow`), the training stages' funnels (#2066,
395
  `renderStages`), the filters and their URL state (`F`), the
396
  chips of the selection, the sections, the fit of a source, the command, the SQL and the CSV.
397
  A source is named by its title wherever it is read (the largest material sets, the source table, the
 
514
  `markdown-it@15.0.3` (the manual's Markdown, loaded with `manual.js` the first time the manual is shown).
515
  There is no build step. To look at it locally: `python -m http.server` in this folder (without a
516
  `first.json` the hero shows skeletons until the index is read; `03_data_line/steps/portal.py --write-first first.json`
517
+ writes one; without a `stages.json` the filter by stage is off; without a `funnel.json` the figure from the corpus to
518
+ the training stages says so).
519
 
520
  ## Deploy
521
 
 
532
  commit has it).
533
  #2066: the page draws the counts of `first.json` at once; the first screen's sizes are its figure by layer and collection, drawn from the index's `meta/size_figures.json` once the page has read it.
534
  #2006: it also adds `stages.json` (format `portal-stages/1`), the stage table of its code tree evaluated over the stage fields of the same index commit (section Query parameters), and records the sha256 of the table in `portal.json`, so a changed table is deployed even when the index has not changed.
535
+ #2287: and `funnel.json` (format `portal-funnel/1`), the funnel's index half of the newest release that merged one, which the portal asset hands the step from that release's `release_published` metadata (`03_data_line/orchestration/definitions.py` `published_funnel`), and records that release's merge commit in `portal.json`, so a new funnel is deployed even when nothing else changed;
536
+ when the newest release merged nothing, the Space keeps the `funnel.json` it has.
537
 
538
  #1617, #1633, #2060: the manual's texts are Jinja2 templates, the engine of the index's card and docs, and go to the
539
  Space rendered, both with the same values of their markers, computed once per deploy from the index commit it reads.
app.js CHANGED
@@ -1817,12 +1817,52 @@ function renderStages() {
1817
  }
1818
  }
1819
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1820
  // ---- the Plot charts of the page (plots.js), drawn again once the module is loaded and when their width changes
1821
  let plotWidths = '';
1822
- const plotFigs = () => ['size-fig', 'where-src', 'history-fig', ...IX.stageList.map((st) => `funnel-${st.id}`)].filter((id) => $(id));
1823
  function drawPlots() {
1824
  renderSizes();
1825
  renderWhere();
 
1826
  renderStages();
1827
  renderHistory();
1828
  plotWidths = plotFigs().map(contentWidth).join(' ');
@@ -1951,6 +1991,7 @@ function renderLanguage() {
1951
  renderHeader();
1952
  renderCategories();
1953
  renderCollections();
 
1954
  renderStages();
1955
  fillFilterOptions();
1956
  drawCharts();
@@ -2035,6 +2076,7 @@ async function main() {
2035
  renderHeader();
2036
  renderCategories();
2037
  renderCollections();
 
2038
  renderStages();
2039
  fillFilterRow();
2040
  syncFilterRow();
 
1817
  }
1818
  }
1819
 
1820
+ // ---- #2287: the funnel's index half in one figure (funnel.json beside the page: the metadata of the release's asset
1821
+ // release_published, which the task dashboard reads too, so both give one number): the corpora's rows, the units
1822
+ // chosen for admission by selection files of corpus rows, the units admitted into material sets and what each
1823
+ // training stage keeps, one bar each on one log axis (they span thousands to billions), in the color of the layer
1824
+ // each counts; its Table view the same under each main domain. Without funnel.json the figure says so.
1825
+ const flowDomain = (code) => domainName(code === 'multi' ? MULTI : code === 'unclassified' ? NONE : code);
1826
+ function flowLayers(f) {
1827
+ const ly = f.funnel.layers;
1828
+ const unit = (st) => (st.measure === 'units' ? 'n.unit' : 'n.corpusRow');
1829
+ return [
1830
+ { key: 'corpus', name: t('flow.corpus'), kind: CORPUS, n: ly.corpus.rows, noun: 'n.corpusRow', by: ly.corpus.by_domain,
1831
+ missing: ly.corpus.rows_not_counted },
1832
+ { key: 'chosen', name: t('flow.chosen'), kind: CORPUS, n: ly.chosen.units, noun: 'n.unit', by: ly.chosen.by_domain,
1833
+ missing: ly.chosen.units_not_counted },
1834
+ { key: 'admitted', name: t('flow.admitted'), kind: MATERIAL_SET, n: ly.admitted.units, noun: 'n.unit', by: ly.admitted.by_domain,
1835
+ missing: ly.admitted.units_not_counted,
1836
+ more: t('flow.split', { promoted: fmtInt(ly.admitted.promoted), search: fmtInt(ly.admitted.search) }) },
1837
+ ...f.funnel.stages.map((st) => ({ key: `stage:${st.id}`, name: t('flow.kept', { stage: stageTitle(st) }),
1838
+ kind: st.measure === 'units' ? MATERIAL_SET : CORPUS, n: st.kept, noun: unit(st),
1839
+ by: Object.fromEntries(Object.entries(st.by_domain || {}).map(([d, x]) => [d, x.kept])),
1840
+ more: st.before == null ? null : t('flow.before', { what: plural(unit(st), st.before) }) })),
1841
+ ];
1842
+ }
1843
+ function renderFlow() {
1844
+ const fig = $('flow-fig');
1845
+ const layers = IX.funnel ? flowLayers(IX.funnel) : [];
1846
+ const rows = layers.filter((x) => x.n > 0).map((x) => ({ key: x.key, name: x.name, value: x.n, text: fmtCompact(x.n),
1847
+ color: x.kind === CORPUS ? 'var(--other)' : 'var(--ours)', onBar: x.kind === CORPUS ? 'var(--ink)' : null,
1848
+ title: [x.name, plural(x.noun, x.n) + uncounted(x.missing), x.more].filter(Boolean).join('\n') }));
1849
+ if (!rows.length) { clear(fig).append(el('p', { class: 'small' }, t('flow.none'))); return; }
1850
+ const domains = [...new Set(layers.flatMap((x) => Object.keys(x.by || {})))]
1851
+ .sort((a, b) => (layers[0].by?.[b] || 0) - (layers[0].by?.[a] || 0) || a.localeCompare(b));
1852
+ figureWithToggle(fig, 'flow', t('flow.title', { release: IX.funnel.release || IX.funnel.index?.slice(0, 12) }), {
1853
+ chart: () => plotted(() => P.bars(rows, { width: contentWidth('flow-fig'), log: true, fmt: fmtCompact, label: t('flow.head') })),
1854
+ table: () => table([t('flow.domain'), ...layers.map((x) => ({ label: x.name, r: true }))],
1855
+ domains.map((d) => [flowDomain(d), ...layers.map((x) => (x.n == null ? '' : fmtInt(x.by?.[d] || 0)))])),
1856
+ }, () => el('p', { class: 'small fig-foot' }, tableShown('flow') ? t('flow.byDomain') : t('size.logScale')));
1857
+ }
1858
+
1859
  // ---- the Plot charts of the page (plots.js), drawn again once the module is loaded and when their width changes
1860
  let plotWidths = '';
1861
+ const plotFigs = () => ['size-fig', 'where-src', 'flow-fig', 'history-fig', ...IX.stageList.map((st) => `funnel-${st.id}`)].filter((id) => $(id));
1862
  function drawPlots() {
1863
  renderSizes();
1864
  renderWhere();
1865
+ renderFlow();
1866
  renderStages();
1867
  renderHistory();
1868
  plotWidths = plotFigs().map(contentWidth).join(' ');
 
1991
  renderHeader();
1992
  renderCategories();
1993
  renderCollections();
1994
+ renderFlow();
1995
  renderStages();
1996
  fillFilterOptions();
1997
  drawCharts();
 
2076
  renderHeader();
2077
  renderCategories();
2078
  renderCollections();
2079
+ renderFlow();
2080
  renderStages();
2081
  fillFilterRow();
2082
  syncFilterRow();
data.js CHANGED
@@ -351,7 +351,7 @@ export async function loadIndex(params, onNote) {
351
  const vocabFile = vocabUrl ? () => urlFile(vocabUrl) : has('meta/vocabulary.json') ? () => at('meta/vocabulary.json') : null;
352
 
353
  const [cat, sourcesJson, coverage, batches, releases, changelog, vocabJson, figures, record, stagesJson, translations,
354
- stageFigures] = await Promise.all([
355
  catalogUrl ? readCatalog(urlFile(catalogUrl), /\.json(\?|$)/i.test(catalogUrl)) : readCatalog(at('meta/catalog.parquet'), false),
356
  jsonOf(at('meta/sources.json')),
357
  jsonOf(at('meta/coverage.json')),
@@ -371,6 +371,10 @@ export async function loadIndex(params, onNote) {
371
  // #2066: the Chinese names of collections and sources (#2065), and the counts per stage with what each rule leaves
372
  has(TRANSLATIONS) ? readParquet(at(TRANSLATIONS)) : [],
373
  has(STAGE_FIGURES) ? jsonOf(at(STAGE_FIGURES)) : null,
 
 
 
 
374
  ]);
375
  // #2006: the stages describe the commit the page reads only when stages.json names it and was counted over its stage
376
  // fields; otherwise no source has a stage and the filter by stage says why (#2066: in full, stagesWhy): no
@@ -498,6 +502,9 @@ export async function loadIndex(params, onNote) {
498
  stagesWhy,
499
  stageList: stagesJson?.stages || stageFigures?.stages || [],
500
  stageFigures: parseStageFigures(stageFigures),
 
 
 
501
  // #2066: the Chinese text of each English text the release shows (TRANSLATIONS, #2065): English -> Chinese, empty
502
  // for a commit without the table
503
  zh: new Map(translations.map((r) => [r.en, r.zh])),
 
351
  const vocabFile = vocabUrl ? () => urlFile(vocabUrl) : has('meta/vocabulary.json') ? () => at('meta/vocabulary.json') : null;
352
 
353
  const [cat, sourcesJson, coverage, batches, releases, changelog, vocabJson, figures, record, stagesJson, translations,
354
+ stageFigures, funnelJson] = await Promise.all([
355
  catalogUrl ? readCatalog(urlFile(catalogUrl), /\.json(\?|$)/i.test(catalogUrl)) : readCatalog(at('meta/catalog.parquet'), false),
356
  jsonOf(at('meta/sources.json')),
357
  jsonOf(at('meta/coverage.json')),
 
371
  // #2066: the Chinese names of collections and sources (#2065), and the counts per stage with what each rule leaves
372
  has(TRANSLATIONS) ? readParquet(at(TRANSLATIONS)) : [],
373
  has(STAGE_FIGURES) ? jsonOf(at(STAGE_FIGURES)) : null,
374
+ // #2287: the funnel's index half beside the page (03_data_line/steps/portal.py funnel_screen: the metadata of the
375
+ // release's asset release_published, which the task dashboard reads too), read past the browser's cache as
376
+ // stages.json is; a page without it draws no funnel figure
377
+ fetch('funnel.json', { cache: 'no-cache' }).then((r) => (r.ok ? r.json() : null)).catch(() => null),
378
  ]);
379
  // #2006: the stages describe the commit the page reads only when stages.json names it and was counted over its stage
380
  // fields; otherwise no source has a stage and the filter by stage says why (#2066: in full, stagesWhy): no
 
502
  stagesWhy,
503
  stageList: stagesJson?.stages || stageFigures?.stages || [],
504
  stageFigures: parseStageFigures(stageFigures),
505
+ // #2287: the funnel's index half (funnel.json: release, the release tag it counts, index, its merge commit, and
506
+ // funnel, its layers and stages by domain), whatever commit it names; null without it
507
+ funnel: funnelJson?.funnel ? funnelJson : null,
508
  // #2066: the Chinese text of each English text the release shows (TRANSLATIONS, #2065): English -> Chinese, empty
509
  // for a commit without the table
510
  zh: new Map(translations.map((r) => [r.en, r.zh])),
first.json CHANGED
@@ -2,7 +2,7 @@
2
  "collections": 14,
3
  "date": "2026-11-10",
4
  "format": "portal-first/5",
5
- "index": "5b9f5a4d2bde4bee82f6a0b178b5ab088603564f",
6
  "raw_rows": 1216362128,
7
  "raw_rows_not_counted": 0,
8
  "raw_sources": 139,
 
2
  "collections": 14,
3
  "date": "2026-11-10",
4
  "format": "portal-first/5",
5
+ "index": "83dae072040e8688c08c84fb72fdc8690d0956e5",
6
  "raw_rows": 1216362128,
7
  "raw_rows_not_counted": 0,
8
  "raw_sources": 139,
funnel.json ADDED
@@ -0,0 +1,116 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "format": "portal-funnel/1",
3
+ "funnel": {
4
+ "counted_on": null,
5
+ "domain_names": {
6
+ "D1": "software-engineering",
7
+ "D10": "misc",
8
+ "D2": "system-administration",
9
+ "D3": "data-science",
10
+ "D4": "machine-learning",
11
+ "D5": "science",
12
+ "D6": "operations",
13
+ "D7": "hardware",
14
+ "D8": "security",
15
+ "D9": "media",
16
+ "multi": "many domains",
17
+ "unclassified": "unclassified"
18
+ },
19
+ "format": "index-funnel/1",
20
+ "layers": {
21
+ "admitted": {
22
+ "by_domain": {
23
+ "D1": 479,
24
+ "D10": 59,
25
+ "D2": 274,
26
+ "D3": 1039,
27
+ "D4": 502,
28
+ "D5": 388,
29
+ "D6": 426,
30
+ "D7": 296,
31
+ "D8": 329,
32
+ "D9": 391
33
+ },
34
+ "material_sets": 1799,
35
+ "promoted": 0,
36
+ "search": 4183,
37
+ "units": 4183,
38
+ "units_not_counted": 0
39
+ },
40
+ "chosen": {
41
+ "by_domain": {
42
+ "D8": 0
43
+ },
44
+ "material_sets": 1,
45
+ "units": 0,
46
+ "units_not_counted": 0
47
+ },
48
+ "corpus": {
49
+ "by_domain": {
50
+ "D1": 45267701,
51
+ "D10": 241125,
52
+ "D2": 185826217,
53
+ "D3": 750815212,
54
+ "D4": 9636889,
55
+ "D5": 44633675,
56
+ "D6": 39404642,
57
+ "D7": 8716520,
58
+ "D8": 4521631,
59
+ "D9": 12751612,
60
+ "multi": 109262742,
61
+ "unclassified": 5284162
62
+ },
63
+ "corpora": 139,
64
+ "rows": 1216362128,
65
+ "rows_not_counted": 0
66
+ }
67
+ },
68
+ "stages": [
69
+ {
70
+ "before": null,
71
+ "by_domain": {},
72
+ "bytes": null,
73
+ "domains": [],
74
+ "forms": [],
75
+ "id": "post-training",
76
+ "kept": null,
77
+ "kind": "material_set",
78
+ "measure": "units",
79
+ "name": "Post-training",
80
+ "status": "in use",
81
+ "zh": "ๅŽ่ฎญ็ปƒ"
82
+ },
83
+ {
84
+ "before": null,
85
+ "by_domain": {},
86
+ "bytes": null,
87
+ "domains": [],
88
+ "forms": [],
89
+ "id": "mid-training",
90
+ "kept": null,
91
+ "kind": "corpus",
92
+ "measure": "rows",
93
+ "name": "Mid-training",
94
+ "status": "provisional",
95
+ "zh": "ไธญ่ฎญ็ปƒ"
96
+ },
97
+ {
98
+ "before": null,
99
+ "by_domain": {},
100
+ "bytes": null,
101
+ "domains": [],
102
+ "forms": [],
103
+ "id": "pre-training",
104
+ "kept": null,
105
+ "kind": "corpus",
106
+ "measure": "rows",
107
+ "name": "Pre-training",
108
+ "status": "provisional",
109
+ "zh": "้ข„่ฎญ็ปƒ"
110
+ }
111
+ ],
112
+ "table": 1
113
+ },
114
+ "index": "83dae072040e8688c08c84fb72fdc8690d0956e5",
115
+ "release": "release-2026-11-10.2"
116
+ }
index.html CHANGED
@@ -111,6 +111,15 @@
111
  <ul class="cat-grid" id="req-cards">
112
  <li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li>
113
  </ul>
 
 
 
 
 
 
 
 
 
114
  <!-- #2066: each training stage's funnel (app.js renderStages, plots.js bars): its units or rows through each rule, in
115
  the order of meta/stage_figures.json, down to its count; until the release has counted them, why, in full -->
116
  <div class="band-head sub-head reveal"><h3 id="stages-h" data-i18n="funnel.head">Training stages: what each rule leaves</h3>
 
111
  <ul class="cat-grid" id="req-cards">
112
  <li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li><li class="card cat sk-card"></li>
113
  </ul>
114
+ <!-- #2287: the funnel's index half in one figure (app.js renderFlow, plots.js bars): the corpora's rows, the units
115
+ chosen for admission, the units admitted into material sets and what each training stage keeps, on one log
116
+ axis, its Table view by domain (funnel.json beside the page: the metadata of the release's asset
117
+ release_published, which the task dashboard reads too) -->
118
+ <div class="band-head sub-head reveal"><h3 id="flow-h" data-i18n="flow.head">From the corpus to the training stages</h3>
119
+ <p class="small band-note" data-i18n="flow.note">What the corpora hold, what selection files chose of them for admission, the units admitted into material sets and what each training stage keeps, as the newest release counted them.</p></div>
120
+ <div class="where-grid one">
121
+ <figure class="card chart-card where-card" id="flow-fig"><div class="sk sk-block"></div></figure>
122
+ </div>
123
  <!-- #2066: each training stage's funnel (app.js renderStages, plots.js bars): its units or rows through each rule, in
124
  the order of meta/stage_figures.json, down to its count; until the release has counted them, why, in full -->
125
  <div class="band-head sub-head reveal"><h3 id="stages-h" data-i18n="funnel.head">Training stages: what each rule leaves</h3>
llms.txt CHANGED
@@ -5,7 +5,7 @@
5
  - Size of this release: Whole index: Recorded size 4.45 PB; deduplicated size 4.05 PB; size check: 99.7% match (37,927 of the 38,820 rows read compared); not compared: address unreachable 459, file gone 82, login needed 330, no size from the server 19, other error 3; estimated size of 215,326,513 of the 312,300,338 rows without a recorded size: 202 TB (an estimate; 95% range 171 TB to 245 TB, the sum of its sources' ranges, wider than a joint interval; from 14,161 rows read), the other 96,973,825 with too few rows read for an estimate. Task material: Recorded size 2.89 GB; deduplicated size 2.82 GB; size check: 99.8% match (7,204 of the 7,226 rows read compared); not compared: address unreachable 10, file gone 2, login needed 8, other error 2. Corpora: Recorded size 4.45 PB; deduplicated size 4.05 PB; size check: 99.6% match (30,723 of the 31,594 rows read compared); not compared: address unreachable 449, file gone 80, login needed 322, no size from the server 19, other error 1; estimated size of 215,326,513 of the 312,300,338 rows without a recorded size: 202 TB (an estimate; 95% range 171 TB to 245 TB, the sum of its sources' ranges, wider than a joint interval; from 14,161 rows read), the other 96,973,825 with too few rows read for an estimate. Distinct content is the sum of `size_bytes` over the rows minus the sizes of the rows that repeat another row's content (the same sha256, else sha1, else md5 and size, within the same hash scope; one copy of each content is kept, in the first source by registry order), counted after each release over the data files it published, per source in the catalog column `distinct_bytes`; archive members whose archive the same source also lists whole have other hashes, so they are not subtracted. `tools/fetch.py` removes duplicates over your own selection when it reads it. The recorded size is the sum of `size_bytes` over the rows that have one (catalog column `bytes`). The check reads a sample of rows of every data file (`meta/sizes.parquet`): 20 of its rows with a recorded size, and of its rows without one its share of its source's sample, sized for their mean size within 25% at 95% confidence by the spread of the sizes read so far (at most 400 rows per source per round, 20 for a source with none read yet); its share is the rows that match their recorded size out of the rows it compared (status `match`, `size_mismatch` or `hash_mismatch`), and the rows it could not compare, which never reached their file, are counted beside it by status (`unreachable`, `gone`, `needs_auth`, `no_length`, `error`; catalog column `size_check`). The estimate for a source's rows without a recorded size (catalog columns `unsized_rows` and `unsized_estimate`) is their number times the mean measured size of such rows read, with a 95% BCa bootstrap range (`scipy.stats.bootstrap`, its default resample count); a source with fewer than 10 such rows read has none. Above one source, the estimate is the sum of its sources' estimates and its range the sum of their ranges, which is wider than a joint interval. The figures of every level are in `meta/size_figures.json`.
6
  - Fetch with one command: `uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/main/tools/fetch.py --domain D3 --form spreadsheets --sample 3 --out ./data`. Without uv: `pip install duckdb`, download `tools/fetch.py` and run it with `python`. Select with `--source`, `--kind` (`material_set` or `corpus`: one layer), `--domain`, `--form`, `--tag`, `--ext`, or `--selection <file>` (a selection file the data page saves: the ids of its sources and its commit); `--sample N` takes N units per source, `--limit N` takes N units over the whole selection, `--unit <unit_id>` takes the unit of that `unit_id` whole (with `--source`); before its first download the run prints the units, files and total size it selected.
7
  - It pins the commit it reads and prints it; pass `--revision <commit>` to get the same files later. A name that does not exist stops with exit code 2 and the closest names.
8
- - The default profile removes duplicate copies, rows that matched a benchmark (copies of benchmark data, rows of groupings awaiting a ruling), rows not compared with the benchmark list yet (counted apart, with their own reason), files whose license does not allow shipping (license.ship yes and conditional stay), rows recorded in a language other than English, personal data (catalog pii.level present or suspected, rows marked sensitive), and files above 20 MB. Before downloading it prints how many units each criterion removed; each criterion has its own option (`--keep-duplicates`, `--allow-benchmark`, `--allow-not-compared`, `--any-license`, `--any-language`, `--allow-personal-data`, `--any-size`; `--allow-benchmark` also keeps rows not compared yet), and `--all` turns off every criterion except benchmark and not compared, so rows that matched a benchmark or were not compared yet come in only with their own options. Compared with the benchmark list (keys of 2026-11-05): 1,799 of 1,799 material sets; 133 of 139 corpora. The profile also removes every row whose data file was not compared (`--allow-benchmark` keeps them, unchecked).
9
  - Output: `./data/<domain>/<source>/<unit folder>/...` holds each unit whole, in a folder of its own named after its `unit_id`, its files at their paths within the unit; `./data/<domain>/<source>/` also holds the README.md and LICENSE of a material set; `./data/_meta/manifest.jsonl` has one line per file (source, path, url, sha256, bytes, spdx, licence_url, publisher, retrieved, origin_kind, leaf, unit_id, verified); `./data/_meta/sizes.tsv` one line per source. Every file is checked against the sha256 the index records. A unit stored inside a container (such as a disk image inside a zip) is written as the unit itself: fetch.py takes the member out and checks it against the unit's recorded sha256, so no second tool is needed; only units marked expand (archives meant to be used as folders) are unpacked by `tools/to_pool.py`.
10
  - The index holds material only: what a task checks is decided when the task is built from the material. Keep out of the environment the agent works in whatever the task checks its result against:
11
  - Workbooks (such as the form `spreadsheets`): a published workbook holds the cached results of its formulas, so a workbook put in as fetched hands the agent the values a task may ask it to compute. Strip the cached values first: in .xlsx and .xlsm the `<v>` of every formula cell (`xl/worksheets/*.xml`), the chart caches (`c:numCache` and `c:strCache` in `xl/charts/*.xml`), the pivot caches (`xl/pivotCache/`, definitions and records) and the external link caches (`xl/externalLinks/`); in .ods the value attributes (`office:value` and the like) and the text of formula cells; in .xls the cached result of every FORMULA record and its STRING record.
 
5
  - Size of this release: Whole index: Recorded size 4.45 PB; deduplicated size 4.05 PB; size check: 99.7% match (37,927 of the 38,820 rows read compared); not compared: address unreachable 459, file gone 82, login needed 330, no size from the server 19, other error 3; estimated size of 215,326,513 of the 312,300,338 rows without a recorded size: 202 TB (an estimate; 95% range 171 TB to 245 TB, the sum of its sources' ranges, wider than a joint interval; from 14,161 rows read), the other 96,973,825 with too few rows read for an estimate. Task material: Recorded size 2.89 GB; deduplicated size 2.82 GB; size check: 99.8% match (7,204 of the 7,226 rows read compared); not compared: address unreachable 10, file gone 2, login needed 8, other error 2. Corpora: Recorded size 4.45 PB; deduplicated size 4.05 PB; size check: 99.6% match (30,723 of the 31,594 rows read compared); not compared: address unreachable 449, file gone 80, login needed 322, no size from the server 19, other error 1; estimated size of 215,326,513 of the 312,300,338 rows without a recorded size: 202 TB (an estimate; 95% range 171 TB to 245 TB, the sum of its sources' ranges, wider than a joint interval; from 14,161 rows read), the other 96,973,825 with too few rows read for an estimate. Distinct content is the sum of `size_bytes` over the rows minus the sizes of the rows that repeat another row's content (the same sha256, else sha1, else md5 and size, within the same hash scope; one copy of each content is kept, in the first source by registry order), counted after each release over the data files it published, per source in the catalog column `distinct_bytes`; archive members whose archive the same source also lists whole have other hashes, so they are not subtracted. `tools/fetch.py` removes duplicates over your own selection when it reads it. The recorded size is the sum of `size_bytes` over the rows that have one (catalog column `bytes`). The check reads a sample of rows of every data file (`meta/sizes.parquet`): 20 of its rows with a recorded size, and of its rows without one its share of its source's sample, sized for their mean size within 25% at 95% confidence by the spread of the sizes read so far (at most 400 rows per source per round, 20 for a source with none read yet); its share is the rows that match their recorded size out of the rows it compared (status `match`, `size_mismatch` or `hash_mismatch`), and the rows it could not compare, which never reached their file, are counted beside it by status (`unreachable`, `gone`, `needs_auth`, `no_length`, `error`; catalog column `size_check`). The estimate for a source's rows without a recorded size (catalog columns `unsized_rows` and `unsized_estimate`) is their number times the mean measured size of such rows read, with a 95% BCa bootstrap range (`scipy.stats.bootstrap`, its default resample count); a source with fewer than 10 such rows read has none. Above one source, the estimate is the sum of its sources' estimates and its range the sum of their ranges, which is wider than a joint interval. The figures of every level are in `meta/size_figures.json`.
6
  - Fetch with one command: `uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/main/tools/fetch.py --domain D3 --form spreadsheets --sample 3 --out ./data`. Without uv: `pip install duckdb`, download `tools/fetch.py` and run it with `python`. Select with `--source`, `--kind` (`material_set` or `corpus`: one layer), `--domain`, `--form`, `--tag`, `--ext`, or `--selection <file>` (a selection file the data page saves: the ids of its sources and its commit); `--sample N` takes N units per source, `--limit N` takes N units over the whole selection, `--unit <unit_id>` takes the unit of that `unit_id` whole (with `--source`); before its first download the run prints the units, files and total size it selected.
7
  - It pins the commit it reads and prints it; pass `--revision <commit>` to get the same files later. A name that does not exist stops with exit code 2 and the closest names.
8
+ - The default profile removes duplicate copies, rows that matched a benchmark (copies of benchmark data, rows of groupings awaiting a ruling), rows not compared with the benchmark list yet (counted apart, with their own reason), files whose license does not allow shipping (license.ship yes and conditional stay), rows recorded in a language other than English, personal data (catalog pii.level present or suspected, rows marked sensitive), and files above 20 MB. Before downloading it prints how many units each criterion removed; each criterion has its own option (`--keep-duplicates`, `--allow-benchmark`, `--allow-not-compared`, `--any-license`, `--any-language`, `--allow-personal-data`, `--any-size`; `--allow-benchmark` also keeps rows not compared yet), and `--all` turns off every criterion except benchmark and not compared, so rows that matched a benchmark or were not compared yet come in only with their own options. Compared with the benchmark list (keys of 2026-11-08): 1,799 of 1,799 material sets; 139 of 139 corpora. The profile also removes every row whose data file was not compared (`--allow-benchmark` keeps them, unchecked).
9
  - Output: `./data/<domain>/<source>/<unit folder>/...` holds each unit whole, in a folder of its own named after its `unit_id`, its files at their paths within the unit; `./data/<domain>/<source>/` also holds the README.md and LICENSE of a material set; `./data/_meta/manifest.jsonl` has one line per file (source, path, url, sha256, bytes, spdx, licence_url, publisher, retrieved, origin_kind, leaf, unit_id, verified); `./data/_meta/sizes.tsv` one line per source. Every file is checked against the sha256 the index records. A unit stored inside a container (such as a disk image inside a zip) is written as the unit itself: fetch.py takes the member out and checks it against the unit's recorded sha256, so no second tool is needed; only units marked expand (archives meant to be used as folders) are unpacked by `tools/to_pool.py`.
10
  - The index holds material only: what a task checks is decided when the task is built from the material. Keep out of the environment the agent works in whatever the task checks its result against:
11
  - Workbooks (such as the form `spreadsheets`): a published workbook holds the cached results of its formulas, so a workbook put in as fetched hands the agent the values a task may ask it to compute. Strip the cached values first: in .xlsx and .xlsm the `<v>` of every formula cell (`xl/worksheets/*.xml`), the chart caches (`c:numCache` and `c:strCache` in `xl/charts/*.xml`), the pivot caches (`xl/pivotCache/`, definitions and records) and the external link caches (`xl/externalLinks/`); in .ods the value attributes (`office:value` and the like) and the text of formula cells; in .xls the cached result of every FORMULA record and its STRING record.
locales/en.json CHANGED
@@ -403,6 +403,18 @@
403
  "rule.not compared": "After the rule on rows not compared with the benchmark list",
404
  "rule.personal data": "After the personal-data rule",
405
  "rule.duplicate": "After deduplication: the stage's count",
 
 
 
 
 
 
 
 
 
 
 
 
406
  "status.flagged": "Flagged, not usable as it is",
407
  "fetch.all": "All {{rows}} sampled rows read fetched and matched their recorded size and hash (read {{date}}).",
408
  "fetch.some": "{{fetched}} of the {{rows}} sampled rows read ({{share}}) fetched and matched their recorded size and hash (read {{date}}); the others did not.",
 
403
  "rule.not compared": "After the rule on rows not compared with the benchmark list",
404
  "rule.personal data": "After the personal-data rule",
405
  "rule.duplicate": "After deduplication: the stage's count",
406
+ "flow.head": "From the corpus to the training stages",
407
+ "flow.note": "What the corpora hold, what selection files chose of them for admission, the units admitted into material sets and what each training stage keeps, as the newest release counted them.",
408
+ "flow.title": "Release {{release}}: from the corpus to the training stages",
409
+ "flow.corpus": "Corpus rows",
410
+ "flow.chosen": "Chosen for admission by selection files",
411
+ "flow.admitted": "Admitted into material sets",
412
+ "flow.kept": "{{stage}} keeps",
413
+ "flow.split": "{{promoted}} promoted from the corpus, {{search}} from the search",
414
+ "flow.before": "{{what}} in the stage's condition before the rules",
415
+ "flow.domain": "Domain",
416
+ "flow.byDomain": "Each source is counted under the top-level domain of its first domain code.",
417
+ "flow.none": "This page was deployed without the funnel (no funnel.json beside it): it shows once a release has counted it.",
418
  "status.flagged": "Flagged, not usable as it is",
419
  "fetch.all": "All {{rows}} sampled rows read fetched and matched their recorded size and hash (read {{date}}).",
420
  "fetch.some": "{{fetched}} of the {{rows}} sampled rows read ({{share}}) fetched and matched their recorded size and hash (read {{date}}); the others did not.",
locales/zh.json CHANGED
@@ -373,6 +373,18 @@
373
  "rule.not compared": "ๆœชไธŽๅŸบๅ‡†ๆต‹่ฏ•ๆธ…ๅ•ๆฏ”ๅฏน็š„่กŒ็š„่ง„ๅˆ™ไน‹ๅŽ",
374
  "rule.personal data": "ไธชไบบๆ•ฐๆฎ่ง„ๅˆ™ไน‹ๅŽ",
375
  "rule.duplicate": "ๅŽป้‡ไน‹ๅŽ๏ผš่ฏฅ้˜ถๆฎต็š„่ฎกๆ•ฐ",
 
 
 
 
 
 
 
 
 
 
 
 
376
  "status.flagged": "ๅทฒๆ ‡่ฎฐ๏ผŒไธ่ƒฝ็›ดๆŽฅไฝฟ็”จ",
377
  "fetch.all": "่ฏป่ฟ‡็š„ {{rows}} ไธชๆŠฝๆ ท่กŒๅ…จ้ƒจๅ–ๅˆฐ๏ผŒๅคงๅฐๅ’Œๅ“ˆๅธŒไธŽ่ฎฐๅฝ•ไธ€่‡ด๏ผˆ่ฏปๅ–ไบŽ {{date}}๏ผ‰ใ€‚",
378
  "fetch.some": "่ฏป่ฟ‡็š„ {{rows}} ไธชๆŠฝๆ ท่กŒไธญๆœ‰ {{fetched}} ไธช๏ผˆ{{share}}๏ผ‰ๅ–ๅˆฐ๏ผŒๅคงๅฐๅ’Œๅ“ˆๅธŒไธŽ่ฎฐๅฝ•ไธ€่‡ด๏ผˆ่ฏปๅ–ไบŽ {{date}}๏ผ‰๏ผ›ๅ…ถไฝ™ๆฒกๆœ‰ใ€‚",
 
373
  "rule.not compared": "ๆœชไธŽๅŸบๅ‡†ๆต‹่ฏ•ๆธ…ๅ•ๆฏ”ๅฏน็š„่กŒ็š„่ง„ๅˆ™ไน‹ๅŽ",
374
  "rule.personal data": "ไธชไบบๆ•ฐๆฎ่ง„ๅˆ™ไน‹ๅŽ",
375
  "rule.duplicate": "ๅŽป้‡ไน‹ๅŽ๏ผš่ฏฅ้˜ถๆฎต็š„่ฎกๆ•ฐ",
376
+ "flow.head": "ไปŽ่ฏญๆ–™ๅบ“ๅˆฐๅ„่ฎญ็ปƒ้˜ถๆฎต",
377
+ "flow.note": "่ฏญๆ–™ๅบ“้‡Œๆœ‰ๅคšๅฐ‘ใ€้€‰ๆ‹ฉๆ–‡ไปถไปŽไธญๆŒ‘ๅ‡บๅคšๅฐ‘้€ๅŽปๆ”ถๅฝ•ใ€ๆ”ถๅฝ•่ฟ›ๆๆ–™้›†็š„ๅ•ๅ…ƒ๏ผŒไปฅๅŠๆฏไธช่ฎญ็ปƒ้˜ถๆฎต็•™ไธ‹ๅคšๅฐ‘๏ผŒๅ‡ๆŒ‰ๆœ€ๆ–ฐไธ€ๆฌกๅ‘ๅธƒ็š„่ฎกๆ•ฐใ€‚",
378
+ "flow.title": "ๅ‘ๅธƒ {{release}}๏ผšไปŽ่ฏญๆ–™ๅบ“ๅˆฐๅ„่ฎญ็ปƒ้˜ถๆฎต",
379
+ "flow.corpus": "่ฏญๆ–™ๅบ“ๆ•ฐๆฎ่กŒ",
380
+ "flow.chosen": "้€‰ๆ‹ฉๆ–‡ไปถๆŒ‘ๅ‡บ้€ๅŽปๆ”ถๅฝ•็š„",
381
+ "flow.admitted": "ๆ”ถๅฝ•่ฟ›ๆๆ–™้›†็š„",
382
+ "flow.kept": "{{stage}}็•™ไธ‹็š„",
383
+ "flow.split": "{{promoted}} ไธชไปŽ่ฏญๆ–™ๅบ“ๆๅ‡่€Œๆฅ๏ผŒ{{search}} ไธชๆฅ่‡ชๆœ็ดข",
384
+ "flow.before": "็ฌฆๅˆ่ฏฅ้˜ถๆฎต็š„ๆกไปถใ€ๅœจๅ„ๆก่ง„ๅˆ™ไน‹ๅ‰๏ผš{{what}}",
385
+ "flow.domain": "้ข†ๅŸŸ",
386
+ "flow.byDomain": "ๆฏไธชๆฅๆบ่ฎกๅœจๅฎƒ็ฌฌไธ€ไธช้ข†ๅŸŸไปฃ็ ๆ‰€ๅฑž็š„้กถๅฑ‚้ข†ๅŸŸไธ‹ใ€‚",
387
+ "flow.none": "ๆœฌ้กต้ƒจ็ฝฒๆ—ถๆฒกๆœ‰ๅธฆไธŠๆผๆ–—๏ผˆๆ—่พนๆฒกๆœ‰ funnel.json๏ผ‰๏ผš็ญ‰ๆŸๆฌกๅ‘ๅธƒ่ฎกๆ•ฐไน‹ๅŽๆ‰ไผšๆ˜พ็คบใ€‚",
388
  "status.flagged": "ๅทฒๆ ‡่ฎฐ๏ผŒไธ่ƒฝ็›ดๆŽฅไฝฟ็”จ",
389
  "fetch.all": "่ฏป่ฟ‡็š„ {{rows}} ไธชๆŠฝๆ ท่กŒๅ…จ้ƒจๅ–ๅˆฐ๏ผŒๅคงๅฐๅ’Œๅ“ˆๅธŒไธŽ่ฎฐๅฝ•ไธ€่‡ด๏ผˆ่ฏปๅ–ไบŽ {{date}}๏ผ‰ใ€‚",
390
  "fetch.some": "่ฏป่ฟ‡็š„ {{rows}} ไธชๆŠฝๆ ท่กŒไธญๆœ‰ {{fetched}} ไธช๏ผˆ{{share}}๏ผ‰ๅ–ๅˆฐ๏ผŒๅคงๅฐๅ’Œๅ“ˆๅธŒไธŽ่ฎฐๅฝ•ไธ€่‡ด๏ผˆ่ฏปๅ–ไบŽ {{date}}๏ผ‰๏ผ›ๅ…ถไฝ™ๆฒกๆœ‰ใ€‚",
manual/manual.en.md CHANGED
@@ -8,7 +8,7 @@ It is neither a task list nor a copy of the files.
8
  Entry points: the data page https://huggingface.co/spaces/Battam/agentic-env-catalog (to share a state, use the page itself, https://battam-agentic-env-catalog.static.hf.space/index.html );
9
  the dataset https://huggingface.co/datasets/Battam/agentic-env-index .
10
 
11
- The numbers and command outputs below were filled in at deploy time, from the index commit `5b9f5a4d2bde4bee82f6a0b178b5ab088603564f` (release tag `release-2026-11-10.1`).
12
 
13
  ## Tutorial
14
 
@@ -87,7 +87,7 @@ or add `&open=zenodo-workbooks-cc-by` to the address.
87
  ### 5. Go to "Get the data" and keep the selection
88
 
89
  **Do**: click "Get the data" at the top;
90
- under "Keep this selection" click "Download the selection list (JSON)" and put `agentic-env-selection-5b9f5a4.json` in your fetch folder.
91
 
92
  **See**: the heading "Take 3 units from each of 52 material sets", then "Units per source", the card "1 Take", "What it writes", the six kinds "Left out by default", and "2 Take it again from the file", which reads the list.
93
  The list records release, filters and sources:
@@ -96,7 +96,7 @@ The list records release, filters and sources:
96
  {
97
  "format": "agentic-env-selection/1",
98
  "dataset": "Battam/agentic-env-index",
99
- "revision": "release-2026-11-10.1",
100
  "filters": {
101
  "collection": "A",
102
  "train": "yes",
@@ -120,9 +120,9 @@ for 200 in all, use `--limit`.
120
  **Do**: in the list's folder, run the command of "Take it again from the file" with `--sample 3` replaced by `--limit 220` (220 units in all, in a fixed order: a margin over 200, since some sites fail) and `--out ./data` by `--out ./take`:
121
 
122
  ```sh
123
- uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.1/tools/fetch.py \
124
- --selection agentic-env-selection-5b9f5a4.json \
125
- --revision release-2026-11-10.1 \
126
  --ext xlsx \
127
  --ext xlsm \
128
  --ext xls \
@@ -147,10 +147,10 @@ ok abhishekumarr00-supply-chain-mastery workbooks/abhishekumarr00-supply-chain-m
147
  ok austender-ict-procurement-statistics workbooks/201314austenderictcontractstatistcs.xlsx: 26.3kB
148
  ok austender-ict-procurement-statistics workbooks/2016-17-austender-ict-contract-data-category-sme-and-small-participation.xlsx: 32.9kB
149
  ...
150
- 319 files fetched and checked (96.5MB) into take; details in take/_meta/fetch_run.json
151
- files of units: 3 failed, 217 ok; files of the sources themselves: 102 ok
152
- run_id 20261110T204854.721151Z: 3 failed, 319 ok (lines in take/_meta/fetch_manifest.jsonl)
153
- its lines that are not ok, in DuckDB: SELECT json FROM read_json_objects('take/_meta/fetch_manifest.jsonl') WHERE json_extract_string(json, 'run_id') = '20261110T204854.721151Z' AND json_extract_string(json, 'status') <> 'ok'
154
  ```
155
 
156
  Before the first download, `removed by` gives the units the default profile removed and the option that keeps them, counted in the data files read as far as `--limit` needed;
@@ -165,8 +165,8 @@ failed western-australia-government-workbooks workbooks/wa-gross-regional-produc
165
  incomplete: 1 unit miss a file (the files fetched are kept; _meta/fetch_manifest.jsonl says why): ...
166
  5 files fetched and checked (1.35MB) into probe; details in probe/_meta/fetch_run.json
167
  files of units: 1 failed, 1 ok; files of the sources themselves: 4 ok
168
- run_id 20261110T205223.232212Z: 1 failed, 5 ok (lines in probe/_meta/fetch_manifest.jsonl)
169
- its lines that are not ok, in DuckDB: SELECT json FROM read_json_objects('probe/_meta/fetch_manifest.jsonl') WHERE json_extract_string(json, 'run_id') = '20261110T205223.232212Z' AND json_extract_string(json, 'status') <> 'ok'
170
  ```
171
 
172
  **Get**: every `ok` file matched its recorded size and hash.
@@ -206,18 +206,20 @@ GROUP BY ALL ORDER BY source;
206
  โ”‚ excel_units โ”‚
207
  โ”‚ int64 โ”‚
208
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
209
- โ”‚ 217 โ”‚
210
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
211
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
212
  โ”‚ verified โ”‚ files โ”‚ bytes โ”‚
213
  โ”‚ boolean โ”‚ int64 โ”‚ int128 โ”‚
214
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
215
- โ”‚ true โ”‚ 319 โ”‚ 96538957 โ”‚
216
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
217
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
218
  โ”‚ source โ”‚ reason โ”‚ files โ”‚
219
  โ”‚ varchar โ”‚ varchar โ”‚ int64 โ”‚
220
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
 
 
221
  โ”‚ victoria-government-workbooks โ”‚ https://www.planning.vic.gov.au/__data/assets โ”‚ 2 โ”‚
222
  โ”‚ western-australia-government-workbooks โ”‚ https://catalogue.data.wa.gov.au/dataset/03b1 โ”‚ 1 โ”‚
223
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
@@ -239,7 +241,7 @@ a workbook stores the cached results of its formulas, so a task that asks for co
239
  **See** (truncated):
240
 
241
  ```text
242
- Dataset commit 5b9f5a4d2bde4bee82f6a0b178b5ab088603564f: https://huggingface.co/datasets/Battam/agentic-env-index/tree/5b9f5a4d2bde4bee82f6a0b178b5ab088603564f
243
  ...
244
  - Source: `austender-ict-procurement-statistics`
245
  - License: CC-BY-3.0-AU
@@ -254,7 +256,7 @@ where it says `not stated`, `spdx`, `licence_url` and `publisher` in `take/_meta
254
  ## How-to guides
255
 
256
  One goal each;
257
- commands are pinned to `release-2026-11-10.1`.
258
 
259
  ### Filter by file type and license
260
 
@@ -284,11 +286,11 @@ The dataset also has `cards/<source>.md` and `.yaml`.
284
 
285
  ### Pin a release
286
 
287
- The page's commands pin the script URL and `--revision` to `release-2026-11-10.1`;
288
  the script's first line names the commit:
289
 
290
  ```text
291
- reading hf://datasets/Battam/agentic-env-index@5b9f5a4d2bde4bee82f6a0b178b5ab088603564f (commit 5b9f5a4d2bde4bee82f6a0b178b5ab088603564f); pass --revision 5b9f5a4d2bde4bee82f6a0b178b5ab088603564f to select the same rows later
292
  ```
293
 
294
  Later, `--revision <commit>`, the same tag or the selection list take the same files;
@@ -297,7 +299,7 @@ On the data page, `rev=` reads another commit.
297
  A misspelt source name stops with exit code 2:
298
 
299
  ```text
300
- error: --source dosfstools-fsk: no such source in meta/catalog.parquet at commit 5b9f5a4d2bde4bee82f6a0b178b5ab088603564f; closest: dosfstools-fsck-cases
301
  ```
302
 
303
  ### Take only part
@@ -331,7 +333,7 @@ The query on the dataset card (output truncated):
331
 
332
  ```sql
333
  SELECT source, collection, title, units, files, top_types, license.tier AS license_tier, needs_auth
334
- FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.1/meta/catalog.parquet'
335
  WHERE kind = 'material_set' AND needs_auth = 'no' AND license.tier IN ('permissive', 'share-alike')
336
  ORDER BY collection, units DESC;
337
  ```
@@ -350,7 +352,7 @@ Let DuckDB write the `--source` options, say for the disk image material sets wi
350
 
351
  ```sql
352
  SELECT string_agg('--source ' || source, ' ' ORDER BY source) AS options
353
- FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.1/meta/catalog.parquet'
354
  WHERE kind = 'material_set' AND collection = 'B'
355
  AND license.tier = 'share-alike';
356
  ```
@@ -359,7 +361,7 @@ WHERE kind = 'material_set' AND collection = 'B'
359
  --source adflib-adf-deleted --source adflib-adf-malformed --source btrfs-progs-convert-needs-recovery --source btrfs-progs-fsck-raw-images --source btrfs-progs-fuzz-images --source dfir-ntfs-fat-deleted --source dftt-deleted-files --source dftt-partition-tables --source dissect-hypervisor-disk-fixtures --source dosfstools-fsck-cases --source e2fsprogs-fsck-cases --source exfatprogs-fsck-cases --source libcdio-iso-malformed --source nist-cfreds-dfr-exfat --source ntfs-corrupted-images --source oe-ext4-hostile-fixtures --source qemu-iotests-sample-images
360
  ```
361
 
362
- Put them after `fetch.py --revision release-2026-11-10.1`.
363
  Read only the catalog, not all of `meta/`.
364
 
365
  ### Take disk images and check them
@@ -371,8 +373,8 @@ Run "1 Take" (output truncated):
371
  !["Get the data" after picking three disk image material sets](S09-en.png)
372
 
373
  ```sh
374
- uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.1/tools/fetch.py \
375
- --revision release-2026-11-10.1 \
376
  --source dosfstools-fsck-cases \
377
  --source e2fsprogs-fsck-cases \
378
  --source exfatprogs-fsck-cases \
@@ -388,7 +390,7 @@ ok dosfstools-fsck-cases README.md: 3.68kB
388
  ok exfatprogs-fsck-cases cases/bad_dentries/exfat.img: 5.24MB
389
  22 files fetched and checked (15.9MB) into data; details in data/_meta/fetch_run.json
390
  files of units: 15 ok; files of the sources themselves: 7 ok
391
- run_id 20261110T205228.568116Z: 22 ok (lines in data/_meta/fetch_manifest.jsonl)
392
  ```
393
 
394
  An exFAT image is `exfat.img.tar.xz` upstream;
@@ -429,7 +431,7 @@ One, the catalog's `contamination.level` (values in the reference), say for mate
429
 
430
  ```sql
431
  SELECT contamination.level AS level, count(*) AS material_sets, sum(units) AS units
432
- FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.1/meta/catalog.parquet'
433
  WHERE kind = 'material_set'
434
  AND list_has_any(top_types, ['docx', 'xlsx', 'pptx', 'pdf'])
435
  GROUP BY ALL;
@@ -449,8 +451,8 @@ GROUP BY ALL;
449
  Two, take the picked sources and read `removed by benchmark` and `removed by not compared`, printed before the first download (output truncated):
450
 
451
  ```sh
452
- uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.1/tools/fetch.py \
453
- --revision release-2026-11-10.1 \
454
  --source python-docx-acceptance-test-files \
455
  --source python-pptx-acceptance-test-files \
456
  --source docling-docx-groundtruth \
@@ -479,7 +481,7 @@ Three, match file by file against the overlap table;
479
  ```sql
480
  SELECT o.bench, o.hit_class, count(*) AS files
481
  FROM read_json_auto('office3/_meta/fetch_manifest.jsonl') AS f
482
- JOIN 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.1/meta/bench_overlap.parquet' AS o
483
  ON o.source = f.source AND o.uri = f.uri
484
  WHERE f.status = 'ok'
485
  GROUP BY ALL;
@@ -511,7 +513,7 @@ to cite it:
511
  @misc{agentic_env_index,
512
  title = {agentic-env-index: an index of public materials for building agent environments},
513
  howpublished = {https://huggingface.co/datasets/Battam/agentic-env-index},
514
- note = {Release release-2026-11-10.1},
515
  year = {2026}
516
  }
517
  ```
@@ -593,7 +595,7 @@ Rows per class:
593
 
594
  ```sql
595
  SELECT hit_class, scope, count(*) AS rows
596
- FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.1/meta/bench_overlap.parquet'
597
  GROUP BY ALL ORDER BY hit_class, scope;
598
  ```
599
 
 
8
  Entry points: the data page https://huggingface.co/spaces/Battam/agentic-env-catalog (to share a state, use the page itself, https://battam-agentic-env-catalog.static.hf.space/index.html );
9
  the dataset https://huggingface.co/datasets/Battam/agentic-env-index .
10
 
11
+ The numbers and command outputs below were filled in at deploy time, from the index commit `83dae072040e8688c08c84fb72fdc8690d0956e5` (release tag `release-2026-11-10.2`).
12
 
13
  ## Tutorial
14
 
 
87
  ### 5. Go to "Get the data" and keep the selection
88
 
89
  **Do**: click "Get the data" at the top;
90
+ under "Keep this selection" click "Download the selection list (JSON)" and put `agentic-env-selection-83dae07.json` in your fetch folder.
91
 
92
  **See**: the heading "Take 3 units from each of 52 material sets", then "Units per source", the card "1 Take", "What it writes", the six kinds "Left out by default", and "2 Take it again from the file", which reads the list.
93
  The list records release, filters and sources:
 
96
  {
97
  "format": "agentic-env-selection/1",
98
  "dataset": "Battam/agentic-env-index",
99
+ "revision": "release-2026-11-10.2",
100
  "filters": {
101
  "collection": "A",
102
  "train": "yes",
 
120
  **Do**: in the list's folder, run the command of "Take it again from the file" with `--sample 3` replaced by `--limit 220` (220 units in all, in a fixed order: a margin over 200, since some sites fail) and `--out ./data` by `--out ./take`:
121
 
122
  ```sh
123
+ uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.2/tools/fetch.py \
124
+ --selection agentic-env-selection-83dae07.json \
125
+ --revision release-2026-11-10.2 \
126
  --ext xlsx \
127
  --ext xlsm \
128
  --ext xls \
 
147
  ok austender-ict-procurement-statistics workbooks/201314austenderictcontractstatistcs.xlsx: 26.3kB
148
  ok austender-ict-procurement-statistics workbooks/2016-17-austender-ict-contract-data-category-sme-and-small-participation.xlsx: 32.9kB
149
  ...
150
+ 317 files fetched and checked (96.4MB) into take; details in take/_meta/fetch_run.json
151
+ files of units: 5 failed, 215 ok; files of the sources themselves: 102 ok
152
+ run_id 20261110T233416.347081Z: 5 failed, 317 ok (lines in take/_meta/fetch_manifest.jsonl)
153
+ its lines that are not ok, in DuckDB: SELECT json FROM read_json_objects('take/_meta/fetch_manifest.jsonl') WHERE json_extract_string(json, 'run_id') = '20261110T233416.347081Z' AND json_extract_string(json, 'status') <> 'ok'
154
  ```
155
 
156
  Before the first download, `removed by` gives the units the default profile removed and the option that keeps them, counted in the data files read as far as `--limit` needed;
 
165
  incomplete: 1 unit miss a file (the files fetched are kept; _meta/fetch_manifest.jsonl says why): ...
166
  5 files fetched and checked (1.35MB) into probe; details in probe/_meta/fetch_run.json
167
  files of units: 1 failed, 1 ok; files of the sources themselves: 4 ok
168
+ run_id 20261110T233732.458275Z: 1 failed, 5 ok (lines in probe/_meta/fetch_manifest.jsonl)
169
+ its lines that are not ok, in DuckDB: SELECT json FROM read_json_objects('probe/_meta/fetch_manifest.jsonl') WHERE json_extract_string(json, 'run_id') = '20261110T233732.458275Z' AND json_extract_string(json, 'status') <> 'ok'
170
  ```
171
 
172
  **Get**: every `ok` file matched its recorded size and hash.
 
206
  โ”‚ excel_units โ”‚
207
  โ”‚ int64 โ”‚
208
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
209
+ โ”‚ 215 โ”‚
210
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
211
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
212
  โ”‚ verified โ”‚ files โ”‚ bytes โ”‚
213
  โ”‚ boolean โ”‚ int64 โ”‚ int128 โ”‚
214
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
215
+ โ”‚ true โ”‚ 317 โ”‚ 96365587 โ”‚
216
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
217
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
218
  โ”‚ source โ”‚ reason โ”‚ files โ”‚
219
  โ”‚ varchar โ”‚ varchar โ”‚ int64 โ”‚
220
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
221
+ โ”‚ scotland-government-workbooks โ”‚ size differs: expected 98409, got 0 โ”‚ 1 โ”‚
222
+ โ”‚ scotland-government-workbooks โ”‚ size differs: expected 74961, got 0 โ”‚ 1 โ”‚
223
  โ”‚ victoria-government-workbooks โ”‚ https://www.planning.vic.gov.au/__data/assets โ”‚ 2 โ”‚
224
  โ”‚ western-australia-government-workbooks โ”‚ https://catalogue.data.wa.gov.au/dataset/03b1 โ”‚ 1 โ”‚
225
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
 
241
  **See** (truncated):
242
 
243
  ```text
244
+ Dataset commit 83dae072040e8688c08c84fb72fdc8690d0956e5: https://huggingface.co/datasets/Battam/agentic-env-index/tree/83dae072040e8688c08c84fb72fdc8690d0956e5
245
  ...
246
  - Source: `austender-ict-procurement-statistics`
247
  - License: CC-BY-3.0-AU
 
256
  ## How-to guides
257
 
258
  One goal each;
259
+ commands are pinned to `release-2026-11-10.2`.
260
 
261
  ### Filter by file type and license
262
 
 
286
 
287
  ### Pin a release
288
 
289
+ The page's commands pin the script URL and `--revision` to `release-2026-11-10.2`;
290
  the script's first line names the commit:
291
 
292
  ```text
293
+ reading hf://datasets/Battam/agentic-env-index@83dae072040e8688c08c84fb72fdc8690d0956e5 (commit 83dae072040e8688c08c84fb72fdc8690d0956e5); pass --revision 83dae072040e8688c08c84fb72fdc8690d0956e5 to select the same rows later
294
  ```
295
 
296
  Later, `--revision <commit>`, the same tag or the selection list take the same files;
 
299
  A misspelt source name stops with exit code 2:
300
 
301
  ```text
302
+ error: --source dosfstools-fsk: no such source in meta/catalog.parquet at commit 83dae072040e8688c08c84fb72fdc8690d0956e5; closest: dosfstools-fsck-cases
303
  ```
304
 
305
  ### Take only part
 
333
 
334
  ```sql
335
  SELECT source, collection, title, units, files, top_types, license.tier AS license_tier, needs_auth
336
+ FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.2/meta/catalog.parquet'
337
  WHERE kind = 'material_set' AND needs_auth = 'no' AND license.tier IN ('permissive', 'share-alike')
338
  ORDER BY collection, units DESC;
339
  ```
 
352
 
353
  ```sql
354
  SELECT string_agg('--source ' || source, ' ' ORDER BY source) AS options
355
+ FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.2/meta/catalog.parquet'
356
  WHERE kind = 'material_set' AND collection = 'B'
357
  AND license.tier = 'share-alike';
358
  ```
 
361
  --source adflib-adf-deleted --source adflib-adf-malformed --source btrfs-progs-convert-needs-recovery --source btrfs-progs-fsck-raw-images --source btrfs-progs-fuzz-images --source dfir-ntfs-fat-deleted --source dftt-deleted-files --source dftt-partition-tables --source dissect-hypervisor-disk-fixtures --source dosfstools-fsck-cases --source e2fsprogs-fsck-cases --source exfatprogs-fsck-cases --source libcdio-iso-malformed --source nist-cfreds-dfr-exfat --source ntfs-corrupted-images --source oe-ext4-hostile-fixtures --source qemu-iotests-sample-images
362
  ```
363
 
364
+ Put them after `fetch.py --revision release-2026-11-10.2`.
365
  Read only the catalog, not all of `meta/`.
366
 
367
  ### Take disk images and check them
 
373
  !["Get the data" after picking three disk image material sets](S09-en.png)
374
 
375
  ```sh
376
+ uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.2/tools/fetch.py \
377
+ --revision release-2026-11-10.2 \
378
  --source dosfstools-fsck-cases \
379
  --source e2fsprogs-fsck-cases \
380
  --source exfatprogs-fsck-cases \
 
390
  ok exfatprogs-fsck-cases cases/bad_dentries/exfat.img: 5.24MB
391
  22 files fetched and checked (15.9MB) into data; details in data/_meta/fetch_run.json
392
  files of units: 15 ok; files of the sources themselves: 7 ok
393
+ run_id 20261110T233737.867574Z: 22 ok (lines in data/_meta/fetch_manifest.jsonl)
394
  ```
395
 
396
  An exFAT image is `exfat.img.tar.xz` upstream;
 
431
 
432
  ```sql
433
  SELECT contamination.level AS level, count(*) AS material_sets, sum(units) AS units
434
+ FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.2/meta/catalog.parquet'
435
  WHERE kind = 'material_set'
436
  AND list_has_any(top_types, ['docx', 'xlsx', 'pptx', 'pdf'])
437
  GROUP BY ALL;
 
451
  Two, take the picked sources and read `removed by benchmark` and `removed by not compared`, printed before the first download (output truncated):
452
 
453
  ```sh
454
+ uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.2/tools/fetch.py \
455
+ --revision release-2026-11-10.2 \
456
  --source python-docx-acceptance-test-files \
457
  --source python-pptx-acceptance-test-files \
458
  --source docling-docx-groundtruth \
 
481
  ```sql
482
  SELECT o.bench, o.hit_class, count(*) AS files
483
  FROM read_json_auto('office3/_meta/fetch_manifest.jsonl') AS f
484
+ JOIN 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.2/meta/bench_overlap.parquet' AS o
485
  ON o.source = f.source AND o.uri = f.uri
486
  WHERE f.status = 'ok'
487
  GROUP BY ALL;
 
513
  @misc{agentic_env_index,
514
  title = {agentic-env-index: an index of public materials for building agent environments},
515
  howpublished = {https://huggingface.co/datasets/Battam/agentic-env-index},
516
+ note = {Release release-2026-11-10.2},
517
  year = {2026}
518
  }
519
  ```
 
595
 
596
  ```sql
597
  SELECT hit_class, scope, count(*) AS rows
598
+ FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.2/meta/bench_overlap.parquet'
599
  GROUP BY ALL ORDER BY hit_class, scope;
600
  ```
601
 
manual/manual.zh.md CHANGED
@@ -8,7 +8,7 @@
8
  ๅ…ฅๅฃ๏ผšๆ•ฐๆฎ้กต https://huggingface.co/spaces/Battam/agentic-env-catalog ๏ผˆๅธฆๅ‚ๆ•ฐๅˆ†ไบซๆ—ถ็”จ้กต้ขๆœฌไฝ“ https://battam-agentic-env-catalog.static.hf.space/index.html ๏ผ‰๏ผ›
9
  ๆ•ฐๆฎ้›† https://huggingface.co/datasets/Battam/agentic-env-index ใ€‚
10
 
11
- ไธ‹้ข็š„ๆ•ฐๅญ—ๅ’Œๅ‘ฝไปค่พ“ๅ‡บๆ˜ฏ้ƒจ็ฝฒๆ—ถๅกซๅ…ฅ็š„๏ผŒๅ–่‡ช็ดขๅผ•ๆไบค `5b9f5a4d2bde4bee82f6a0b178b5ab088603564f`๏ผˆๅ‘ๅธƒๆ ‡็ญพ `release-2026-11-10.1`๏ผ‰ใ€‚
12
 
13
  ## ๆ•™็จ‹
14
 
@@ -83,7 +83,7 @@
83
 
84
  ### 5. ๅˆฐใ€Œ่Žทๅ–ๆ•ฐๆฎใ€๏ผŒไฟๅญ˜้€‰ๆ‹ฉๆธ…ๅ•
85
 
86
- **ๅš**๏ผš็‚น้กถ้ƒจใ€Œ่Žทๅ–ๆ•ฐๆฎใ€๏ผŒๅœจใ€Œไฟๅญ˜่ฟ™ๆฌก้€‰ๆ‹ฉใ€้‡Œ็‚นใ€Œไธ‹่ฝฝ้€‰ๆ‹ฉๆธ…ๅ•๏ผˆJSON๏ผ‰ใ€๏ผŒๆŠŠ `agentic-env-selection-5b9f5a4.json` ๆ”พ่ฟ›ๅ–ไปถ็›ฎๅฝ•ใ€‚
87
 
88
  **็œ‹ๅˆฐ**๏ผšๆ ‡้ข˜ใ€ŒไปŽ 52 ไธชๆๆ–™้›†ไธญๅ„่Žทๅ– 3 ไธชๅ•ๅ…ƒใ€๏ผŒไธ‹้ขๆ˜ฏใ€Œๆฏไธชๆฅๆบๅ–ใ€ใ€ๅ‘ฝไปคๅกใ€Œ1 ่Žทๅ–ใ€ใ€ใ€Œๅ†™ๅ…ฅๅ†…ๅฎนใ€ใ€ใ€Œ้ป˜่ฎคๆŽ’้™คใ€ๅ…ญ็ฑปๅ’Œ่ฏป้€‰ๆ‹ฉๆธ…ๅ•็š„ใ€Œ2 ๆŒ‰ๆ–‡ไปถๅ†ๅ–ไธ€ๆฌกใ€ใ€‚
89
  ๆธ…ๅ•่ฎฐ็€็‰ˆๆœฌใ€็ญ›้€‰ๅ’Œๆฅๆบ๏ผš
@@ -92,7 +92,7 @@
92
  {
93
  "format": "agentic-env-selection/1",
94
  "dataset": "Battam/agentic-env-index",
95
- "revision": "release-2026-11-10.1",
96
  "filters": {
97
  "collection": "A",
98
  "train": "yes",
@@ -116,9 +116,9 @@
116
  **ๅš**๏ผšๅœจๆธ…ๅ•ๆ‰€ๅœจ็›ฎๅฝ•่ท‘ใ€ŒๆŒ‰ๆ–‡ไปถๅ†ๅ–ไธ€ๆฌกใ€้‚ฃๆกๅ‘ฝไปค๏ผŒ`--sample 3` ๆขๆˆ `--limit 220`๏ผˆไธ€ๅ…ฑ 220 ไธชๅ•ๅ…ƒ๏ผŒๅ›บๅฎš้กบๅบ๏ผ›ๆฏ” 200 ๅคšๅ–ไธ€ไบ›็•™ไฝ™้‡๏ผŒๆœ‰็š„ๅŽŸ็ซ™ไผšๅคฑๆ•ˆ๏ผ‰๏ผŒ`--out ./data` ๆขๆˆ `--out ./take`๏ผš
117
 
118
  ```sh
119
- uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.1/tools/fetch.py \
120
- --selection agentic-env-selection-5b9f5a4.json \
121
- --revision release-2026-11-10.1 \
122
  --ext xlsx \
123
  --ext xlsm \
124
  --ext xls \
@@ -143,10 +143,10 @@ ok abhishekumarr00-supply-chain-mastery workbooks/abhishekumarr00-supply-chain-m
143
  ok austender-ict-procurement-statistics workbooks/201314austenderictcontractstatistcs.xlsx: 26.3kB
144
  ok austender-ict-procurement-statistics workbooks/2016-17-austender-ict-contract-data-category-sme-and-small-participation.xlsx: 32.9kB
145
  ...
146
- 319 files fetched and checked (96.5MB) into take; details in take/_meta/fetch_run.json
147
- files of units: 3 failed, 217 ok; files of the sources themselves: 102 ok
148
- run_id 20261110T204854.721151Z: 3 failed, 319 ok (lines in take/_meta/fetch_manifest.jsonl)
149
- its lines that are not ok, in DuckDB: SELECT json FROM read_json_objects('take/_meta/fetch_manifest.jsonl') WHERE json_extract_string(json, 'run_id') = '20261110T204854.721151Z' AND json_extract_string(json, 'status') <> 'ok'
150
  ```
151
 
152
  ็ฌฌไธ€ๆฌกไธ‹่ฝฝไน‹ๅ‰๏ผŒ`removed by` ๆ˜ฏ้ป˜่ฎค้…็ฝฎๅŽปๆމ็š„ๅ•ๅ…ƒๅ’Œไฟ็•™ๅฎƒไปฌ็š„ๅผ€ๅ…ณ๏ผŒๆ•ฐ็š„ๆ˜ฏ `--limit` ้œ€่ฆ่ฏปๅˆฐ็š„้‚ฃไบ›ๆ•ฐๆฎๆ–‡ไปถ๏ผ›
@@ -161,8 +161,8 @@ failed western-australia-government-workbooks workbooks/wa-gross-regional-produc
161
  incomplete: 1 unit miss a file (the files fetched are kept; _meta/fetch_manifest.jsonl says why): ...
162
  5 files fetched and checked (1.35MB) into probe; details in probe/_meta/fetch_run.json
163
  files of units: 1 failed, 1 ok; files of the sources themselves: 4 ok
164
- run_id 20261110T205223.232212Z: 1 failed, 5 ok (lines in probe/_meta/fetch_manifest.jsonl)
165
- its lines that are not ok, in DuckDB: SELECT json FROM read_json_objects('probe/_meta/fetch_manifest.jsonl') WHERE json_extract_string(json, 'run_id') = '20261110T205223.232212Z' AND json_extract_string(json, 'status') <> 'ok'
166
  ```
167
 
168
  **ๅพ—ๅˆฐ**๏ผš`ok` ็š„ๆ–‡ไปถๅทฒๆŒ‰่ฎฐๅฝ•็š„ๅคงๅฐๅ’Œๅ“ˆๅธŒๆ ธๅฏนใ€‚
@@ -201,18 +201,20 @@ GROUP BY ALL ORDER BY source;
201
  โ”‚ excel_units โ”‚
202
  โ”‚ int64 โ”‚
203
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
204
- โ”‚ 217 โ”‚
205
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
206
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€๏ฟฝ๏ฟฝ๏ฟฝโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
207
  โ”‚ verified โ”‚ files โ”‚ bytes โ”‚
208
  โ”‚ boolean โ”‚ int64 โ”‚ int128 โ”‚
209
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
210
- โ”‚ true โ”‚ 319 โ”‚ 96538957 โ”‚
211
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
212
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
213
  โ”‚ source โ”‚ reason โ”‚ files โ”‚
214
  โ”‚ varchar โ”‚ varchar โ”‚ int64 โ”‚
215
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
 
 
216
  โ”‚ victoria-government-workbooks โ”‚ https://www.planning.vic.gov.au/__data/assets โ”‚ 2 โ”‚
217
  โ”‚ western-australia-government-workbooks โ”‚ https://catalogue.data.wa.gov.au/dataset/03b1 โ”‚ 1 โ”‚
218
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
@@ -233,7 +235,7 @@ GROUP BY ALL ORDER BY source;
233
  **็œ‹ๅˆฐ**๏ผˆๆˆช็Ÿญ๏ผ‰๏ผš
234
 
235
  ```text
236
- Dataset commit 5b9f5a4d2bde4bee82f6a0b178b5ab088603564f: https://huggingface.co/datasets/Battam/agentic-env-index/tree/5b9f5a4d2bde4bee82f6a0b178b5ab088603564f
237
  ...
238
  - Source: `austender-ict-procurement-statistics`
239
  - License: CC-BY-3.0-AU
@@ -247,7 +249,7 @@ Dataset commit 5b9f5a4d2bde4bee82f6a0b178b5ab088603564f: https://huggingface.co/
247
 
248
  ## ๆ“ไฝœๆŒ‡ๅ—
249
 
250
- ๆฏๆกไธ€ไธช็›ฎๆ ‡๏ผŒๅ‘ฝไปค้ƒฝ้’‰ๅœจ `release-2026-11-10.1`ใ€‚
251
 
252
  ### ๆŒ‰ๆ–‡ไปถ็ฑปๅž‹ไธŽ่ฎธๅฏ็ญ›
253
 
@@ -275,10 +277,10 @@ DuckDB ้‡Œ็”จ็›ฎๅฝ•่กจ็š„ `license.tier` ๅ’Œ `top_types`๏ผˆๆœ€ๅธธ่ง็š„ๆ‰ฉๅฑ•ๅ
275
 
276
  ### ้’‰ไฝๅ‘ๅธƒ็‰ˆๆœฌๅ–
277
 
278
- ้กต้ข็ป™็š„ๅ‘ฝไปคๆŠŠ่„šๆœฌ็ฝ‘ๅ€ๅ’Œ `--revision` ้ƒฝ้’‰ๅœจ `release-2026-11-10.1`๏ผŒ่„šๆœฌ้ฆ–่กŒๆ‰“ๅฐๅฏนๅบ”็š„ๆไบค๏ผš
279
 
280
  ```text
281
- reading hf://datasets/Battam/agentic-env-index@5b9f5a4d2bde4bee82f6a0b178b5ab088603564f (commit 5b9f5a4d2bde4bee82f6a0b178b5ab088603564f); pass --revision 5b9f5a4d2bde4bee82f6a0b178b5ab088603564f to select the same rows later
282
  ```
283
 
284
  ๆ—ฅๅŽ็”จ `--revision <ๆไบคๅท>`ใ€ๅŒไธ€ไธชๆ ‡็ญพๆˆ–้€‰ๆ‹ฉๆธ…ๅ•๏ผŒๅ–ๅˆฐ็š„ๅฐฑๆ˜ฏๅŒๆ ท็š„ๆ–‡ไปถ๏ผ›
@@ -287,7 +289,7 @@ reading hf://datasets/Battam/agentic-env-index@5b9f5a4d2bde4bee82f6a0b178b5ab088
287
  ๆฅๆบๅๅ†™้”™ๆ—ถ้€€ๅ‡บ็  2๏ผš
288
 
289
  ```text
290
- error: --source dosfstools-fsk: no such source in meta/catalog.parquet at commit 5b9f5a4d2bde4bee82f6a0b178b5ab088603564f; closest: dosfstools-fsck-cases
291
  ```
292
 
293
  ### ๅชๅ–ไธ€้ƒจๅˆ†
@@ -319,7 +321,7 @@ error: --source dosfstools-fsk: no such source in meta/catalog.parquet at commit
319
 
320
  ```sql
321
  SELECT source, collection, title, units, files, top_types, license.tier AS license_tier, needs_auth
322
- FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.1/meta/catalog.parquet'
323
  WHERE kind = 'material_set' AND needs_auth = 'no' AND license.tier IN ('permissive', 'share-alike')
324
  ORDER BY collection, units DESC;
325
  ```
@@ -338,7 +340,7 @@ ORDER BY collection, units DESC;
338
 
339
  ```sql
340
  SELECT string_agg('--source ' || source, ' ' ORDER BY source) AS options
341
- FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.1/meta/catalog.parquet'
342
  WHERE kind = 'material_set' AND collection = 'B'
343
  AND license.tier = 'share-alike';
344
  ```
@@ -347,7 +349,7 @@ WHERE kind = 'material_set' AND collection = 'B'
347
  --source adflib-adf-deleted --source adflib-adf-malformed --source btrfs-progs-convert-needs-recovery --source btrfs-progs-fsck-raw-images --source btrfs-progs-fuzz-images --source dfir-ntfs-fat-deleted --source dftt-deleted-files --source dftt-partition-tables --source dissect-hypervisor-disk-fixtures --source dosfstools-fsck-cases --source e2fsprogs-fsck-cases --source exfatprogs-fsck-cases --source libcdio-iso-malformed --source nist-cfreds-dfr-exfat --source ntfs-corrupted-images --source oe-ext4-hostile-fixtures --source qemu-iotests-sample-images
348
  ```
349
 
350
- ๆŽฅๅœจ `fetch.py --revision release-2026-11-10.1` ๅŽ้ขใ€‚
351
  ๅช่ฏป็›ฎๅฝ•่กจ๏ผŒๅˆซๆ•ดไธชไธ‹่ฝฝ `meta/`ใ€‚
352
 
353
  ### ๅ–็ฃ็›˜้•œๅƒ๏ผŒๅ–ๅ›žๅŽๆ ธๅฏนๆฒกๅ
@@ -359,8 +361,8 @@ WHERE kind = 'material_set' AND collection = 'B'
359
  ![ๆŒ‘้€‰ไธ‰ไธช็ฃ็›˜้•œๅƒๆๆ–™้›†ๅŽ็š„ใ€Œ่Žทๅ–ๆ•ฐๆฎใ€](S09-zh.png)
360
 
361
  ```sh
362
- uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.1/tools/fetch.py \
363
- --revision release-2026-11-10.1 \
364
  --source dosfstools-fsck-cases \
365
  --source e2fsprogs-fsck-cases \
366
  --source exfatprogs-fsck-cases \
@@ -376,7 +378,7 @@ ok dosfstools-fsck-cases README.md: 3.68kB
376
  ok exfatprogs-fsck-cases cases/bad_dentries/exfat.img: 5.24MB
377
  22 files fetched and checked (15.9MB) into data; details in data/_meta/fetch_run.json
378
  files of units: 15 ok; files of the sources themselves: 7 ok
379
- run_id 20261110T205228.568116Z: 22 ok (lines in data/_meta/fetch_manifest.jsonl)
380
  ```
381
 
382
  exFAT ้•œๅƒๅœจๅŽŸ็ซ™ๆ˜ฏ `exfat.img.tar.xz`๏ผŒ่„šๆœฌ่งฃๅ‡บๅŽๅ†ๆŒ‰ๅ•ๅ…ƒ่‡ชๅทฑ็š„ SHA-256 ๆ ธๅฏน๏ผŒ็›˜ไธŠๅช็•™้•œๅƒใ€‚
@@ -416,7 +418,7 @@ Pass 1: Checking inodes, blocks, and sizes
416
 
417
  ```sql
418
  SELECT contamination.level AS level, count(*) AS material_sets, sum(units) AS units
419
- FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.1/meta/catalog.parquet'
420
  WHERE kind = 'material_set'
421
  AND list_has_any(top_types, ['docx', 'xlsx', 'pptx', 'pdf'])
422
  GROUP BY ALL;
@@ -436,8 +438,8 @@ GROUP BY ALL;
436
  ไบŒ๏ผŒๅ–ๅ›žๆŒ‘ไธญ็š„ๆฅๆบ๏ผŒ็œ‹็ฌฌไธ€ๆฌกไธ‹่ฝฝไน‹ๅ‰ๆ‰“ๅฐ็š„ `removed by benchmark` ๅ’Œ `removed by not compared`๏ผˆ่พ“ๅ‡บๆˆช็Ÿญ๏ผ‰๏ผš
437
 
438
  ```sh
439
- uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.1/tools/fetch.py \
440
- --revision release-2026-11-10.1 \
441
  --source python-docx-acceptance-test-files \
442
  --source python-pptx-acceptance-test-files \
443
  --source docling-docx-groundtruth \
@@ -465,7 +467,7 @@ fetching: 41 units: 45 files (4 files of the sources themselves among them), 25.
465
  ```sql
466
  SELECT o.bench, o.hit_class, count(*) AS files
467
  FROM read_json_auto('office3/_meta/fetch_manifest.jsonl') AS f
468
- JOIN 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.1/meta/bench_overlap.parquet' AS o
469
  ON o.source = f.source AND o.uri = f.uri
470
  WHERE f.status = 'ok'
471
  GROUP BY ALL;
@@ -497,7 +499,7 @@ GDPval ๅœจๆฏ”ๅฏนๆธ…ๅ• `meta/benchmarks.parquet` ้‡Œ๏ผˆ220 ้“้ข˜๏ผ‰๏ผŒ่ฟ™ไธ€็‰ˆ
497
  @misc{agentic_env_index,
498
  title = {agentic-env-index: an index of public materials for building agent environments},
499
  howpublished = {https://huggingface.co/datasets/Battam/agentic-env-index},
500
- note = {Release release-2026-11-10.1},
501
  year = {2026}
502
  }
503
  ```
@@ -579,7 +581,7 @@ scope ไธบ `container` ๆ—ถๅ‘ฝไธญๆ•ดไธชๅˆ†็ป„๏ผˆๅชๅœจๅˆ†็ป„ๆ˜ฏไธ€ไธชๅฎŒๆ•ดๅ•ไฝ
579
 
580
  ```sql
581
  SELECT hit_class, scope, count(*) AS rows
582
- FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.1/meta/bench_overlap.parquet'
583
  GROUP BY ALL ORDER BY hit_class, scope;
584
  ```
585
 
 
8
  ๅ…ฅๅฃ๏ผšๆ•ฐๆฎ้กต https://huggingface.co/spaces/Battam/agentic-env-catalog ๏ผˆๅธฆๅ‚ๆ•ฐๅˆ†ไบซๆ—ถ็”จ้กต้ขๆœฌไฝ“ https://battam-agentic-env-catalog.static.hf.space/index.html ๏ผ‰๏ผ›
9
  ๆ•ฐๆฎ้›† https://huggingface.co/datasets/Battam/agentic-env-index ใ€‚
10
 
11
+ ไธ‹้ข็š„ๆ•ฐๅญ—ๅ’Œๅ‘ฝไปค่พ“ๅ‡บๆ˜ฏ้ƒจ็ฝฒๆ—ถๅกซๅ…ฅ็š„๏ผŒๅ–่‡ช็ดขๅผ•ๆไบค `83dae072040e8688c08c84fb72fdc8690d0956e5`๏ผˆๅ‘ๅธƒๆ ‡็ญพ `release-2026-11-10.2`๏ผ‰ใ€‚
12
 
13
  ## ๆ•™็จ‹
14
 
 
83
 
84
  ### 5. ๅˆฐใ€Œ่Žทๅ–ๆ•ฐๆฎใ€๏ผŒไฟๅญ˜้€‰ๆ‹ฉๆธ…ๅ•
85
 
86
+ **ๅš**๏ผš็‚น้กถ้ƒจใ€Œ่Žทๅ–ๆ•ฐๆฎใ€๏ผŒๅœจใ€Œไฟๅญ˜่ฟ™ๆฌก้€‰ๆ‹ฉใ€้‡Œ็‚นใ€Œไธ‹่ฝฝ้€‰ๆ‹ฉๆธ…ๅ•๏ผˆJSON๏ผ‰ใ€๏ผŒๆŠŠ `agentic-env-selection-83dae07.json` ๆ”พ่ฟ›ๅ–ไปถ็›ฎๅฝ•ใ€‚
87
 
88
  **็œ‹ๅˆฐ**๏ผšๆ ‡้ข˜ใ€ŒไปŽ 52 ไธชๆๆ–™้›†ไธญๅ„่Žทๅ– 3 ไธชๅ•ๅ…ƒใ€๏ผŒไธ‹้ขๆ˜ฏใ€Œๆฏไธชๆฅๆบๅ–ใ€ใ€ๅ‘ฝไปคๅกใ€Œ1 ่Žทๅ–ใ€ใ€ใ€Œๅ†™ๅ…ฅๅ†…ๅฎนใ€ใ€ใ€Œ้ป˜่ฎคๆŽ’้™คใ€ๅ…ญ็ฑปๅ’Œ่ฏป้€‰ๆ‹ฉๆธ…ๅ•็š„ใ€Œ2 ๆŒ‰ๆ–‡ไปถๅ†ๅ–ไธ€ๆฌกใ€ใ€‚
89
  ๆธ…ๅ•่ฎฐ็€็‰ˆๆœฌใ€็ญ›้€‰ๅ’Œๆฅๆบ๏ผš
 
92
  {
93
  "format": "agentic-env-selection/1",
94
  "dataset": "Battam/agentic-env-index",
95
+ "revision": "release-2026-11-10.2",
96
  "filters": {
97
  "collection": "A",
98
  "train": "yes",
 
116
  **ๅš**๏ผšๅœจๆธ…ๅ•ๆ‰€ๅœจ็›ฎๅฝ•่ท‘ใ€ŒๆŒ‰ๆ–‡ไปถๅ†ๅ–ไธ€ๆฌกใ€้‚ฃๆกๅ‘ฝไปค๏ผŒ`--sample 3` ๆขๆˆ `--limit 220`๏ผˆไธ€ๅ…ฑ 220 ไธชๅ•ๅ…ƒ๏ผŒๅ›บๅฎš้กบๅบ๏ผ›ๆฏ” 200 ๅคšๅ–ไธ€ไบ›็•™ไฝ™้‡๏ผŒๆœ‰็š„ๅŽŸ็ซ™ไผšๅคฑๆ•ˆ๏ผ‰๏ผŒ`--out ./data` ๆขๆˆ `--out ./take`๏ผš
117
 
118
  ```sh
119
+ uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.2/tools/fetch.py \
120
+ --selection agentic-env-selection-83dae07.json \
121
+ --revision release-2026-11-10.2 \
122
  --ext xlsx \
123
  --ext xlsm \
124
  --ext xls \
 
143
  ok austender-ict-procurement-statistics workbooks/201314austenderictcontractstatistcs.xlsx: 26.3kB
144
  ok austender-ict-procurement-statistics workbooks/2016-17-austender-ict-contract-data-category-sme-and-small-participation.xlsx: 32.9kB
145
  ...
146
+ 317 files fetched and checked (96.4MB) into take; details in take/_meta/fetch_run.json
147
+ files of units: 5 failed, 215 ok; files of the sources themselves: 102 ok
148
+ run_id 20261110T233416.347081Z: 5 failed, 317 ok (lines in take/_meta/fetch_manifest.jsonl)
149
+ its lines that are not ok, in DuckDB: SELECT json FROM read_json_objects('take/_meta/fetch_manifest.jsonl') WHERE json_extract_string(json, 'run_id') = '20261110T233416.347081Z' AND json_extract_string(json, 'status') <> 'ok'
150
  ```
151
 
152
  ็ฌฌไธ€ๆฌกไธ‹่ฝฝไน‹ๅ‰๏ผŒ`removed by` ๆ˜ฏ้ป˜่ฎค้…็ฝฎๅŽปๆމ็š„ๅ•ๅ…ƒๅ’Œไฟ็•™ๅฎƒไปฌ็š„ๅผ€ๅ…ณ๏ผŒๆ•ฐ็š„ๆ˜ฏ `--limit` ้œ€่ฆ่ฏปๅˆฐ็š„้‚ฃไบ›ๆ•ฐๆฎๆ–‡ไปถ๏ผ›
 
161
  incomplete: 1 unit miss a file (the files fetched are kept; _meta/fetch_manifest.jsonl says why): ...
162
  5 files fetched and checked (1.35MB) into probe; details in probe/_meta/fetch_run.json
163
  files of units: 1 failed, 1 ok; files of the sources themselves: 4 ok
164
+ run_id 20261110T233732.458275Z: 1 failed, 5 ok (lines in probe/_meta/fetch_manifest.jsonl)
165
+ its lines that are not ok, in DuckDB: SELECT json FROM read_json_objects('probe/_meta/fetch_manifest.jsonl') WHERE json_extract_string(json, 'run_id') = '20261110T233732.458275Z' AND json_extract_string(json, 'status') <> 'ok'
166
  ```
167
 
168
  **ๅพ—ๅˆฐ**๏ผš`ok` ็š„ๆ–‡ไปถๅทฒๆŒ‰่ฎฐๅฝ•็š„ๅคงๅฐๅ’Œๅ“ˆๅธŒๆ ธๅฏนใ€‚
 
201
  โ”‚ excel_units โ”‚
202
  โ”‚ int64 โ”‚
203
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
204
+ โ”‚ 215 โ”‚
205
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
206
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€๏ฟฝ๏ฟฝ๏ฟฝโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
207
  โ”‚ verified โ”‚ files โ”‚ bytes โ”‚
208
  โ”‚ boolean โ”‚ int64 โ”‚ int128 โ”‚
209
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
210
+ โ”‚ true โ”‚ 317 โ”‚ 96365587 โ”‚
211
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
212
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
213
  โ”‚ source โ”‚ reason โ”‚ files โ”‚
214
  โ”‚ varchar โ”‚ varchar โ”‚ int64 โ”‚
215
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
216
+ โ”‚ scotland-government-workbooks โ”‚ size differs: expected 98409, got 0 โ”‚ 1 โ”‚
217
+ โ”‚ scotland-government-workbooks โ”‚ size differs: expected 74961, got 0 โ”‚ 1 โ”‚
218
  โ”‚ victoria-government-workbooks โ”‚ https://www.planning.vic.gov.au/__data/assets โ”‚ 2 โ”‚
219
  โ”‚ western-australia-government-workbooks โ”‚ https://catalogue.data.wa.gov.au/dataset/03b1 โ”‚ 1 โ”‚
220
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
 
235
  **็œ‹ๅˆฐ**๏ผˆๆˆช็Ÿญ๏ผ‰๏ผš
236
 
237
  ```text
238
+ Dataset commit 83dae072040e8688c08c84fb72fdc8690d0956e5: https://huggingface.co/datasets/Battam/agentic-env-index/tree/83dae072040e8688c08c84fb72fdc8690d0956e5
239
  ...
240
  - Source: `austender-ict-procurement-statistics`
241
  - License: CC-BY-3.0-AU
 
249
 
250
  ## ๆ“ไฝœๆŒ‡ๅ—
251
 
252
+ ๆฏๆกไธ€ไธช็›ฎๆ ‡๏ผŒๅ‘ฝไปค้ƒฝ้’‰ๅœจ `release-2026-11-10.2`ใ€‚
253
 
254
  ### ๆŒ‰ๆ–‡ไปถ็ฑปๅž‹ไธŽ่ฎธๅฏ็ญ›
255
 
 
277
 
278
  ### ้’‰ไฝๅ‘ๅธƒ็‰ˆๆœฌๅ–
279
 
280
+ ้กต้ข็ป™็š„ๅ‘ฝไปคๆŠŠ่„šๆœฌ็ฝ‘ๅ€ๅ’Œ `--revision` ้ƒฝ้’‰ๅœจ `release-2026-11-10.2`๏ผŒ่„šๆœฌ้ฆ–่กŒๆ‰“ๅฐๅฏนๅบ”็š„ๆไบค๏ผš
281
 
282
  ```text
283
+ reading hf://datasets/Battam/agentic-env-index@83dae072040e8688c08c84fb72fdc8690d0956e5 (commit 83dae072040e8688c08c84fb72fdc8690d0956e5); pass --revision 83dae072040e8688c08c84fb72fdc8690d0956e5 to select the same rows later
284
  ```
285
 
286
  ๆ—ฅๅŽ็”จ `--revision <ๆไบคๅท>`ใ€ๅŒไธ€ไธชๆ ‡็ญพๆˆ–้€‰ๆ‹ฉๆธ…ๅ•๏ผŒๅ–ๅˆฐ็š„ๅฐฑๆ˜ฏๅŒๆ ท็š„ๆ–‡ไปถ๏ผ›
 
289
  ๆฅๆบๅๅ†™้”™ๆ—ถ้€€ๅ‡บ็  2๏ผš
290
 
291
  ```text
292
+ error: --source dosfstools-fsk: no such source in meta/catalog.parquet at commit 83dae072040e8688c08c84fb72fdc8690d0956e5; closest: dosfstools-fsck-cases
293
  ```
294
 
295
  ### ๅชๅ–ไธ€้ƒจๅˆ†
 
321
 
322
  ```sql
323
  SELECT source, collection, title, units, files, top_types, license.tier AS license_tier, needs_auth
324
+ FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.2/meta/catalog.parquet'
325
  WHERE kind = 'material_set' AND needs_auth = 'no' AND license.tier IN ('permissive', 'share-alike')
326
  ORDER BY collection, units DESC;
327
  ```
 
340
 
341
  ```sql
342
  SELECT string_agg('--source ' || source, ' ' ORDER BY source) AS options
343
+ FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.2/meta/catalog.parquet'
344
  WHERE kind = 'material_set' AND collection = 'B'
345
  AND license.tier = 'share-alike';
346
  ```
 
349
  --source adflib-adf-deleted --source adflib-adf-malformed --source btrfs-progs-convert-needs-recovery --source btrfs-progs-fsck-raw-images --source btrfs-progs-fuzz-images --source dfir-ntfs-fat-deleted --source dftt-deleted-files --source dftt-partition-tables --source dissect-hypervisor-disk-fixtures --source dosfstools-fsck-cases --source e2fsprogs-fsck-cases --source exfatprogs-fsck-cases --source libcdio-iso-malformed --source nist-cfreds-dfr-exfat --source ntfs-corrupted-images --source oe-ext4-hostile-fixtures --source qemu-iotests-sample-images
350
  ```
351
 
352
+ ๆŽฅๅœจ `fetch.py --revision release-2026-11-10.2` ๅŽ้ขใ€‚
353
  ๅช่ฏป็›ฎๅฝ•่กจ๏ผŒๅˆซๆ•ดไธชไธ‹่ฝฝ `meta/`ใ€‚
354
 
355
  ### ๅ–็ฃ็›˜้•œๅƒ๏ผŒๅ–ๅ›žๅŽๆ ธๅฏนๆฒกๅ
 
361
  ![ๆŒ‘้€‰ไธ‰ไธช็ฃ็›˜้•œๅƒๆๆ–™้›†ๅŽ็š„ใ€Œ่Žทๅ–ๆ•ฐๆฎใ€](S09-zh.png)
362
 
363
  ```sh
364
+ uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.2/tools/fetch.py \
365
+ --revision release-2026-11-10.2 \
366
  --source dosfstools-fsck-cases \
367
  --source e2fsprogs-fsck-cases \
368
  --source exfatprogs-fsck-cases \
 
378
  ok exfatprogs-fsck-cases cases/bad_dentries/exfat.img: 5.24MB
379
  22 files fetched and checked (15.9MB) into data; details in data/_meta/fetch_run.json
380
  files of units: 15 ok; files of the sources themselves: 7 ok
381
+ run_id 20261110T233737.867574Z: 22 ok (lines in data/_meta/fetch_manifest.jsonl)
382
  ```
383
 
384
  exFAT ้•œๅƒๅœจๅŽŸ็ซ™ๆ˜ฏ `exfat.img.tar.xz`๏ผŒ่„šๆœฌ่งฃๅ‡บๅŽๅ†ๆŒ‰ๅ•ๅ…ƒ่‡ชๅทฑ็š„ SHA-256 ๆ ธๅฏน๏ผŒ็›˜ไธŠๅช็•™้•œๅƒใ€‚
 
418
 
419
  ```sql
420
  SELECT contamination.level AS level, count(*) AS material_sets, sum(units) AS units
421
+ FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.2/meta/catalog.parquet'
422
  WHERE kind = 'material_set'
423
  AND list_has_any(top_types, ['docx', 'xlsx', 'pptx', 'pdf'])
424
  GROUP BY ALL;
 
438
  ไบŒ๏ผŒๅ–ๅ›žๆŒ‘ไธญ็š„ๆฅๆบ๏ผŒ็œ‹็ฌฌไธ€ๆฌกไธ‹่ฝฝไน‹ๅ‰ๆ‰“ๅฐ็š„ `removed by benchmark` ๅ’Œ `removed by not compared`๏ผˆ่พ“ๅ‡บๆˆช็Ÿญ๏ผ‰๏ผš
439
 
440
  ```sh
441
+ uv run https://huggingface.co/datasets/Battam/agentic-env-index/resolve/release-2026-11-10.2/tools/fetch.py \
442
+ --revision release-2026-11-10.2 \
443
  --source python-docx-acceptance-test-files \
444
  --source python-pptx-acceptance-test-files \
445
  --source docling-docx-groundtruth \
 
467
  ```sql
468
  SELECT o.bench, o.hit_class, count(*) AS files
469
  FROM read_json_auto('office3/_meta/fetch_manifest.jsonl') AS f
470
+ JOIN 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.2/meta/bench_overlap.parquet' AS o
471
  ON o.source = f.source AND o.uri = f.uri
472
  WHERE f.status = 'ok'
473
  GROUP BY ALL;
 
499
  @misc{agentic_env_index,
500
  title = {agentic-env-index: an index of public materials for building agent environments},
501
  howpublished = {https://huggingface.co/datasets/Battam/agentic-env-index},
502
+ note = {Release release-2026-11-10.2},
503
  year = {2026}
504
  }
505
  ```
 
581
 
582
  ```sql
583
  SELECT hit_class, scope, count(*) AS rows
584
+ FROM 'hf://datasets/Battam/agentic-env-index@release-2026-11-10.2/meta/bench_overlap.parquet'
585
  GROUP BY ALL ORDER BY hit_class, scope;
586
  ```
587
 
portal.json CHANGED
@@ -1,8 +1,9 @@
1
  {
2
- "commit": "bea725842af5b8c9b57d76452483fa9c937425b3",
3
- "digest": "9682bf827630bf1a3b042d02a8620efdffb1907bad6f7a8c34e60e4b3e6609ce",
4
  "files": 19,
5
  "format": "portal-deploy/1",
6
- "index": "5b9f5a4d2bde4bee82f6a0b178b5ab088603564f",
 
7
  "stages": "70a0fac1235edf355cc4272613f43bb75be76c9759c43f8dc68cb254cea303cf"
8
  }
 
1
  {
2
+ "commit": "a015777686bf1cedb9df5ff6afd17d43fff21610",
3
+ "digest": "63a115cc1a819d10fc6a572ea53d2b8ee4aa63e824ef957d062826b8892bf2ff",
4
  "files": 19,
5
  "format": "portal-deploy/1",
6
+ "funnel": "83dae072040e8688c08c84fb72fdc8690d0956e5",
7
+ "index": "83dae072040e8688c08c84fb72fdc8690d0956e5",
8
  "stages": "70a0fac1235edf355cc4272613f43bb75be76c9759c43f8dc68cb254cea303cf"
9
  }
stages.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
  "counted": false,
3
  "format": "portal-stages/1",
4
- "index": "5b9f5a4d2bde4bee82f6a0b178b5ab088603564f",
5
  "rules": [
6
  "benchmark",
7
  "not compared",
 
1
  {
2
  "counted": false,
3
  "format": "portal-stages/1",
4
+ "index": "83dae072040e8688c08c84fb72fdc8690d0956e5",
5
  "rules": [
6
  "benchmark",
7
  "not compared",