Buckets:

SaylorTwift
/

agent-traces-analysis

Files

xet

SaylorTwift/agent-traces-analysis / README.md

SaylorTwift

21 days ago

preview code

download

raw

1.78 kB

# Agent traces on the Hub — analysis (Oct 2026)

What models, harnesses, tasks and session sizes show up in the Hub datasets tagged `agent-traces` (505 datasets).

## data/
| file | one row per | notes |
|---|---|---|
| `datasets.jsonl` | dataset (505) | harness, files, bytes, dates, downloads, license, models named in the card, `copy_of` |
| `sessions.jsonl` | session (12,323 after content dedup) | harness, models, providers, user msgs, tool calls (+ per-tool counts), tool errors, tokens, cost (pi), compactions, aborts, wall/active seconds, `category` |
| `prompts.jsonl` | unique user message (32,670) | dataset, format, turn index, text (≤ 3k chars), number of copies |
| `prompt_labels.jsonl` | labelled message (3,185; ≤ 40 per dataset) | intent, domain, follow-up kind, human/automated, scope — labelled by GLM-5.3-Flash |
| `people_datasets.json` | — | datasets counted as people working with an agent |
| `icml_repro_sessions.jsonl`, `icml_repro_harness.json` | ICML 2026 reproduction-challenge logbooks | harness recorded per logbook (Trackio format) |

`category` in `sessions.jsonl`: `real` (people: multi-turn, not templated), `synthetic-dump` (one generated prompt per session), `benchmark/harness` (eval sweeps), `paper-repro-bot` (ICML challenge runs).

Coverage: pi datasets read in full; Claude Code / Codex ≤ 40 files per dataset. Copies and re-uploads collapsed by session content. Active time ignores gaps > 30 min.

## charts/
Sticker-style charts + `9_infographic.png`. `thread.txt` is the accompanying Twitter thread.

## scripts/
`inventory.py` (dataset list) → `extract_prompts.py` (user messages) → `sessions.py` (per-session stats) → `agg_sessions.py` (dedup, categories) → `label_asks.py` (LLM labels) → `make_charts.py`.

Xet Storage Details

Size:: 1.78 kB
Xet hash:: be9f00484a2cb51a056ea1456ad920787e02d3cb03a07ff6b1fe34a4933de7cb

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	# Agent traces on the Hub — analysis (Oct 2026)

	What models, harnesses, tasks and session sizes show up in the Hub datasets tagged `agent-traces` (505 datasets).

	## data/
	\| file \| one row per \| notes \|
	\|---\|---\|---\|
	\| `datasets.jsonl` \| dataset (505) \| harness, files, bytes, dates, downloads, license, models named in the card, `copy_of` \|
	\| `sessions.jsonl` \| session (12,323 after content dedup) \| harness, models, providers, user msgs, tool calls (+ per-tool counts), tool errors, tokens, cost (pi), compactions, aborts, wall/active seconds, `category` \|
	\| `prompts.jsonl` \| unique user message (32,670) \| dataset, format, turn index, text (≤ 3k chars), number of copies \|
	\| `prompt_labels.jsonl` \| labelled message (3,185; ≤ 40 per dataset) \| intent, domain, follow-up kind, human/automated, scope — labelled by GLM-5.3-Flash \|
	\| `people_datasets.json` \| — \| datasets counted as people working with an agent \|
	\| `icml_repro_sessions.jsonl`, `icml_repro_harness.json` \| ICML 2026 reproduction-challenge logbooks \| harness recorded per logbook (Trackio format) \|

	`category` in `sessions.jsonl`: `real` (people: multi-turn, not templated), `synthetic-dump` (one generated prompt per session), `benchmark/harness` (eval sweeps), `paper-repro-bot` (ICML challenge runs).

	Coverage: pi datasets read in full; Claude Code / Codex ≤ 40 files per dataset. Copies and re-uploads collapsed by session content. Active time ignores gaps > 30 min.

	## charts/
	Sticker-style charts + `9_infographic.png`. `thread.txt` is the accompanying Twitter thread.

	## scripts/
	`inventory.py` (dataset list) → `extract_prompts.py` (user messages) → `sessions.py` (per-session stats) → `agg_sessions.py` (dedup, categories) → `label_asks.py` (LLM labels) → `make_charts.py`.