# Task 16: Documentation, Examples, and Migration Closure Complete the ROOT-GNN rewrite by making GNN4Colliders understandable, reproducible, and usable without relying on knowledge of the legacy repository. This task should not introduce new scientific functionality. It should consolidate the architecture, document the supported workflows, provide end-to-end examples, close migration gaps, and clearly separate: ```text supported behavior intentional redesigns legacy-only behavior deferred work ``` This task builds on: ```text Task 6: ROOT/Awkward ingestion Task 7: metadata-aware caching, splits, batching Task 8: ROOT-GNN model and transfer/fine-tuning Task 9: classification tasks, losses, metrics Task 10: training lifecycle Task 11: checkpointing and resume Task 12: evaluation, prediction, and outputs Task 13: Hydra configuration and CLI Task 14: DDP, Perlmutter, and Slurm execution Task 15: profiling and performance optimization ``` Before making changes, read: ```text AGENTS.md README.md docs/architecture.md docs/migration.md docs/performance.md ``` if `docs/performance.md` exists. Then inspect: ```text configs/ benchmarks/ scripts/ src/gnn4colliders/ tests/ legacy/ ``` Also inspect the active example/reference configurations introduced in Tasks 13–15. Do not assume documentation is correct merely because it already exists. Verify examples against the actual current APIs and CLI. --- # Goal Finish the ROOT-GNN rewrite as a coherent v1-quality workflow. A new user should be able to understand and execute: ```text install ↓ prepare data ↓ pretrain ROOT-GNN ↓ resume training ↓ fine-tune from pretrained backbone ↓ evaluate ↓ predict ↓ run on Perlmutter ``` without reading legacy implementation code. The final documentation should explain the architecture clearly enough that a future ROOT-Transformer implementation can reuse the shared infrastructure correctly. --- # 1. Treat this as migration closure The primary purpose of this task is not to add more features. It is to answer: ```text What is implemented? What is parity-tested? What changed intentionally? What remains legacy-only? What is deferred? How does a user run the new stack? ``` Do not expand scope into new model families or major infrastructure. --- # 2. Audit documentation against production code Before editing documentation, verify the current behavior of: ```text CLI commands Hydra config names model constructors task names checkpoint workflows inference formats distributed launch benchmark commands ``` Update docs to match code. Do not document aspirational commands that do not work. --- # 3. README should become the primary entry point Rewrite or substantially improve: ```text README.md ``` so a new user can understand the project quickly. Prefer a structure such as: ```text GNN4Colliders Overview Installation Quick start Core concepts Pretraining Fine-tuning Evaluation Prediction Checkpoint/resume Perlmutter Configuration Development Architecture Migration status ``` Keep the README useful but not encyclopedic. Link to detailed docs for deeper material. --- # 4. Project description Clearly describe GNN4Colliders as a collider-ML toolkit rather than a permanently GNN-only package. For example, communicate the conceptual architecture: ```text ROOT ↓ EventSample ↓ shared collider features ├── GraphSample -> ROOT-GNN └── future SequenceSample -> ROOT-Transformer ``` Do not imply graph-specific infrastructure is the universal shared representation. --- # 5. Explain package naming Clarify: ```text repository: GNN4Colliders Python package: gnn4colliders model family: ROOT-GNN config/module identifier: root_gnn ``` Use consistent naming throughout all docs. Do not alternate between legacy names casually. --- # 6. Installation documentation Document the canonical development installation. Use the actual current project setup. For example, if still correct: ```bash uv sync --extra root-gnn ``` Document: ```text supported Python version core install ROOT-GNN optional dependencies development/test dependencies ``` Do not document dependency combinations that are no longer validated. --- # 7. Python version Ensure all documentation consistently states the canonical supported Python version/range from `pyproject.toml`. Do not leave stale references to legacy Python 3.8 or earlier Task 2 experiments. --- # 8. DGL / PyTorch environment Document the validated ROOT-GNN environment clearly. If DGL installation still requires a nonstandard wheel source or environment-specific setup, explain it accurately. Do not include obsolete installation workarounds from earlier migration debugging. Keep environment-specific caveats concise. --- # 9. Quick-start workflow Add a minimal quick start that demonstrates the current stable flow. Conceptually: ```bash uv sync --extra root-gnn uv run gnn4colliders prepare ... uv run gnn4colliders train \ task=pretraining_multiclass \ model=root_gnn/edge_network uv run gnn4colliders evaluate \ inference.checkpoint=/path/to/checkpoint.pt ``` Use the exact current CLI syntax. The example should be executable with documented sample/tiny data where feasible. --- # 10. Document `prepare` Explain: ```text ROOT input -> EventSample -> features -> graphs -> cache ``` Document: ```text what is cached where cache is written how cache compatibility is checked what invalidates a cache ``` Do not expose implementation details that users do not need. --- # 11. Document metadata semantics Clearly explain that the new architecture does not use public positional: ```text tracking[:, N] ``` fields. Document named metadata concepts such as: ```text fold weight sample_id ``` and any active additional fields. Explain that legacy mappings are handled only at compatibility boundaries. --- # 12. Document `EventSample` Describe the architecture-neutral event representation. Conceptually: ```text EventSample objects label global_features metadata ``` Use the actual current API. Make clear this object is shared infrastructure and is not ROOT-GNN-specific. --- # 13. Document graph construction Explain ROOT-GNN graph semantics: ```text directed fully connected graph no self-loops N * (N - 1) edges ``` Document active node feature schema: ```text pt eta phi energy btag charge node_type ``` and edge features: ```text deta dphi dR ``` Do not reproduce unnecessary implementation code in the README. Link to architecture docs for details. --- # 14. Document feature semantics Document scientifically relevant compatibility behavior such as: ```text energy calculation phi wrapping feature scaling object ordering ``` where needed for reproducibility. Clearly distinguish user-visible scientific contracts from internal implementation details. --- # 15. Document ROOT-GNN Describe: ```text EdgeNetwork ``` at a conceptual level: ```text node/edge/global encoders -> iterative edge/node/global message passing -> graph representation -> classifier -> raw logits ``` State explicitly that the model returns raw logits rather than sigmoid/softmax outputs. --- # 16. Document transfer/fine-tuning This is a first-class workflow and must be documented clearly. Explain: ```text pretrained multiclass ROOT-GNN ↓ load reusable backbone ↓ replace source classifier ↓ binary/target-task classifier ↓ fine-tune ``` Document: ```text freeze_backbone=true freeze_backbone=false ``` and their meaning. Do not describe transfer learning as resume training. --- # 17. Distinguish resume vs transfer learning Create a prominent explanation. ```text Resume: continue the same training run/task restore optimizer/scheduler/trainer state Transfer: reuse pretrained model/backbone create a new task/head create a new optimizer ``` This distinction should appear in both README and checkpoint documentation. --- # 18. Pretraining example Provide one complete example for multiclass pretraining. Use the actual current config names. Conceptually: ```bash uv run gnn4colliders train \ model=root_gnn/edge_network \ task=pretraining_multiclass ``` Include the important overrides a user is likely to need: ```text data path output path batch size epochs ``` Keep the example readable. --- # 19. Fine-tuning example Provide one complete transfer-learning example. Conceptually: ```bash uv run gnn4colliders train \ model=root_gnn/fine_tuned_edge_network \ task=binary_classification \ checkpoint.pretrained=/path/to/pretrained.pt ``` Use the actual final configuration layout. Also show how to choose: ```text freeze_backbone=true ``` or: ```text freeze_backbone=false ``` --- # 20. Resume example Provide a separate example: ```bash uv run gnn4colliders train \ checkpoint.resume=/path/to/checkpoint.pt ``` Use the actual CLI. Explain exactly what state is restored. --- # 21. Evaluation example Document: ```bash uv run gnn4colliders evaluate ... ``` Explain: ```text which split is evaluated how metrics are computed where event weights are used how full-split ROC AUC is handled ``` Do not duplicate Task 9 formulas unnecessarily in README. Link to more detailed docs. --- # 22. Prediction example Document: ```bash uv run gnn4colliders predict ... ``` Cover: ```text NPZ output ROOT output if supported labeled vs unlabeled datasets sample_id preservation ``` Explain that inference output uses named fields rather than legacy positional tracking arrays. --- # 23. Output schema documentation Document the primary NPZ output schema. For example, if implemented: ```text sample_id logits scores predictions labels weight fold ``` Use only actual current fields. Clearly distinguish required and optional fields. --- # 24. ROOT output documentation If ROOT score writing exists, document: ```text tree handling score branch naming selection_pass behavior event alignment ``` Do not leave this behavior discoverable only through tests. If ROOT writing was intentionally deferred, say so clearly instead. --- # 25. Checkpoint documentation Create or improve a detailed checkpoint section. Document: ```text new checkpoint schema schema version model metadata task metadata trainer state optimizer state scheduler state early stopping ``` Explain that checkpoints are self-describing to the supported extent. --- # 26. Legacy checkpoint compatibility Document exactly what historical checkpoints are supported. For example: ```text model-weight loading: supported pretrained-backbone loading: supported full legacy optimizer resume: supported / unsupported module. prefix normalization: supported _orig_mod. normalization: supported ``` Use actual implemented behavior. Do not claim compatibility that has not been tested. --- # 27. Configuration documentation Add a dedicated configuration guide if helpful, for example: ```text docs/configuration.md ``` Explain Hydra config groups: ```text data model task trainer checkpoint inference environment distributed ``` Use actual current groups. --- # 28. Explain configuration philosophy Document the rule: ```text new experiment -> configuration/YAML new algorithm or behavior -> Python ``` Explain that config files use semantic names rather than arbitrary module/class import paths. This is an important departure from the legacy system. --- # 29. Hydra override examples Provide concise examples such as: ```bash uv run gnn4colliders train \ trainer.max_epochs=50 \ data.batch_size=64 ``` and relevant model/task/environment overrides. Avoid an enormous override catalog. --- # 30. Resolved configuration Document that runs persist their fully resolved config. Explain where it is written. This should be presented as part of experiment reproducibility. --- # 31. Environment profiles Document: ```text environment=local environment=perlmutter ``` or the actual final names. Explain what environment profiles control and what they should not contain. Do not encourage users to put scientific model settings in environment configs. --- # 32. Perlmutter guide Add or improve: ```text docs/perlmutter.md ``` if a dedicated guide is useful. Cover: ```text environment setup uv environment single-GPU run multi-GPU run Slurm scripts output paths common debugging checks ``` Use only validated current commands. --- # 33. Slurm examples Document the scripts under: ```text scripts/slurm/ ``` Explain which script to use for: ```text single GPU single-node multi-GPU multi-node evaluation benchmarking ``` if those scripts exist. Do not duplicate the entire script contents in README. --- # 34. DDP semantics Document distributed behavior important to users: ```text batch_size is per GPU/process effective global batch size = batch_size * world_size num_workers is per process only rank 0 writes shared artifacts ``` Also explain global metric aggregation at a conceptual level. --- # 35. Reproducibility documentation Document the current reproducibility model. Cover: ```text base seed DataLoader seed sampler set_epoch behavior rank-specific seed derivation model initialization CPU determinism GPU nondeterminism caveats ``` Do not promise bitwise GPU determinism unless actually validated. --- # 36. Performance guide Link to: ```text docs/performance.md ``` or: ```text benchmarks/README.md ``` Document validated recommendations for: ```text num_workers pin_memory persistent_workers precision DDP ``` where Task 15 produced evidence. Do not turn hardware-specific benchmark results into universal defaults. --- # 37. Architecture documentation Review and consolidate: ```text docs/architecture.md ``` It should clearly describe the current architecture. At minimum cover: ```text ROOT/Awkward I/O EventSample shared features GraphSample cache GraphBatch ROOT-GNN task layer trainer checkpointing inference configuration/CLI distributed execution ``` Show responsibility boundaries. --- # 38. Architecture diagram Add a concise textual/Markdown diagram. For example: ```text ROOT ↓ EventSample ↓ shared collider features ↓ GraphSample ↓ cache / split / batch ↓ GraphBatch ↓ ROOT-GNN ↓ logits ↓ Task ├── loss ├── predictions └── metrics ↓ Trainer / Predictor ``` Also show future extensibility: ```text shared features ├── GraphSample -> ROOT-GNN └── SequenceSample -> ROOT-Transformer ``` Do not make ROOT-Transformer appear implemented if it is not. --- # 39. Responsibility table Consider adding a concise table in architecture docs. For example: ```text Layer Responsibility data ROOT/Awkward ingestion and samples features shared collider feature construction graphs graph representation/topology models/root_gnn ROOT-GNN architecture tasks losses/predictions/metrics training lifecycle/checkpoints inference prediction/output distributed DDP execution utilities cli/config user-facing composition ``` Use current code structure. --- # 40. Migration document Turn: ```text docs/migration.md ``` into a clear migration-status document. For each major legacy area, mark: ```text migrated compatibility adapter intentionally redesigned deferred not supported ``` Avoid vague status such as "mostly done." --- # 41. Legacy migration matrix Add a matrix for active legacy behavior. Include areas such as: ```text ROOT ingestion node features edge construction cache folds weights batching padding Edge_Network fine-tuning loss metrics training early stopping checkpoints inference ROOT outputs DDP Slurm ONNX ``` For each, document current status. --- # 42. Intentional redesigns Explicitly document important departures from legacy architecture. At minimum include: ```text tracking[:, 0] -> metadata.fold tracking[:, 1] -> metadata.weight tracking arrays -> named metadata dynamic YAML module/class imports -> semantic Hydra configs large training script -> layered Trainer/Task/Checkpoint APIs persistent graph mutation -> scoped temporary graph state model constructor global RNG mutation -> explicit external seeding ``` Only list changes actually implemented. --- # 43. Preserve behavior vs preserve bugs Migration docs should explain that compatibility targets externally relevant scientific behavior. Do not imply every legacy implementation quirk is intentionally retained. Classify known legacy quirks as appropriate: ```text compatibility requirement observed behavior intentional cleanup known legacy bug deferred investigation ``` --- # 44. Outstanding ambiguities Review earlier characterization work and document unresolved issues. Examples may include: ```text negative event weights rare empty graphs padding edge cases legacy validation/test naming historical checkpoint variants ``` Do not silently erase unresolved ambiguities from the migration record. If later tasks resolved them, update their status. --- # 45. Example directory Add a small examples area if useful: ```text examples/ README.md pretraining/ finetuning/ inference/ ``` Prefer configuration examples over duplicate Python scripts. Do not create a parallel application framework under `examples/`. --- # 46. Example configs Provide complete runnable or nearly runnable examples for: ```text multiclass pretraining binary fine-tuning resume evaluation prediction ``` Use placeholders only for values that necessarily depend on local data paths. Clearly mark placeholders. --- # 47. Tiny smoke workflow If the repository has tiny ROOT fixtures or can safely provide a tiny generated fixture workflow, document a smoke-test path. The goal is something like: ```text prepare tiny data train one epoch save checkpoint evaluate predict ``` that runs quickly on CPU. Prefer reusing test fixtures/tooling rather than committing large binary datasets. --- # 48. Do not misuse test-only fixtures If tests generate their own ROOT fixtures dynamically, do not tell users to depend on internal pytest-only APIs. If useful, extract a small supported example-data generator into an appropriate script. Only do this if it materially improves onboarding. --- # 49. Developer guide Add or improve a development section. Document: ```bash uv sync --extra root-gnn uv run pytest uv run ruff check . uv run ruff format --check . ``` and relevant benchmark commands. Explain the repository rule: ```text one logical task -> one coherent commit ``` if this is an established project convention. --- # 50. Testing guide Document the purpose of: ```text tests/unit/ tests/integration/ tests/parity/ ``` Explain that parity tests protect migration behavior. Do not list every test file. --- # 51. Parity philosophy Document the rule: ```text legacy executable behavior is characterized first new implementation is compared against deterministic fixtures ``` Explain that parity tests are especially important for: ```text features graph topology model forward loss semantics metrics checkpoint loading inference outputs ``` --- # 52. Benchmark guide Document how to run Task 15 benchmarks. For example: ```bash uv run python benchmarks/benchmark_preprocessing.py uv run python benchmarks/benchmark_training.py --device cuda ``` Use actual working commands. Explain that benchmark results depend on hardware and configuration. --- # 53. CLI help audit Run: ```bash uv run gnn4colliders --help uv run gnn4colliders prepare --help uv run gnn4colliders train --help uv run gnn4colliders evaluate --help uv run gnn4colliders predict --help ``` Review for confusing/stale names. Minor help-text improvements are in scope. Do not redesign the CLI. --- # 54. Config naming audit Review config names for consistency. Remove or rename obviously stale/experimental config names only if doing so does not unnecessarily break established use. Prefer: ```text pretraining_multiclass binary_classification fine_tuning ``` or the actual established semantic naming. Document aliases if compatibility requires them. --- # 55. Dead documentation cleanup Remove stale documentation that refers to: ```text legacy positional tracking APIs old script entry points dynamic import YAML obsolete dependency versions abandoned architecture plans ``` unless explicitly preserved in migration-history sections. Do not leave contradictory instructions in different docs. --- # 56. Docstrings Add or improve docstrings for important public APIs where absent. Prioritize: ```text EventSample GraphSample GraphBatch EdgeNetwork FineTunedEdgeNetwork classification tasks Trainer CheckpointManager Predictor DistributedContext ``` Keep docstrings concise. Do not write essays inside source files. --- # 57. Type/API clarity While reviewing docs, minor fixes to public annotations or obvious naming inconsistencies are acceptable. Do not refactor major APIs in this documentation task. If a serious design flaw is discovered, document it as follow-up rather than destabilizing the finished migration. --- # 58. Public API examples Ensure documented imports work. For example: ```python from gnn4colliders.models.root_gnn import EdgeNetwork from gnn4colliders.tasks import BinaryClassificationTask from gnn4colliders.training import Trainer from gnn4colliders.inference import Predictor ``` Use only exports that actually exist. Do not teach users to import private modules. --- # 59. Python API example Add one minimal Python example for users who do not want the CLI. Conceptually: ```python model = ... task = ... trainer = ... ``` Keep it short. The primary supported user workflow can remain CLI/Hydra. --- # 60. CLI remains primary experiment interface Document the intended split: ```text CLI/Hydra: normal experiments Python APIs: libraries, testing, custom workflows ``` Do not force all users into notebooks or scripts. --- # 61. Notebooks Do not create a large notebook suite in this task. If existing notebooks are stale, either: ```text update them mark them exploratory remove references to them ``` as appropriate. Notebooks should not be the canonical production workflow. --- # 62. Legacy directory documentation Explain the purpose of: ```text legacy/ ``` Clearly state that it is a frozen behavioral reference and not production implementation. Do not tell users to add `legacy/` to `PYTHONPATH`. --- # 63. Legacy old rewrite attempt If: ```text legacy/physicsnemo/ ``` or an equivalent abandoned rewrite remains, document clearly that it is not the behavioral target. Do not accidentally present it as a supported backend. --- # 64. Avoid deleting legacy in Task 16 Do not delete the legacy implementation yet. It remains useful for: ```text parity tests historical reference checkpoint compatibility investigation ``` Legacy cleanup/deletion belongs to a later dedicated task. --- # 65. Create a migration completion checklist At the end of `docs/migration.md`, include a concise checklist. For example: ```text [x] active data path [x] graph construction [x] EdgeNetwork [x] transfer learning [x] task losses/metrics [x] trainer [x] checkpoints [x] inference [x] CLI [x] DDP [x] profiling [ ] ONNX export [ ] ROOT-Transformer ``` Use actual status. Do not mark something complete solely because there is placeholder code. --- # 66. Define ROOT-GNN v1 completion Add a concise statement describing what constitutes the completed ROOT-GNN rewrite. For example: ```text ROOT-GNN v1 is complete when the supported new stack can: - prepare active datasets - reproduce validated legacy feature/graph/model/task behavior - train from scratch - fine-tune pretrained backbones - resume checkpoints - evaluate/predict - run single-process and DDP on Perlmutter ``` Tailor this to actual implemented functionality. --- # 67. Known limitations Add a clear known-limitations section. Possible examples, if still true: ```text ONNX export deferred distributed ROOT writing limited full historical optimizer-resume compatibility incomplete very large inference uses in-memory result accumulation ROOT-Transformer not yet implemented ``` Do not hide limitations. --- # 68. Future roadmap Keep the roadmap short. After ROOT-GNN v1, likely items include: ```text ONNX/export legacy cleanup packaging/release hardening ROOT-Transformer large-scale inference improvements ``` Do not turn README into a speculative roadmap document. --- # 69. Validate every command in documentation For every command intended to be runnable, execute it or at least its safe smoke equivalent. Especially validate: ```text installation --help config composition tiny prepare tiny train tiny fine-tune tiny evaluate tiny predict benchmark command syntax ``` Do not ship copied commands that have not been checked. --- # 70. Documentation tests If helpful, add lightweight tests to protect key docs/API snippets. For example: ```text CLI help smoke tests example config composition tests public import tests ``` Do not introduce a heavyweight documentation-testing framework unless already used. --- # 71. Config smoke tests Validate all reference configs compose successfully. For each committed reference configuration, ensure Hydra can resolve it without performing expensive training. Fail on stale field names. --- # 72. Example smoke tests If example configs/scripts are added, add small tests ensuring they do not rot. Keep these tests fast. Do not require production ROOT files. --- # 73. Documentation consistency Use consistent terminology everywhere. Prefer: ```text training validation test ``` rather than reproducing confusing legacy naming. Prefer: ```text metadata.weight metadata.fold ``` rather than tracking columns. Prefer: ```text fine-tuning ``` or one consistent spelling throughout. --- # 74. Scientific terminology Preserve established collider terminology accurately. Do not rename scientifically meaningful fields merely for stylistic reasons. Where names have historical meanings, explain them. --- # 75. Code comments Do not compensate for unclear docs by adding excessive inline comments everywhere. Use source comments only for non-obvious implementation constraints. Documentation belongs primarily in README/docs and concise public docstrings. --- # 76. No new major dependencies This task should add no major runtime dependency. Documentation tooling dependencies are also unnecessary unless the project already has a docs build system. Plain Markdown is sufficient. --- # 77. Do not build a documentation website Do not introduce: ```text Sphinx MkDocs Docusaurus ``` unless already present and clearly desired. The task is content completion, not documentation-platform migration. --- # 78. Review `.gitignore` Ensure generated artifacts from documented workflows are not accidentally committed. Check handling of: ```text outputs/ checkpoints/ predictions/ profiles/ benchmark-results/ __pycache__/ *.egg-info/ ``` Only change `.gitignore` where appropriate. Do not ignore source example configs or benchmark scripts. --- # 79. Repository cleanliness Remove accidental generated development artifacts if they are tracked or unignored, such as: ```text __pycache__/ src/gnn4colliders.egg-info/ temporary outputs profiling traces ``` Do not delete user data. Only clean repository-generated artifacts. --- # 80. Architecture-neutral audit Use this task to verify the shared design still supports a future non-graph model family. Review especially: ```text data/ features/ training/ tasks/ inference/ distributed/ config/ ``` for unnecessary ROOT-GNN-specific naming or assumptions. Do not perform major refactors. Document any remaining coupling as follow-up work. --- # 81. ROOT-Transformer readiness note Add a concise architecture note for future development. Conceptually: ```text ROOT -> EventSample -> shared feature representation ├── graph adapter -> ROOT-GNN └── future sequence adapter -> ROOT-Transformer ``` Identify the intended extension points. Do not implement the transformer here. --- # 82. Migration debt audit Search for remaining new-code references to legacy concepts such as: ```text tracking tracking_info sys.path legacy module/class dynamic import hardcoded ROOT-GNN assumptions in shared code ``` Classify each result: ```text required compatibility test-only documentation/history migration debt ``` Fix only obvious low-risk leftovers. Document the rest. --- # 83. TODO/FIXME audit Search: ```text TODO FIXME HACK XXX ``` through production code. For each: * resolve it if trivial and in scope * convert it into a clear documented limitation * remove stale notes * leave substantial future work for later tasks Do not turn Task 16 into an unbounded cleanup sprint. --- # 84. Warning/error message audit Review major user-facing failures. Examples: ```text invalid config missing input files incompatible cache incompatible checkpoint wrong model/task combination unsupported output format distributed misconfiguration ``` Improve obviously cryptic messages where low-risk. Do not redesign exception architecture. --- # 85. Final end-to-end smoke workflow Create one documented validation workflow that exercises the new system from start to finish using tiny inputs. Conceptually: ```text prepare ↓ train multiclass model briefly ↓ save checkpoint ↓ load pretrained backbone ↓ fine-tune binary model briefly ↓ evaluate ↓ predict NPZ ``` This does not need scientific convergence. Its purpose is integration confidence. --- # 86. Do not use legacy code in the smoke workflow The final new-stack smoke workflow must run without importing production logic from: ```text legacy/ ``` Parity tests may continue to use legacy. The actual user workflow must not. --- # 87. Optional end-to-end script If useful, add something like: ```text scripts/dev/smoke_end_to_end.sh ``` or: ```text scripts/dev/smoke_end_to_end.py ``` Keep it thin and built entirely on the public CLI. Do not duplicate application logic. --- # 88. Smoke script requirements A smoke script should: * use temporary/tiny data * avoid hardcoded user paths * fail on command errors * be readable * not require Slurm * not require GPU Do not make it part of normal package runtime. --- # 89. Release-readiness checklist Add a concise internal checklist to docs if useful. For example: ```text tests pass parity tests pass CLI examples validated reference configs compose README current migration matrix current no generated artifacts tracked ``` Do not introduce formal release tooling yet. --- # 90. Testing Run: ```bash uv run pytest ``` Run focused parity: ```bash uv run pytest tests/parity -v ``` Run distributed tests: ```bash uv run pytest tests/unit/distributed -v uv run pytest tests/integration -k distributed -v ``` Run lint/format: ```bash uv run ruff check . uv run ruff format --check . ``` Validate public imports. Validate CLI: ```bash uv run gnn4colliders --help uv run gnn4colliders prepare --help uv run gnn4colliders train --help uv run gnn4colliders evaluate --help uv run gnn4colliders predict --help ``` Validate all committed reference configs compose. Validate the documented tiny end-to-end workflow. If Slurm scripts are present: ```bash bash -n scripts/slurm/*.sh ``` or the appropriate individual scripts. Inspect: ```bash git status git diff ``` Verify: * no scientific behavior changed unexpectedly * no production imports from legacy were introduced * docs match current CLI/config APIs * no stale tracking-array usage is presented as public API * no generated artifacts are tracked * no major new dependency was added * no unrelated refactor is included --- # Completion criteria Task 16 is complete when: 1. README accurately describes the current project. 2. Installation instructions are validated. 3. Project/package/model naming is consistent. 4. The quick-start workflow is documented. 5. Data preparation is documented. 6. `EventSample` and named metadata concepts are documented. 7. ROOT-GNN graph semantics are documented. 8. ROOT-GNN model behavior is documented. 9. Transfer/fine-tuning is documented as a first-class workflow. 10. Resume and transfer learning are clearly distinguished. 11. Pretraining example is validated. 12. Fine-tuning example is validated. 13. Resume example is validated. 14. Evaluation example is validated. 15. Prediction example is validated. 16. Checkpoint format/workflows are documented. 17. Supported legacy checkpoint compatibility is documented accurately. 18. Hydra configuration groups and overrides are documented. 19. Environment/Perlmutter execution is documented. 20. DDP batch-size and worker semantics are documented. 21. Reproducibility behavior is documented accurately. 22. Performance guidance from Task 15 is documented. 23. `docs/architecture.md` reflects the final implementation. 24. `docs/migration.md` contains a clear status matrix. 25. Intentional redesigns from legacy are documented. 26. Remaining ambiguities/limitations are explicit. 27. Future ROOT-Transformer extension points are documented without implementing it. 28. Reference configs compose successfully. 29. Public imports shown in docs work. 30. CLI help commands work. 31. Tiny end-to-end new-stack smoke workflow succeeds. 32. Full unit/integration/parity suite passes. 33. Distributed tests still pass. 34. Lint/format checks pass. 35. Legacy code remains frozen. 36. No major new feature or dependency has been introduced. --- # Completion report Report: 1. files created 2. files modified 3. README structure 4. installation instructions finalized 5. quick-start workflow 6. pretraining example 7. fine-tuning example 8. resume example 9. evaluation example 10. prediction example 11. output schema documentation 12. checkpoint documentation 13. Hydra/configuration documentation 14. Perlmutter/DDP documentation 15. reproducibility documentation 16. performance documentation 17. architecture-document updates 18. migration matrix summary 19. intentional legacy redesigns documented 20. unresolved migration items 21. architecture-neutrality audit findings 22. ROOT-Transformer extension points 23. stale/dead documentation removed 24. generated repository artifacts cleaned 25. TODO/FIXME audit results 26. public import validation 27. CLI help validation 28. reference config validation 29. end-to-end smoke-test results 30. full test results 31. lint/format results 32. any follow-up tasks identified After validation succeeds, create one Git commit containing only Task 16 changes. Use: ```text docs: finalize ROOT-GNN documentation and migration ``` Before committing, inspect the final diff and ensure no unrelated implementation changes or generated artifacts are included.