|
Download README.md from tmlr-embedding-artifacts/recursive-geodesic-checkpoints: direct link, hf CLI and curl.
- Browser
- Download file 10.1 kB
-
https://huggingface.co/tmlr-embedding-artifacts/recursive-geodesic-checkpoints/resolve/main/README.md
- Command line
-
hf download hf://tmlr-embedding-artifacts/recursive-geodesic-checkpoints/README.md
-
curl -L -o README.md https://huggingface.co/tmlr-embedding-artifacts/recursive-geodesic-checkpoints/resolve/main/README.md
10.1 kB
| language: | |
| - en | |
| tags: | |
| - pytorch | |
| - document-embeddings | |
| - reproducibility | |
| - hierarchical-composition | |
| # One Vector from Many: Hierarchical Long-Document Representation through Recursive Geodesic Transformer Composition | |
| Research checkpoints for recursive composition of ordered document-chunk embeddings. The model combines a token-mass-weighted spherical reference with a learned bounded tangent correction and recursively reduces contiguous groups. The strict decoder predicts from the parent representation alone. These custom PyTorch checkpoints are not a Transformers `AutoModel` package. | |
| ## Artifacts and roles | |
| `MANIFEST.json` records exact filenames, original artifact roles, seeds, hosted paths, evaluation installation paths, byte sizes, SHA-256 hashes and metadata-only transformation mappings. `SHA256SUMS` independently lists the hosted checkpoint hashes. | |
| | Directory | Files | Role | | |
| | --- | ---: | --- | | |
| | `final/` | 3 | Final clean-v6 checkpoints used for final-model inference and the corresponding evaluations. | | |
| | `source/` | 3 | Seed-matched continuation initialization checkpoints; not interchangeable with the final models. | | |
| | `historical_probes/` | 6 | Historical fitted candidate/source parent-only probes, also included unchanged in the small supplement. | | |
| | `gradient_path/` | 6 | Separately trained attached/detached decoder-parent-gradient intervention outputs; not replacements for the main final models. | | |
| | `decoder_probes/` | 45 | Additional TRAIN-fitted direct/bounded/unbounded probe-family controls; distinct from the historical probes. | | |
| | `intermediate/` | 3 | Shared warm intervention states. See each manifest entry's transformation record before using these as provenance artifacts. | | |
| Only verified, publishable artifacts are listed in the release manifest. An intermediate's metadata-sanitized derivative is a separate serialized artifact with its own hash; original experiment records are not rebound to that derivative. See the review source's checkpoint access guide for the exact transformation and any limitations. | |
| ## Download and verify | |
| [Anonymous review source and reproduction instructions](https://anonymous.4open.science/r/recursive-geodesic-composition-0F32/). | |
| From that review checkout or the extracted supplementary ZIP: | |
| ```bash | |
| python scripts/download_checkpoints.py --group final --verify | |
| python scripts/download_checkpoints.py --group source --group historical_probes --verify | |
| ``` | |
| The standard-library helper uses the immutable 40-hex Hugging Face revision in `reproducibility/HOSTED_CHECKPOINTS.json`, sends no authentication credentials, downloads only the requested groups, verifies size/SHA-256, and installs exact evaluation protocol/lookup paths. Existing matching files and included probes are reused. `--group all` explicitly selects the entire collection. No token, gated-access approval or Hugging Face account is required. | |
| For a manual Hub download from the same checkout, obtain the immutable release revision from the companion manifest, not from moving `main`: | |
| ```bash | |
| REVISION=$(python -c 'import json; print(json.load(open("reproducibility/HOSTED_CHECKPOINTS.json"))["revision"])') | |
| hf download tmlr-embedding-artifacts/recursive-geodesic-checkpoints \ | |
| --revision "$REVISION" --local-dir checkpoint-files | |
| cd checkpoint-files | |
| sha256sum --check SHA256SUMS | |
| ``` | |
| The manifest inside this model repository describes the payload at the revision containing it; the companion source/ZIP manifest records that enclosing immutable revision. This avoids a self-referential commit hash in a file that would itself change that hash. Checksums detect byte changes; they are not authenticity signatures. | |
| ## Separately versioned recovered caches | |
| Six numeric cache bundles are an additive release, **not additions to or replacements for the immutable 66-checkpoint inventory**. `CACHES_MANIFEST.json` records all 8,075 extracted files, archive/member hashes and the two explicitly mapped metadata derivatives; `CACHE_SHA256SUMS` covers the six bundles and that manifest. The review source's `reproducibility/HOSTED_CACHES.json` pins their enclosing immutable revision separately from the weights. | |
| | Cache directory | Role | | |
| | --- | --- | | |
| | `cache_kary4_strict_autoencoder_v6_clean_train` | Original clean-v6 TRAIN leaves/teacher targets; original manifest matches the preserved historical checksum. | | |
| | `cache_kary4_strict_autoencoder_v6_clean_development` | Clean-v6 development safe/full leaves and full-document indexes. | | |
| | `cache_kary4_strict_autoencoder_official_test_v6` | Official safe/full leaves; explicitly identified anonymous manifest/COMPLETE derivatives. | | |
| | `cache_kary4_1024_expanded_source_seed424_v1` | Earlier expanded-source TRAIN/validation leaves and targets. | | |
| | `cache_kary4_leaf510_vs1024_v1_leaf1024` | Earlier leaf1024 TRAIN/validation tensors and sidecars; no original root manifest/COMPLETE was recovered. | | |
| | `cache_kary4_1024_validation3_strict_autoencoder_seed424_v2` | Separate validation3 leaves and targets. | | |
| The bundles total **911,050,856 bytes compressed**, **1,089,336,987 bytes extracted**. All **8,032 PT files are unchanged originals**. The official manifest alone replaces an identifying implementation-parent Git label with `anonymous-source-snapshot-01`; its COMPLETE points to the derivative manifest. Original/new metadata hashes are explicit. All tensor, index and sidecar bytes are preserved. Archive timestamps, ownership and permissions are neutralized. | |
| All 8,032 tensor files loaded with `weights_only=True` on CPU and contained finite floating tensors. One real row per cache directory passed the published final model's recursive reducer and strict decoder. Complete static pickle/ZIP metadata and CRC inspection found no other tested identity/credential markers or recognized raw-text/token-sequence fields. These are scoped integrity/loadability checks, not a historical metric rerun, perfect privacy proof or full reproduction. | |
| **Do not bypass frozen guards.** The official derivative is internally consistent but cannot satisfy the original manifest hash. Original config paths/hashes and some provenance hashes also differ from the normalized review layout. Frozen results/configs/guards remain unchanged. Some other study-specific vectors, tokens and task caches are still absent. | |
| ### Upstream attribution and terms | |
| These are numeric representations derived from the following datasets and pretrained teacher, not raw corpus text, token sequences or redistributed teacher weights. Dataset cards identify the licenses below; those labels do not settle every underlying web-page right or whether a particular derived representation triggers attribution/share-alike obligations. Retain applicable source terms; this project asserts no replacement blanket license. | |
| | Upstream creators/resource | Pinned source | Stated terms | | |
| | --- | --- | --- | | |
| | Hugging Face FineWeb maintainers and source contributors | [FineWeb, sample-10BT](https://huggingface.co/datasets/HuggingFaceFW/fineweb/tree/9bb295ddab0e05d785b879661af7260fed5140fc) | [ODC-BY 1.0](https://opendatacommons.org/licenses/by/1-0/) | | |
| | Hugging Face FineWeb-Edu maintainers and source contributors | [FineWeb-Edu, sample-10BT](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu/tree/87f09149ef4734204d70ed1d046ddc9ca3f2b8f9) | [ODC-BY 1.0](https://opendatacommons.org/licenses/by/1-0/) | | |
| | OpenWebText maintainers and source contributors | [OpenWebText](https://huggingface.co/datasets/Skylion007/openwebtext/tree/79d93d786212f7344586290adb811d4ae6a1762c) | [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/) | | |
| | Wikimedia/Wikipedia contributors | [Wikipedia, 20231101.en](https://huggingface.co/datasets/wikimedia/wikipedia/tree/b04c8d1ceb2f5cd4588862100d08de323dccfbaa) | [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/), [GFDL](https://www.gnu.org/licenses/fdl-1.3.html), as stated by the pinned card | | |
| | Alibaba Tongyi Lab | [gte-modernbert-base](https://huggingface.co/Alibaba-NLP/gte-modernbert-base/tree/e7f32e3c00f91d699e8c43b53106206bcc72bb22) | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) | | |
| Document/source-index identifiers and content hashes remain intentionally available for provenance and can link cached representations to upstream documents. Embeddings can retain information about their inputs; no source-document privacy guarantee is made. | |
| ## Scientific and reproduction limits | |
| - The released final clean-v6 models use **300 decoder-only updates plus 100 joint updates from the exact continuation-source checkpoint**, with a reinitialized decoder. They are not a from-scratch reconstruction. Recovered historical recipes/statuses trace the earlier source models through two encoder stages, but those earlier tensors and complete historical trainer/execution bindings remain missing; see the review source's partial lineage record. | |
| - Restoring weights and the six recovered caches does not supply every pinned teacher/corpus/task input, study-specific token/vector cache, compatible software or GPU resource. Historical source-fingerprint discrepancies remain documented; no numerical result or old hash is rewritten to make a rerun pass. | |
| - Separate older K-ary scaling checkpoints/probes and matched-pooling probes are outside the released 66-file inventory and remain unavailable unless explicitly listed in the companion access guide. | |
| - The warm intermediates are not required to load the six final gradient-intervention outputs. Metadata-only copies do not satisfy guards requiring original serialized-byte hashes. | |
| - Saved-score/statistical replay is not fresh model evaluation. Full-document roots have no direct full-root teacher target; retrieval results do not establish consistent superiority over pooling. | |
| - Publication asserts no new license grant; consult the anonymous review source's terms. Public scientific titles, results and exact artifact hashes remain linkable to already-public research. This repository does not promise perfect anonymity against deliberate searches or platform operators. | |