| # Ordinary Git code and Xet assets in one repository |
|
|
| One Hugging Face repository stores this project: [nmarshall/urt-rs](https://huggingface.co/nmarshall/urt-rs), SSH remote `hf` at `git@hf.co:nmarshall/urt-rs`. Code, scripts, modern WGSL sources, documentation and readable research metadata are ordinary Git text. Non-empty files below `assets/` are canonical Git LFS pointers; payload uploads/downloads use the locally registered Git Xet transfer agent. A single commit identifies code and derived assets. No separate dataset or nested asset repository is used. |
|
|
| ## Layout |
|
|
| ```text |
| urban-terror-rs/ |
| .gitattributes # Required text filter; assets/** uses LFS |
| .githooks/ # Commit/index/history guards and LFS upload hook |
| scripts/ # Setup, extraction and storage-policy code |
| docs/ # Findings and per-occurrence metadata in ordinary Git |
| assets/ |
| README.md # Uses LFS |
| extracted/ # All PK3 variants; archive/member-index isolation |
| converted/ # Future Bevy/Solari runtime content |
| captures/ # Future reference media and traces |
| .cache/UrbanTerror43/ # The original installation; ignored and unchanged |
| ``` |
|
|
| The 39 original installation files belong in `.cache/` only. Their copied `assets/original/` tree is removed and ignored. Both the original directory path and LFS pointers with any of the 39 installation-file SHA-256 values are rejected by the storage guard, preventing renamed original files from returning to published assets. The immutable source identity remains recorded in `docs/inventory/snapshot.json`. |
|
|
| `assets/extracted/` contains every PK3 member occurrence, as described in [the extraction findings](extraction/README.md). None of its payload hashes equals an original installation-file hash. Standalone binaries, libraries, license documents and installation configs remain available for research in `.cache/`; extraction does not discard them or claim their reverse engineering is complete. The retired unpublished nested checkout metadata also remains under ignored `.cache/retired-assets-checkout/`. |
|
|
| ## Filters and guards |
|
|
| ```gitattributes |
| * text=auto eol=lf filter=urt-code |
| assets/** filter=lfs diff=lfs merge=lfs -text !eol |
| ``` |
|
|
| The required `urt-code` clean filter passes through valid UTF-8 source/documentation. It rejects known asset/binary extensions, asset/build/cache directories, NUL bytes, invalid encoding, LFS pointers outside `assets/` and ordinary Git files above 2 MiB. Readable inventory/extraction manifests are metadata, not game payloads. New asset formats must extend the policy before ingestion. |
|
|
| Every asset path has LFS attributes, including small configs and original shader text extracted from PK3s. Git LFS writes a SHA-256 pointer for non-empty files. Its specification defines an empty file as its own empty pointer; the three zero-byte skins therefore contain zero bytes in Git and need no remote payload object. No non-empty raw asset data is accepted in ordinary Git. See the [Git LFS pointer specification](https://github.com/git-lfs/git-lfs/blob/main/docs/spec.md). |
|
|
| Pre-commit validates staged content, cached routing attributes and local filter/Xet configuration. Pre-push repeats those checks and validates every newly reachable commit, including payloads deleted later. A bounded `git cat-file --batch` stream handles thousands of pointers without launching Git for each file. Guards reject symlinks, unresolved entries, nested repositories, cached original-file identities and non-pointer payloads. The push hook uploads through `git lfs pre-push` before Git refs advance. |
|
|
| Git Xet is registered as the custom LFS transfer agent; `filter=lfs` is the supported Git filter. See [Hugging Face's Git/Xet integration](https://huggingface.co/docs/hub/xet/using-xet-storage). Setup changes local repository settings only. Clones do not install hooks automatically; deliberate disabling of hooks/configuration can bypass local checks. Future CI/server protection should run the same content policy. |
|
|
| ## Configure or clone |
|
|
| Requirements: Git, Python 3, Git LFS, Git Xet and Hugging Face SSH access. |
|
|
| ```sh |
| GIT_LFS_SKIP_SMUDGE=1 git clone git@hf.co:nmarshall/urt-rs |
| cd urt-rs |
| python3 scripts/setup-repository.py |
| git lfs pull |
| git lfs fsck |
| python3 scripts/storage_policy.py check --require-setup |
| ``` |
|
|
| The clone names its remote `origin`; this checkout names it `hf`. Substitute the configured name in push commands. For further extraction, provide the verified original installation under `.cache/UrbanTerror43/`; Git intentionally does not distribute it. Setup is repeatable and refuses to overwrite an unrelated hooks directory. |
|
|
| ## Work and publish |
|
|
| ```sh |
| python3 scripts/test_storage_policy.py |
| python3 scripts/test_unpack_pk3.py |
| python3 scripts/unpack_pk3.py extract |
| python3 scripts/unpack_pk3.py verify |
| git add . |
| python3 scripts/storage_policy.py check --require-setup |
| git commit -m "Describe the change" |
| git lfs fsck |
| GIT_LFS_FORCE_PROGRESS=1 git push hf main |
| ``` |
|
|
| For large uploads, enable progress and optionally set `GIT_LFS_PROGRESS` to an absolute ignored log path. If a Xet worker stops advancing with no live transfer connections, stop only that push process group and retry; already completed objects remain uploaded. A retry with `git -c lfs.concurrenttransfers=1 push hf main` successfully completed the earlier initial installation upload. It is a per-command setting, rather than a change to repository routing. |
|
|
| A non-empty asset's `git show HEAD:assets/path` must show a small LFS pointer; the working file contains the payload. Source files must show their source text directly. If a derived asset was staged incorrectly, restore setup/attributes, run `git add --renormalize assets/extracted`, and validate before committing. Remove copied installation files instead of renormalizing them. |
|
|
| The first extraction upload exposed another distinct failure: the Hub rejected LFS batch calls after the account reached **1,000 API requests per five minutes**. The attempt did not advance Git refs. Completed object uploads were reused on retry. Thousands of small files can exhaust request quotas even when bandwidth and disk usage are modest. Allow the rate-limit window to reset before retrying, reduce per-command transfer concurrency (for example `-c lfs.concurrenttransfers=2`), and retain an ignored diagnostic log. Do not repeatedly retry immediately or infer success from progress reaching “done”; check the Git command's exit status and remote ref. See the [Hub rate-limit documentation](https://huggingface.co/docs/hub/rate-limits). |
|
|
| ## Original payloads and remote history cleanup |
|
|
| Deleting files in a normal commit does not remove their earlier pointers or stored LFS/Xet objects. Hugging Face supports targeted [permanent LFS object deletion with history rewriting](https://huggingface.co/docs/huggingface_hub/en/package_reference/hf_api#huggingface_hub.HfApi.permanently_delete_lfs_files); object identity is the SHA-256, and deletion affects every reference to that object. This needs authenticated Hub access in addition to SSH Git access. The repository's Settings → List Large Files also exposes this operation. |
|
|
| The cleanup selects exactly the 39 source installation hashes from the snapshot, preserves the repository's asset README and code/docs, and checks that no extracted object shares those hashes. It avoids squashing unrelated source/documentation history. After server-side rewriting, fetch the remote's rewritten main and base the extraction commit on it so an old local parent does not reintroduce the removed pointers. Clones retaining the previous history must likewise resynchronize before pushing. |
|
|
| Execution evidence and final cleanup status are recorded in [the extraction research ledger](extraction/REMOTE_CLEANUP.md). Provider quota accounting can lag object cleanup; the Hub [storage documentation](https://huggingface.co/docs/hub/storage-limits) describes delays of up to 36 hours. Removing objects from this repository does not establish physical erasure of shared provider-level Xet chunks. |
|
|
