Upload PAG checkpoint, document identifiers, and Lost-in-Decoding evaluation artifacts

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+identifiers/rq_docids.json filter=lfs diff=lfs merge=lfs -text
+identifiers/set_docids.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,116 @@
+---
+license: apache-2.0
+tags:
+  - generative-retrieval
+  - information-retrieval
+  - t5
+  - msmarco
+  - robustness
+  - reproducibility
+---
+
+# Lost in Decoding? — PAG Checkpoint & Evaluation Artifacts
+
+This repository hosts the model checkpoint, document identifiers, and evaluation
+artifacts used by:
+
+> **Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in
+> Generative Retrieval** (SIGIR 2026 Reproducibility Track, submission #298)
+> [arXiv:2604.23396](https://arxiv.org/abs/2604.23396) ·
+> [Code](https://github.com/kidist-amde/lost-in-decoding)
+
+It is a **reproducibility and robustness study** of the PAG (Planner-Assisted
+Generative retrieval) system. It does not introduce a new trained model — the
+checkpoint and document identifiers below are the **original artifacts released
+by the PAG authors**, re-hosted here for convenience and long-term availability.
+The perturbed-query evaluation sets are **new artifacts created for this paper**.
+
+## What's in this repository, and who made it
+
+| Artifact | Created by | Description |
+|---|---|---|
+| `pytorch_model.bin`, `config.json`, tokenizer files | **PAG authors** (Zeng & Zamani, 2024) | Original PAG checkpoint (`ripor_direct_lng_knp_seq2seq_1`), unmodified. |
+| `identifiers/rq_docids.json` | **PAG authors** | Sequential ("RQ"/AQ) document identifiers (`aq_smtid/docid_to_tokenids.json` in the upstream release). |
+| `identifiers/set_docids.json` | **PAG authors** | Set-based ("lexical"/bag-of-words) document identifiers (`top_bow/docid_to_tokenids.json` in the upstream release). |
+| `lost_in_decoding_artifacts/perturbed_queries/` | **This paper (Lost in Decoding)** | Perturbed MS MARCO Dev queries (misspelling, reordering, synonym replacement, paraphrasing, naturality normalization) used for the RQ2 robustness evaluation. |
+| `lost_in_decoding_artifacts/metadata/` | **This paper (Lost in Decoding)** | Metadata describing generation settings (seeds, attack methods) for the perturbed-query sets. |
+
+Multilingual (mMARCO) queries used for RQ3 are **not** re-hosted here; they are
+third-party data. See the *Source data* section below.
+
+## Model
+
+- **Architecture:** `T5ForLexicalSemanticGeneration`, 12-layer encoder/decoder, 768 hidden size (T5-base scale).
+- **Format:** `pytorch_model.bin` (`transformers` 4.17 era), not safetensors — uploaded as originally released.
+- **Task:** Generative document retrieval over MS MARCO, decoding a two-stage (set-based + sequential) document identifier per query.
+
+```python
+from transformers import AutoTokenizer, AutoConfig
+# Model class (T5ForLexicalSemanticGeneration) is defined in the Lost-in-Decoding /
+# upstream PAG codebase, not in transformers. Load config/tokenizer directly,
+# and use the repository's modeling code to load the checkpoint:
+# https://github.com/kidist-amde/lost-in-decoding/blob/main/t5_pretrainer/modeling/t5_generative_retriever.py
+
+tokenizer = AutoTokenizer.from_pretrained("kiyam/lost-in-decoding-pag")
+config = AutoConfig.from_pretrained("kiyam/lost-in-decoding-pag")
+```
+
+See the [Lost-in-Decoding repository](https://github.com/kidist-amde/lost-in-decoding)
+for the full two-stage constrained-decoding pipeline (lexical planning →
+sequential constrained beam search) required to actually run retrieval with
+this checkpoint.
+
+## Document identifiers (`identifiers/`)
+
+Both files map `docid (str) -> list[int]` (token ID sequences forming each
+document's identifier). See [`identifiers/README.md`](identifiers/README.md)
+for details.
+
+## Evaluation artifacts (`lost_in_decoding_artifacts/`)
+
+Perturbed MS MARCO Dev queries used in the RQ2 robustness evaluation
+(Table 6 in the paper, corrected — see the main repository README for the
+correction notice). See
+[`lost_in_decoding_artifacts/perturbed_queries/README.md`](lost_in_decoding_artifacts/README.md)
+for format and generation details.
+
+## Source data
+
+- **MS MARCO** passage ranking dataset (queries, qrels, collection): [microsoft/MSMARCO](https://microsoft.github.io/msmarco/) — used under its original terms.
+- **TREC Deep Learning 2019 / 2020** query sets and qrels: [NIST TREC-DL](https://microsoft.github.io/msmarco/TREC-Deep-Learning.html).
+- **mMARCO** (multilingual queries, RQ3): [unicamp-dl/mmarco](https://huggingface.co/datasets/unicamp-dl/mmarco) — not redistributed here; download via `cross_lingual/scripts/download_mmarco.sh` in the code repository.
+
+## Citation
+
+If you use the perturbed-query artifacts or the RQ2/RQ3 evaluation results, please cite:
+
+```bibtex
+@article{lost-in-decoding-2026,
+  title   = {Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval},
+  journal = {arXiv preprint arXiv:2604.23396},
+  year    = {2026},
+  note    = {SIGIR 2026 Reproducibility Track, submission \#298}
+}
+```
+
+If you use the PAG checkpoint or document identifiers, please also cite the
+original PAG paper:
+
+```bibtex
+@article{zeng2024pag,
+  title   = {Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding},
+  author  = {Zeng, Hansi and Zamani, Hamed},
+  journal = {arXiv preprint arXiv:2404.14600},
+  year    = {2024}
+}
+```
+
+- PAG paper: [arXiv:2404.14600](https://arxiv.org/pdf/2404.14600)
+- PAG upstream repository: [github.com/HansiZeng/PAG](https://github.com/HansiZeng/PAG/tree/main)
+
+## License
+
+Apache 2.0, matching the [Lost-in-Decoding repository license](https://github.com/kidist-amde/lost-in-decoding/blob/main/LICENSE).
+The PAG checkpoint and document identifiers are redistributed under the same
+terms as the upstream PAG repository; consult the upstream repository for its
+original license terms.

args.json

@@ -0,0 +1,69 @@
+{
+    "model_name_or_path": "t5-base",
+    "num_decoder_layers": null,
+    "teacher_score_path": "\/home\/ec2-user\/quic-efs\/user\/hansizeng\/work\/term_generative_retriever\/experiments-full-lexical-ripor\/t5-full-dense-1-5e-4-12l\/out\/MSMARCO_TRAIN\/qrel_added_merged_teacher_scores.json",
+    "collection_path": "\/home\/ec2-user\/quic-efs\/user\/hansizeng\/work\/data\/msmarco-full\/full_collection\/",
+    "queries_path": "\/home\/ec2-user\/quic-efs\/user\/hansizeng\/work\/data\/msmarco-full\/all_train_queries\/train_queries",
+    "qrels_path": "\/home\/ec2-user\/quic-efs\/user\/hansizeng\/work\/data\/msmarco\/train_queries\/qrels.json",
+    "output_dir": "\/home\/ec2-user\/quic-efs\/user\/hansizeng\/work\/term_generative_retriever\/experiments-full-lexical-ripor\/lexical_ripor_direct_lng_knp_seq2seq_1\/checkpoint",
+    "example_path": null,
+    "pseudo_queries_to_docid_path": null,
+    "pseudo_queries_to_mul_docid_path": null,
+    "docid_to_smtid_path": "",
+    "docid_to_tokenids_path": null,
+    "qid_to_smtid_path": null,
+    "first_centroid_path": null,
+    "second_centroid_path": null,
+    "third_centroid_path": null,
+    "qid_to_rrpids_path": null,
+    "docid_decode_eval_path": null,
+    "centroid_path": null,
+    "centroid_idx": null,
+    "triple_margin_mse_path": null,
+    "query_to_docid_path": null,
+    "teacher_rerank_nway_path": null,
+    "bce_example_path": null,
+    "smt_docid_to_smtid_path": "\/home\/ec2-user\/quic-efs\/user\/hansizeng\/work\/term_generative_retriever\/experiments-full-lexical-ripor\/t5-full-dense-1-5e-4-12l\/aq_smtid\/docid_to_tokenids.json",
+    "lex_docid_to_smtid_path": "\/home\/ec2-user\/quic-efs\/user\/hansizeng\/work\/term_generative_retriever\/experiments-splade\/t5-splade-0-12l\/\/top_bow\/docid_to_tokenids.json",
+    "run_name": "lexical_ripor_direct_lng_knp_seq2seq_1",
+    "pretrained_path": "\/home\/ec2-user\/quic-efs\/user\/hansizeng\/work\/term_generative_retriever\/experiments-full-lexical-ripor\/ripor_direct_lng_knp_seq2seq_1\/checkpoint\/",
+    "loss_type": "direct_lng_knp_margin_mse",
+    "model_type": "lexical_ripor",
+    "do_eval": false,
+    "max_length": 64,
+    "learning_rate": 0.0005,
+    "warmup_ratio": 0.04,
+    "per_device_train_batch_size": 64,
+    "logging_steps": 50,
+    "max_steps": -1,
+    "epochs": 120,
+    "local_rank": 0,
+    "task_names": [
+        "rank_4",
+        "rank",
+        "lexical_rank"
+    ],
+    "ln_to_weight": {
+        "rank_4": 1.0,
+        "rank": 1.0,
+        "lexical_rank": 1.0
+    },
+    "multi_weights": null,
+    "nway_label_type": null,
+    "nway_rrpids": 24,
+    "nway": 12,
+    "eval_steps": 50,
+    "use_fp16": true,
+    "multi_vocab_sizes": false,
+    "save_steps": 15000,
+    "wandb_project_name": "full_lexical_ripor",
+    "pad_token_id": null,
+    "smtid_as_docid": false,
+    "apply_lex_loss": false,
+    "eval_collection_path": null,
+    "eval_queries_path": null,
+    "full_rank_eval_qrel_path": null,
+    "full_rank_eval_topk": 200,
+    "full_rank_index_dir": null,
+    "full_rank_out_dir": null
+}

config.json

@@ -0,0 +1,57 @@
+{
+  "_name_or_path": "/home/ec2-user/quic-efs/user/hansizeng/work/term_generative_retriever/experiments-full-lexical-ripor/ripor_direct_lng_knp_seq2seq_1/checkpoint/",
+  "architectures": [
+    "T5ForLexicalSemanticGeneration"
+  ],
+  "d_ff": 3072,
+  "d_kv": 64,
+  "d_model": 768,
+  "decoder_start_token_id": 0,
+  "dropout_rate": 0.1,
+  "eos_token_id": 1,
+  "feed_forward_proj": "relu",
+  "initializer_factor": 1.0,
+  "is_encoder_decoder": true,
+  "layer_norm_epsilon": 1e-06,
+  "model_type": "t5",
+  "n_positions": 512,
+  "num_decoder_layers": 12,
+  "num_heads": 12,
+  "num_layers": 12,
+  "output_past": true,
+  "pad_token_id": 0,
+  "relative_attention_num_buckets": 32,
+  "task_specific_params": {
+    "summarization": {
+      "early_stopping": true,
+      "length_penalty": 2.0,
+      "max_length": 200,
+      "min_length": 30,
+      "no_repeat_ngram_size": 3,
+      "num_beams": 4,
+      "prefix": "summarize: "
+    },
+    "translation_en_to_de": {
+      "early_stopping": true,
+      "max_length": 300,
+      "num_beams": 4,
+      "prefix": "translate English to German: "
+    },
+    "translation_en_to_fr": {
+      "early_stopping": true,
+      "max_length": 300,
+      "num_beams": 4,
+      "prefix": "translate English to French: "
+    },
+    "translation_en_to_ro": {
+      "early_stopping": true,
+      "max_length": 300,
+      "num_beams": 4,
+      "prefix": "translate English to Romanian: "
+    }
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.17.0",
+  "use_cache": true,
+  "vocab_size": 48484
+}

identifiers/README.md

@@ -0,0 +1,43 @@
+# Document Identifiers
+
+**Provenance:** created by the PAG authors (Zeng & Zamani, 2024) and released
+in the [upstream PAG repository](https://github.com/HansiZeng/PAG). Re-hosted
+here unmodified.
+
+Both files are JSON objects mapping `docid (str) -> list[int]`, where the list
+is the sequence of token IDs that make up that document's generative
+identifier under the PAG tokenizer/vocabulary.
+
+## `rq_docids.json`
+
+Sequential document identifiers produced by PAG's residual-quantization (AQ)
+stage. Corresponds to `aq_smtid/docid_to_tokenids.json` in the upstream
+release. Used by the sequential constrained-decoding stage of the PAG
+pipeline (Stage 2).
+
+## `set_docids.json`
+
+Set-based (bag-of-words) document identifiers produced by PAG's lexical/SPLADE
+planning stage. Corresponds to `top_bow/docid_to_tokenids.json` in the
+upstream release. Used by the lexical planning stage of the PAG pipeline
+(Stage 1).
+
+## Usage
+
+See the two-stage constrained-decoding pipeline in the
+[Lost-in-Decoding repository](https://github.com/kidist-amde/lost-in-decoding)
+(`t5_pretrainer/evaluate.py`) for how these identifier files are consumed
+during retrieval.
+
+## Citation
+
+If you use these identifiers, please cite the original PAG paper:
+
+```bibtex
+@article{zeng2024pag,
+  title   = {Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding},
+  author  = {Zeng, Hansi and Zamani, Hamed},
+  journal = {arXiv preprint arXiv:2404.14600},
+  year    = {2024}
+}
+```

identifiers/rq_docids.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe2b082d95316560bc8637d36d96e32f5559712602dd863db1ad21e07c5b0f56
+size 529398271

identifiers/set_docids.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29e6b3b0cb74db6dcbc34f5f7e9cb6c928e60e0a381bd460da5de5a7d7a624b2
+size 2970761161

lost_in_decoding_artifacts/README.md

@@ -0,0 +1,53 @@
+# Lost in Decoding — Evaluation Artifacts
+
+**Provenance:** created for this paper (*Lost in Decoding? Reproducing and
+Stress-Testing the Look-Ahead Prior in Generative Retrieval*,
+[arXiv:2604.23396](https://arxiv.org/abs/2604.23396)), not part of the
+original PAG release.
+
+## `perturbed_queries/`
+
+Perturbed MS MARCO Dev query sets used for the RQ2 robustness evaluation
+(query-perturbation sensitivity of PAG's look-ahead prior).
+
+- **Split:** MS MARCO Dev (6980 queries).
+- **Attack methods:** `mispelling`, `ordering` (word reordering), `synonym`
+  (synonym replacement), `paraphrase`, `naturality` (naturality
+  normalization). Implemented in
+  [`robustness/query_variations/penha/`](https://github.com/kidist-amde/lost-in-decoding/tree/main/robustness/query_variations/penha)
+  in the code repository.
+- **Seeds:** `1999`, `5`, `27`, `2016`, `2026`.
+- **File naming:** `msmarco_test_attacked_queries_seed_{seed}_attack_method_{attack_method}.json`
+- **Format:** JSON object, `{qid (str): perturbed_query_text (str)}`.
+- **Note (`naturality`):** this transformation is a deterministic
+  style-normalization rule rather than a randomized perturbation, so the five
+  seed files for `naturality` are identical to each other by construction.
+
+**TREC-DL 2019 / TREC-DL 2020 perturbed queries are not included as static
+files.** They are produced by the same code path
+(`robustness/scripts/generate_query_variations_sub.sh`) using the same seeds
+and attack methods, and were consumed directly during RQ2 evaluation without
+being cached to disk. Regenerate them with:
+
+```bash
+bash robustness/scripts/generate_query_variations_sub.sh
+```
+
+## `metadata/`
+
+`perturbed_queries_metadata.json` documents the generation settings (split,
+seeds, attack methods, file naming convention) for the released perturbed
+query sets.
+
+## Citation
+
+If you use these artifacts, please cite:
+
+```bibtex
+@article{lost-in-decoding-2026,
+  title   = {Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval},
+  journal = {arXiv preprint arXiv:2604.23396},
+  year    = {2026},
+  note    = {SIGIR 2026 Reproducibility Track, submission \#298}
+}
+```

lost_in_decoding_artifacts/metadata/perturbed_queries_metadata.json

@@ -0,0 +1,19 @@
+{
+  "description": "Metadata for the perturbed MS MARCO Dev query sets released alongside 'Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval' (SIGIR 2026, submission #298).",
+  "split": "msmarco_dev",
+  "num_queries_per_file": 6980,
+  "attack_methods": [
+    "mispelling",
+    "ordering",
+    "synonym",
+    "paraphrase",
+    "naturality"
+  ],
+  "seeds": [1999, 5, 27, 2016, 2026],
+  "file_naming": "msmarco_test_attacked_queries_seed_{seed}_attack_method_{attack_method}.json",
+  "format": "JSON object mapping qid (string) -> perturbed query text (string)",
+  "generation_code": "robustness/query_variations (penha transformations) in the Lost-in-Decoding repository",
+  "note": "TREC-DL 2019 and TREC-DL 2020 perturbed queries are not included as static files here. They are generated on demand by the same code path (robustness/scripts/generate_query_variations_sub.sh) using the same seeds and attack methods, and were consumed directly by the RQ2 evaluation pipeline without being cached to disk.",
+  "source_paper": "Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval (arXiv:2604.23396)",
+  "source_repository": "https://github.com/kidist-amde/lost-in-decoding"
+}

model_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c71b3f8892a3ca56cac5615e641aa34a8f2500bbebbdbc7406d99166dad49555
+size 431

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9069588676ced21d94df2234690a93aaafa9bac4b2a4018b11c11874c480c4a1
+size 941976511

special_tokens_map.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "additional_special_tokens": ["<extra_id_0>", "<extra_id_1>", "<extra_id_2>", "<extra_id_3>", "<extra_id_4>", "<extra_id_5>", "<extra_id_6>", "<extra_id_7>", "<extra_id_8>", "<extra_id_9>", "<extra_id_10>", "<extra_id_11>", "<extra_id_12>", "<extra_id_13>", "<extra_id_14>", "<extra_id_15>", "<extra_id_16>", "<extra_id_17>", "<extra_id_18>", "<extra_id_19>", "<extra_id_20>", "<extra_id_21>", "<extra_id_22>", "<extra_id_23>", "<extra_id_24>", "<extra_id_25>", "<extra_id_26>", "<extra_id_27>", "<extra_id_28>", "<extra_id_29>", "<extra_id_30>", "<extra_id_31>", "<extra_id_32>", "<extra_id_33>", "<extra_id_34>", "<extra_id_35>", "<extra_id_36>", "<extra_id_37>", "<extra_id_38>", "<extra_id_39>", "<extra_id_40>", "<extra_id_41>", "<extra_id_42>", "<extra_id_43>", "<extra_id_44>", "<extra_id_45>", "<extra_id_46>", "<extra_id_47>", "<extra_id_48>", "<extra_id_49>", "<extra_id_50>", "<extra_id_51>", "<extra_id_52>", "<extra_id_53>", "<extra_id_54>", "<extra_id_55>", "<extra_id_56>", "<extra_id_57>", "<extra_id_58>", "<extra_id_59>", "<extra_id_60>", "<extra_id_61>", "<extra_id_62>", "<extra_id_63>", "<extra_id_64>", "<extra_id_65>", "<extra_id_66>", "<extra_id_67>", "<extra_id_68>", "<extra_id_69>", "<extra_id_70>", "<extra_id_71>", "<extra_id_72>", "<extra_id_73>", "<extra_id_74>", "<extra_id_75>", "<extra_id_76>", "<extra_id_77>", "<extra_id_78>", "<extra_id_79>", "<extra_id_80>", "<extra_id_81>", "<extra_id_82>", "<extra_id_83>", "<extra_id_84>", "<extra_id_85>", "<extra_id_86>", "<extra_id_87>", "<extra_id_88>", "<extra_id_89>", "<extra_id_90>", "<extra_id_91>", "<extra_id_92>", "<extra_id_93>", "<extra_id_94>", "<extra_id_95>", "<extra_id_96>", "<extra_id_97>", "<extra_id_98>", "<extra_id_99>"]}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d60acb128cf7b7f2536e8f38a5b18a05535c9e14c7a355904270e15b0945ea86
+size 791656

tokenizer_config.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids": 100, "additional_special_tokens": ["<extra_id_0>", "<extra_id_1>", "<extra_id_2>", "<extra_id_3>", "<extra_id_4>", "<extra_id_5>", "<extra_id_6>", "<extra_id_7>", "<extra_id_8>", "<extra_id_9>", "<extra_id_10>", "<extra_id_11>", "<extra_id_12>", "<extra_id_13>", "<extra_id_14>", "<extra_id_15>", "<extra_id_16>", "<extra_id_17>", "<extra_id_18>", "<extra_id_19>", "<extra_id_20>", "<extra_id_21>", "<extra_id_22>", "<extra_id_23>", "<extra_id_24>", "<extra_id_25>", "<extra_id_26>", "<extra_id_27>", "<extra_id_28>", "<extra_id_29>", "<extra_id_30>", "<extra_id_31>", "<extra_id_32>", "<extra_id_33>", "<extra_id_34>", "<extra_id_35>", "<extra_id_36>", "<extra_id_37>", "<extra_id_38>", "<extra_id_39>", "<extra_id_40>", "<extra_id_41>", "<extra_id_42>", "<extra_id_43>", "<extra_id_44>", "<extra_id_45>", "<extra_id_46>", "<extra_id_47>", "<extra_id_48>", "<extra_id_49>", "<extra_id_50>", "<extra_id_51>", "<extra_id_52>", "<extra_id_53>", "<extra_id_54>", "<extra_id_55>", "<extra_id_56>", "<extra_id_57>", "<extra_id_58>", "<extra_id_59>", "<extra_id_60>", "<extra_id_61>", "<extra_id_62>", "<extra_id_63>", "<extra_id_64>", "<extra_id_65>", "<extra_id_66>", "<extra_id_67>", "<extra_id_68>", "<extra_id_69>", "<extra_id_70>", "<extra_id_71>", "<extra_id_72>", "<extra_id_73>", "<extra_id_74>", "<extra_id_75>", "<extra_id_76>", "<extra_id_77>", "<extra_id_78>", "<extra_id_79>", "<extra_id_80>", "<extra_id_81>", "<extra_id_82>", "<extra_id_83>", "<extra_id_84>", "<extra_id_85>", "<extra_id_86>", "<extra_id_87>", "<extra_id_88>", "<extra_id_89>", "<extra_id_90>", "<extra_id_91>", "<extra_id_92>", "<extra_id_93>", "<extra_id_94>", "<extra_id_95>", "<extra_id_96>", "<extra_id_97>", "<extra_id_98>", "<extra_id_99>"], "model_max_length": 512, "special_tokens_map_file": null, "name_or_path": "t5-base", "tokenizer_class": "T5Tokenizer"}