ChnIRuI111's picture
Add files using upload-large-folder tool
972b692 verified
|
Raw History Blame Contribute Delete
3.83 kB
---
language:
- en
tags:
- port
- machine-unlearning
- llm2vec
- text-classification
base_model:
- McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp
- McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised
license: llama3
---
# PoRT TOFU Post-Judgment Classifier
Recovered TOFU question-answer classifier checkpoint from training run 0815_1628, used by the PoRT pipeline.
Code: https://github.com/ChnIRuI/PoRT_LLM_Unlearning
## Important loading details
model.safetensors contains the encoder, adapter tensors, and the two-class MLP head. It is approximately 15.21 GB; despite the pipeline option name --classifier_head_ckpt, it is not a head-only checkpoint.
Use the SelectiveLLM2VecClassifier implementation in the project code. This repository is not a standard AutoModelForSequenceClassification export.
The constructor uses these upstream repositories:
- https://huggingface.co/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp
- https://huggingface.co/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised
Install the project requirements. From the root of the PoRT code checkout:
~~~python
import sys
from pathlib import Path
from huggingface_hub import snapshot_download
from safetensors.torch import load_file
sys.path.insert(0, str(Path("post_classifier/train").resolve()))
from train_classifier import SelectiveLLM2VecClassifier
model_dir = snapshot_download("ChnIRuI111/PoRT-TOFU-Classifier")
base_model = "McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp"
model = SelectiveLLM2VecClassifier(model_name=base_model)
state = load_file(str(Path(model_dir) / "model.safetensors"), device="cpu")
incompatible = model.load_state_dict(state, strict=False)
print("Missing keys:", incompatible.missing_keys)
print("Unexpected keys:", incompatible.unexpected_keys)
model.eval()
~~~
This follows the archived pipeline loading method. The constructor also initializes upstream models/adapters; those dependencies must be accessible. Check any missing or unexpected keys before running experiments.
Set the pipeline arguments:
~~~text
--classifier_base_model McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp
--classifier_head_ckpt <downloaded-model-directory>/model.safetensors
~~~
For a private repository, the downloading account must be authorized and logged in to Hugging Face.
## Provenance and validation
- Multiple archived TOFU, ablation, and threshold-study launch scripts explicitly reference this root checkpoint.
- Training records identify the TOFU v2.0 dataset, question-answer inputs, CCL-SC training, and run 0815_1628.
- Retained trainer state selects checkpoint-6000 as the best checkpoint; the retained training code loads the best model before exporting the root checkpoint. The intermediate checkpoint weights are no longer present for a byte-for-byte comparison.
- The safetensors header contains 742 tensors: 738 under encoder and 4 under classifier.
- MLP weight shapes are [2048, 4096] and [2, 2048], with matching biases.
- This is the TOFU checkpoint, not the WMDP fine-tuned classifier.
- File structure, header-derived size, and SHA-256 hashes were checked. Full inference and exact reproduction of the paper tables have not been rerun for this release.
SHA256SUMS identifies the uploaded payload files. Tokenizer files from the recovered export are included for reference; the project constructor obtains its tokenizer from the upstream base repository.
## Upstream licenses and attribution
Built with Meta Llama 3.
This checkpoint includes Llama 3-derived encoder weights. The Meta Llama 3 Community License and Acceptable Use Policy are included in LICENSE-META-LLAMA3 and USE_POLICY.md. The upstream LLM2Vec model cards list MIT licensing for their contributions; see the two linked upstream repositories. These notices do not replace the applicable upstream terms.