[2026-06-15 23:59:32,808][trainer][INFO] - GradDiff Trainer loaded, output_dir: ./saves/unlearn/multilingual/books.all/chapters.all/GradDiff.NLL/de
[2026-06-16 01:01:47,085][lm_eval.models.huggingface][WARNING] - `pretrained` model kwarg is not of type `str`. Many other model arguments may be ignored. Please do not launch via accelerate or use `parallelize=True` if passing an existing model this way.
[2026-06-16 01:01:48,459][lm_eval.models.huggingface][WARNING] - Passed an already-initialized model through `pretrained`, assuming single-process call to evaluate() or custom distributed integration
[2026-06-16 01:01:48,461][lm_eval.evaluator][INFO] - Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
[2026-06-16 01:01:48,462][lm_eval.evaluator][INFO] - Using pre-initialized model
[2026-06-16 01:03:32,644][lm_eval.evaluator_utils][INFO] - Selected tasks:
[2026-06-16 01:03:32,648][lm_eval.evaluator_utils][INFO] - Group: mmlu
[2026-06-16 01:03:32,649][lm_eval.evaluator_utils][INFO] -   Group: mmlu_stem
[2026-06-16 01:03:32,650][lm_eval.evaluator_utils][INFO] -     Task: mmlu_abstract_algebra (mmlu/default/mmlu_abstract_algebra.yaml)
[2026-06-16 01:03:32,651][lm_eval.evaluator_utils][INFO] -     Task: mmlu_anatomy (mmlu/default/mmlu_anatomy.yaml)
[2026-06-16 01:03:32,652][lm_eval.evaluator_utils][INFO] -     Task: mmlu_astronomy (mmlu/default/mmlu_astronomy.yaml)
[2026-06-16 01:03:32,653][lm_eval.evaluator_utils][INFO] -     Task: mmlu_college_biology (mmlu/default/mmlu_college_biology.yaml)
[2026-06-16 01:03:32,653][lm_eval.evaluator_utils][INFO] -     Task: mmlu_college_chemistry (mmlu/default/mmlu_college_chemistry.yaml)
[2026-06-16 01:03:32,654][lm_eval.evaluator_utils][INFO] -     Task: mmlu_college_computer_science (mmlu/default/mmlu_college_computer_science.yaml)
[2026-06-16 01:03:32,655][lm_eval.evaluator_utils][INFO] -     Task: mmlu_college_mathematics (mmlu/default/mmlu_college_mathematics.yaml)
[2026-06-16 01:03:32,656][lm_eval.evaluator_utils][INFO] -     Task: mmlu_college_physics (mmlu/default/mmlu_college_physics.yaml)
[2026-06-16 01:03:32,657][lm_eval.evaluator_utils][INFO] -     Task: mmlu_computer_security (mmlu/default/mmlu_computer_security.yaml)
[2026-06-16 01:03:32,657][lm_eval.evaluator_utils][INFO] -     Task: mmlu_conceptual_physics (mmlu/default/mmlu_conceptual_physics.yaml)
[2026-06-16 01:03:32,658][lm_eval.evaluator_utils][INFO] -     Task: mmlu_electrical_engineering (mmlu/default/mmlu_electrical_engineering.yaml)
[2026-06-16 01:03:32,659][lm_eval.evaluator_utils][INFO] -     Task: mmlu_elementary_mathematics (mmlu/default/mmlu_elementary_mathematics.yaml)
[2026-06-16 01:03:32,659][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_biology (mmlu/default/mmlu_high_school_biology.yaml)
[2026-06-16 01:03:32,660][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_chemistry (mmlu/default/mmlu_high_school_chemistry.yaml)
[2026-06-16 01:03:32,661][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_computer_science (mmlu/default/mmlu_high_school_computer_science.yaml)
[2026-06-16 01:03:32,662][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_mathematics (mmlu/default/mmlu_high_school_mathematics.yaml)
[2026-06-16 01:03:32,662][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_physics (mmlu/default/mmlu_high_school_physics.yaml)
[2026-06-16 01:03:32,663][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_statistics (mmlu/default/mmlu_high_school_statistics.yaml)
[2026-06-16 01:03:32,664][lm_eval.evaluator_utils][INFO] -     Task: mmlu_machine_learning (mmlu/default/mmlu_machine_learning.yaml)
[2026-06-16 01:03:32,665][lm_eval.evaluator_utils][INFO] -   Group: mmlu_other
[2026-06-16 01:03:32,665][lm_eval.evaluator_utils][INFO] -     Task: mmlu_business_ethics (mmlu/default/mmlu_business_ethics.yaml)
[2026-06-16 01:03:32,666][lm_eval.evaluator_utils][INFO] -     Task: mmlu_clinical_knowledge (mmlu/default/mmlu_clinical_knowledge.yaml)
[2026-06-16 01:03:32,667][lm_eval.evaluator_utils][INFO] -     Task: mmlu_college_medicine (mmlu/default/mmlu_college_medicine.yaml)
[2026-06-16 01:03:32,667][lm_eval.evaluator_utils][INFO] -     Task: mmlu_global_facts (mmlu/default/mmlu_global_facts.yaml)
[2026-06-16 01:03:32,668][lm_eval.evaluator_utils][INFO] -     Task: mmlu_human_aging (mmlu/default/mmlu_human_aging.yaml)
[2026-06-16 01:03:32,669][lm_eval.evaluator_utils][INFO] -     Task: mmlu_management (mmlu/default/mmlu_management.yaml)
[2026-06-16 01:03:32,670][lm_eval.evaluator_utils][INFO] -     Task: mmlu_marketing (mmlu/default/mmlu_marketing.yaml)
[2026-06-16 01:03:32,670][lm_eval.evaluator_utils][INFO] -     Task: mmlu_medical_genetics (mmlu/default/mmlu_medical_genetics.yaml)
[2026-06-16 01:03:32,671][lm_eval.evaluator_utils][INFO] -     Task: mmlu_miscellaneous (mmlu/default/mmlu_miscellaneous.yaml)
[2026-06-16 01:03:32,672][lm_eval.evaluator_utils][INFO] -     Task: mmlu_nutrition (mmlu/default/mmlu_nutrition.yaml)
[2026-06-16 01:03:32,672][lm_eval.evaluator_utils][INFO] -     Task: mmlu_professional_accounting (mmlu/default/mmlu_professional_accounting.yaml)
[2026-06-16 01:03:32,673][lm_eval.evaluator_utils][INFO] -     Task: mmlu_professional_medicine (mmlu/default/mmlu_professional_medicine.yaml)
[2026-06-16 01:03:32,674][lm_eval.evaluator_utils][INFO] -     Task: mmlu_virology (mmlu/default/mmlu_virology.yaml)
[2026-06-16 01:03:32,674][lm_eval.evaluator_utils][INFO] -   Group: mmlu_social_sciences
[2026-06-16 01:03:32,675][lm_eval.evaluator_utils][INFO] -     Task: mmlu_econometrics (mmlu/default/mmlu_econometrics.yaml)
[2026-06-16 01:03:32,676][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_geography (mmlu/default/mmlu_high_school_geography.yaml)
[2026-06-16 01:03:32,676][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_government_and_politics (mmlu/default/mmlu_high_school_government_and_politics.yaml)
[2026-06-16 01:03:32,677][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_macroeconomics (mmlu/default/mmlu_high_school_macroeconomics.yaml)
[2026-06-16 01:03:32,678][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_microeconomics (mmlu/default/mmlu_high_school_microeconomics.yaml)
[2026-06-16 01:03:32,679][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_psychology (mmlu/default/mmlu_high_school_psychology.yaml)
[2026-06-16 01:03:32,679][lm_eval.evaluator_utils][INFO] -     Task: mmlu_human_sexuality (mmlu/default/mmlu_human_sexuality.yaml)
[2026-06-16 01:03:32,680][lm_eval.evaluator_utils][INFO] -     Task: mmlu_professional_psychology (mmlu/default/mmlu_professional_psychology.yaml)
[2026-06-16 01:03:32,680][lm_eval.evaluator_utils][INFO] -     Task: mmlu_public_relations (mmlu/default/mmlu_public_relations.yaml)
[2026-06-16 01:03:32,681][lm_eval.evaluator_utils][INFO] -     Task: mmlu_security_studies (mmlu/default/mmlu_security_studies.yaml)
[2026-06-16 01:03:32,682][lm_eval.evaluator_utils][INFO] -     Task: mmlu_sociology (mmlu/default/mmlu_sociology.yaml)
[2026-06-16 01:03:32,683][lm_eval.evaluator_utils][INFO] -     Task: mmlu_us_foreign_policy (mmlu/default/mmlu_us_foreign_policy.yaml)
[2026-06-16 01:03:32,683][lm_eval.evaluator_utils][INFO] -   Group: mmlu_humanities
[2026-06-16 01:03:32,684][lm_eval.evaluator_utils][INFO] -     Task: mmlu_formal_logic (mmlu/default/mmlu_formal_logic.yaml)
[2026-06-16 01:03:32,685][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_european_history (mmlu/default/mmlu_high_school_european_history.yaml)
[2026-06-16 01:03:32,685][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_us_history (mmlu/default/mmlu_high_school_us_history.yaml)
[2026-06-16 01:03:32,686][lm_eval.evaluator_utils][INFO] -     Task: mmlu_high_school_world_history (mmlu/default/mmlu_high_school_world_history.yaml)
[2026-06-16 01:03:32,687][lm_eval.evaluator_utils][INFO] -     Task: mmlu_international_law (mmlu/default/mmlu_international_law.yaml)
[2026-06-16 01:03:32,687][lm_eval.evaluator_utils][INFO] -     Task: mmlu_jurisprudence (mmlu/default/mmlu_jurisprudence.yaml)
[2026-06-16 01:03:32,688][lm_eval.evaluator_utils][INFO] -     Task: mmlu_logical_fallacies (mmlu/default/mmlu_logical_fallacies.yaml)
[2026-06-16 01:03:32,689][lm_eval.evaluator_utils][INFO] -     Task: mmlu_moral_disputes (mmlu/default/mmlu_moral_disputes.yaml)
[2026-06-16 01:03:32,689][lm_eval.evaluator_utils][INFO] -     Task: mmlu_moral_scenarios (mmlu/default/mmlu_moral_scenarios.yaml)
[2026-06-16 01:03:32,690][lm_eval.evaluator_utils][INFO] -     Task: mmlu_philosophy (mmlu/default/mmlu_philosophy.yaml)
[2026-06-16 01:03:32,691][lm_eval.evaluator_utils][INFO] -     Task: mmlu_prehistory (mmlu/default/mmlu_prehistory.yaml)
[2026-06-16 01:03:32,691][lm_eval.evaluator_utils][INFO] -     Task: mmlu_professional_law (mmlu/default/mmlu_professional_law.yaml)
[2026-06-16 01:03:32,692][lm_eval.evaluator_utils][INFO] -     Task: mmlu_world_religions (mmlu/default/mmlu_world_religions.yaml)
[2026-06-16 01:03:32,693][lm_eval.evaluator_utils][INFO] - Group: multilingual_unlearning
[2026-06-16 01:03:32,693][lm_eval.evaluator_utils][INFO] -   Task: multilingual_unlearning_harry_potter_en (multilingual_unlearning/multilingual_unlearning_harry_potter_en.yaml)
[2026-06-16 01:03:32,694][lm_eval.evaluator_utils][INFO] -   Task: multilingual_unlearning_harry_potter_fr (multilingual_unlearning/multilingual_unlearning_harry_potter_fr.yaml)
[2026-06-16 01:03:32,695][lm_eval.evaluator_utils][INFO] -   Task: multilingual_unlearning_harry_potter_ja (multilingual_unlearning/multilingual_unlearning_harry_potter_ja.yaml)
[2026-06-16 01:03:32,695][lm_eval.evaluator_utils][INFO] -   Task: multilingual_unlearning_harry_potter_vi (multilingual_unlearning/multilingual_unlearning_harry_potter_vi.yaml)
[2026-06-16 01:03:32,696][lm_eval.evaluator_utils][INFO] -   Task: multilingual_unlearning_harry_potter_zh (multilingual_unlearning/multilingual_unlearning_harry_potter_zh.yaml)
[2026-06-16 01:03:32,697][lm_eval.evaluator_utils][INFO] -   Task: multilingual_unlearning_harry_potter_ko (multilingual_unlearning/multilingual_unlearning_harry_potter_ko.yaml)
[2026-06-16 01:03:32,697][lm_eval.evaluator_utils][INFO] -   Task: multilingual_unlearning_harry_potter_es (multilingual_unlearning/multilingual_unlearning_harry_potter_es.yaml)
[2026-06-16 01:03:32,698][lm_eval.evaluator_utils][INFO] -   Task: multilingual_unlearning_harry_potter_de (multilingual_unlearning/multilingual_unlearning_harry_potter_de.yaml)
[2026-06-16 01:03:32,699][lm_eval.evaluator_utils][INFO] -   Task: multilingual_unlearning_harry_potter_th (multilingual_unlearning/multilingual_unlearning_harry_potter_th.yaml)
[2026-06-16 01:03:32,706][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_en on rank 0...
[2026-06-16 01:03:40,067][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_fr on rank 0...
[2026-06-16 01:03:46,226][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_ja on rank 0...
[2026-06-16 01:03:53,479][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_vi on rank 0...
[2026-06-16 01:03:59,871][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_zh on rank 0...
[2026-06-16 01:04:06,101][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_ko on rank 0...
[2026-06-16 01:04:12,773][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_es on rank 0...
[2026-06-16 01:04:18,865][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_de on rank 0...
[2026-06-16 01:04:24,935][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_th on rank 0...
[2026-06-16 01:04:31,166][lm_eval.api.task][INFO] - Building contexts for mmlu_abstract_algebra on rank 0...
[2026-06-16 01:04:31,272][lm_eval.api.task][INFO] - Building contexts for mmlu_anatomy on rank 0...
[2026-06-16 01:04:31,417][lm_eval.api.task][INFO] - Building contexts for mmlu_astronomy on rank 0...
[2026-06-16 01:04:31,580][lm_eval.api.task][INFO] - Building contexts for mmlu_college_biology on rank 0...
[2026-06-16 01:04:31,732][lm_eval.api.task][INFO] - Building contexts for mmlu_college_chemistry on rank 0...
[2026-06-16 01:04:31,839][lm_eval.api.task][INFO] - Building contexts for mmlu_college_computer_science on rank 0...
[2026-06-16 01:04:31,943][lm_eval.api.task][INFO] - Building contexts for mmlu_college_mathematics on rank 0...
[2026-06-16 01:04:32,047][lm_eval.api.task][INFO] - Building contexts for mmlu_college_physics on rank 0...
[2026-06-16 01:04:32,156][lm_eval.api.task][INFO] - Building contexts for mmlu_computer_security on rank 0...
[2026-06-16 01:04:32,260][lm_eval.api.task][INFO] - Building contexts for mmlu_conceptual_physics on rank 0...
[2026-06-16 01:04:32,504][lm_eval.api.task][INFO] - Building contexts for mmlu_electrical_engineering on rank 0...
[2026-06-16 01:04:32,655][lm_eval.api.task][INFO] - Building contexts for mmlu_elementary_mathematics on rank 0...
[2026-06-16 01:04:33,589][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_biology on rank 0...
[2026-06-16 01:04:33,908][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_chemistry on rank 0...
[2026-06-16 01:04:34,118][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_computer_science on rank 0...
[2026-06-16 01:04:34,225][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_mathematics on rank 0...
[2026-06-16 01:04:34,502][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_physics on rank 0...
[2026-06-16 01:04:34,659][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_statistics on rank 0...
[2026-06-16 01:04:34,880][lm_eval.api.task][INFO] - Building contexts for mmlu_machine_learning on rank 0...
[2026-06-16 01:04:34,998][lm_eval.api.task][INFO] - Building contexts for mmlu_business_ethics on rank 0...
[2026-06-16 01:04:35,101][lm_eval.api.task][INFO] - Building contexts for mmlu_clinical_knowledge on rank 0...
[2026-06-16 01:04:35,373][lm_eval.api.task][INFO] - Building contexts for mmlu_college_medicine on rank 0...
[2026-06-16 01:04:35,551][lm_eval.api.task][INFO] - Building contexts for mmlu_global_facts on rank 0...
[2026-06-16 01:04:35,657][lm_eval.api.task][INFO] - Building contexts for mmlu_human_aging on rank 0...
[2026-06-16 01:04:35,886][lm_eval.api.task][INFO] - Building contexts for mmlu_management on rank 0...
[2026-06-16 01:04:35,994][lm_eval.api.task][INFO] - Building contexts for mmlu_marketing on rank 0...
[2026-06-16 01:04:36,236][lm_eval.api.task][INFO] - Building contexts for mmlu_medical_genetics on rank 0...
[2026-06-16 01:04:36,340][lm_eval.api.task][INFO] - Building contexts for mmlu_miscellaneous on rank 0...
[2026-06-16 01:04:37,147][lm_eval.api.task][INFO] - Building contexts for mmlu_nutrition on rank 0...
[2026-06-16 01:04:37,461][lm_eval.api.task][INFO] - Building contexts for mmlu_professional_accounting on rank 0...
[2026-06-16 01:04:37,754][lm_eval.api.task][INFO] - Building contexts for mmlu_professional_medicine on rank 0...
[2026-06-16 01:04:38,037][lm_eval.api.task][INFO] - Building contexts for mmlu_virology on rank 0...
[2026-06-16 01:04:38,210][lm_eval.api.task][INFO] - Building contexts for mmlu_econometrics on rank 0...
[2026-06-16 01:04:38,332][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_geography on rank 0...
[2026-06-16 01:04:38,540][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_government_and_politics on rank 0...
[2026-06-16 01:04:38,741][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_macroeconomics on rank 0...
[2026-06-16 01:04:39,146][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_microeconomics on rank 0...
[2026-06-16 01:04:39,391][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_psychology on rank 0...
[2026-06-16 01:04:39,954][lm_eval.api.task][INFO] - Building contexts for mmlu_human_sexuality on rank 0...
[2026-06-16 01:04:40,091][lm_eval.api.task][INFO] - Building contexts for mmlu_professional_psychology on rank 0...
[2026-06-16 01:04:40,724][lm_eval.api.task][INFO] - Building contexts for mmlu_public_relations on rank 0...
[2026-06-16 01:04:40,840][lm_eval.api.task][INFO] - Building contexts for mmlu_security_studies on rank 0...
[2026-06-16 01:04:41,096][lm_eval.api.task][INFO] - Building contexts for mmlu_sociology on rank 0...
[2026-06-16 01:04:41,304][lm_eval.api.task][INFO] - Building contexts for mmlu_us_foreign_policy on rank 0...
[2026-06-16 01:04:41,409][lm_eval.api.task][INFO] - Building contexts for mmlu_formal_logic on rank 0...
[2026-06-16 01:04:41,540][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_european_history on rank 0...
[2026-06-16 01:04:41,716][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_us_history on rank 0...
[2026-06-16 01:04:41,933][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_world_history on rank 0...
[2026-06-16 01:04:42,184][lm_eval.api.task][INFO] - Building contexts for mmlu_international_law on rank 0...
[2026-06-16 01:04:42,311][lm_eval.api.task][INFO] - Building contexts for mmlu_jurisprudence on rank 0...
[2026-06-16 01:04:42,427][lm_eval.api.task][INFO] - Building contexts for mmlu_logical_fallacies on rank 0...
[2026-06-16 01:04:42,597][lm_eval.api.task][INFO] - Building contexts for mmlu_moral_disputes on rank 0...
[2026-06-16 01:04:42,954][lm_eval.api.task][INFO] - Building contexts for mmlu_moral_scenarios on rank 0...
[2026-06-16 01:04:43,892][lm_eval.api.task][INFO] - Building contexts for mmlu_philosophy on rank 0...
[2026-06-16 01:04:44,214][lm_eval.api.task][INFO] - Building contexts for mmlu_prehistory on rank 0...
[2026-06-16 01:04:44,551][lm_eval.api.task][INFO] - Building contexts for mmlu_professional_law on rank 0...
[2026-06-16 01:04:46,141][lm_eval.api.task][INFO] - Building contexts for mmlu_world_religions on rank 0...
[2026-06-16 01:04:46,317][lm_eval.evaluator][INFO] - Running loglikelihood requests