YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Evaluating: hellaswag_zeroshot (0-shot, type: multiple_choice)... accuracy: 0.5511 | centered: 0.4014 | time: 58.63s Evaluating: jeopardy (10-shot, type: language_modeling)... accuracy: 0.2518 | centered: 0.2518 | time: 12.50s Evaluating: bigbench_qa_wikidata (10-shot, type: language_modeling)... accuracy: 0.5716 | centered: 0.5716 | time: 119.92s Evaluating: arc_easy (10-shot, type: multiple_choice)... accuracy: 0.6860 | centered: 0.5814 | time: 14.57s Evaluating: arc_challenge (10-shot, type: multiple_choice)... accuracy: 0.3985 | centered: 0.1980 | time: 7.18s Evaluating: copa (0-shot, type: multiple_choice)... accuracy: 0.7000 | centered: 0.4000 | time: 0.58s Evaluating: commonsense_qa (10-shot, type: multiple_choice)... accuracy: 0.2473 | centered: 0.0592 | time: 7.57s Evaluating: piqa (10-shot, type: multiple_choice)... accuracy: 0.7323 | centered: 0.4646 | time: 11.02s Evaluating: openbook_qa (0-shot, type: multiple_choice)... accuracy: 0.3880 | centered: 0.1840 | time: 2.90s Evaluating: lambada_openai (0-shot, type: language_modeling)... accuracy: 0.4679 | centered: 0.4679 | time: 29.68s Evaluating: hellaswag (10-shot, type: multiple_choice)... accuracy: 0.5595 | centered: 0.4126 | time: 65.66s Evaluating: winograd (0-shot, type: schema)... accuracy: 0.7289 | centered: 0.4579 | time: 1.53s Evaluating: winogrande (0-shot, type: schema)... accuracy: 0.5699 | centered: 0.1397 | time: 7.12s Evaluating: bigbench_dyck_languages (10-shot, type: language_modeling)... accuracy: 0.1340 | centered: 0.1340 | time: 5.99s Evaluating: agi_eval_lsat_ar (3-shot, type: multiple_choice)... accuracy: 0.2130 | centered: 0.0163 | time: 1.46s Evaluating: bigbench_cs_algorithms (10-shot, type: language_modeling)... accuracy: 0.4114 | centered: 0.4114 | time: 7.78s Evaluating: bigbench_operators (10-shot, type: language_modeling)... accuracy: 0.2000 | centered: 0.2000 | time: 1.23s Evaluating: bigbench_repeat_copy_logic (10-shot, type: language_modeling)... accuracy: 0.0000 | centered: 0.0000 | time: 0.19s Evaluating: squad (10-shot, type: language_modeling)... accuracy: 0.2967 | centered: 0.2967 | time: 67.21s Evaluating: coqa (0-shot, type: language_modeling)... accuracy: 0.3024 | centered: 0.3024 | time: 47.20s Evaluating: boolq (10-shot, type: multiple_choice)... accuracy: 0.5361 | centered: -0.2208 | time: 21.41s Evaluating: bigbench_language_identification (10-shot, type: multiple_choice)... accuracy: 0.2500 | centered: 0.1749 | time: 77.11s