MMLU-by-task-Leaderboard / result_data_processor.py

Commit History

  1. Added clickable links (#1)⁠

    Corey
    Unverified signature
    59c6dd2

  1. moved organization column to the front

    Corey Morris
    7d69bda

  1. WIP. Loading data from csv

    Corey Morris
    1a1910c

  1. Changed error logging from print statements to logger. It is not currently working to save to a file locally

    Corey Morris
    d96fdf9
  2. Catching exceptions in processing files. As new data is introduced, I want to know which files may have different formats and cause problems, but the application shouldn't halt if it can't process a single file

    Corey Morris
    68bce52

  1. Returning just a single file per model directory. Manually removing gpt-j-6b for now because there is something that is causing problems with processing the data

    Corey Morris
    794b32b

  1. removing models that are known to have training data contaminated with evaluations

    Corey Morris
    a5840fb

  1. Fixed type error

    Corey Morris
    e79bcf3
  2. WIP commit. Currently have nlargest error

    Corey Morris
    d506f10
  3. Added organization to dataframe

    Corey Morris
    52d3b03
  4. removed code to print the number of outliers. could add it back later as logging potentially

    Corey Morris
    cd21f99
  5. MC1 column had 8 rows with a value of 1. It didn't make sense given the next highest value was 0.47 . Assuming they were data errors, they were removed

    Corey Morris
    e03b231
  6. truthfulqa data added to dataframe

    Corey Morris
    abac22e

  1. Refactor to make later code changes easier

    Corey Morris
    6d41115
  2. Added test for removal of undesired columns. fixed code error in column removal

    Corey Morris
    9549fcc

  1. Moved rank data into a separate method and dataframe

    Corey Morris
    31bed1a

  1. Extracting parameter data from the names of the models

    Corey Morris
    dcadab7

  1. Refactor. Extracted methods.

    Corey Morris
    ee5ac8e
  2. Refactoring. Moved ResultDataProcessor class to a separate file to make it easier to use with experimentation in a jupyter notebook

    Corey Morris
    843a5ef