deceit1 / evaluate.py

Commit History

  1. improve: abstention penalty, better prompt, mixed curriculum, more steps

    Jayant-Kernel
    253d1ff
  2. evaluate: switch to 0.5B model comparison, 200 episodes

    Jayant-Kernel
    6b64fd2
  3. fix: set N_EPISODES=200 constant (was still 30)

    Jayant-Kernel
    e662a77

  1. update: increase evaluation to 200 episodes per model

    Jayant-Kernel
    a5be204
  2. fix: parse_action confidence bug, numeric answers bug, missing reasoning field bug

    Jayant-Kernel
    66bdd16
  3. fix: debug model output parsing in evaluation

    Jayant-Kernel
    3d9195a
  4. add: evaluate 1.5B base vs trained, upload chart to HF Hub

    Jayant-Kernel
    77e0352
  5. add: evaluate 1.5B base vs trained, upload charts

    Jayant-Kernel
    68e5af2
  6. update: compare 0.5B vs 1.5B trained models

    Jayant-Kernel
    0bdaeb6
  7. update: evaluate retrained model, upload charts to HF Hub

    Jayant-Kernel
    Unverified signature
    b84ec51
  8. update: evaluate on 200 episodes each for more reliable results

    Jayant-Kernel
    Unverified signature
    a178a66
  9. fix: remove erroneous del model outside evaluate_model scope

    Jayant-Kernel
    Unverified signature
    c6e06ad
  10. fix: free GPU memory between model evaluations

    Jayant-Kernel
    Unverified signature
    4c67564
  11. add: evaluation script - base vs trained model comparison

    Jayant-Kernel
    Unverified signature
    8fb443c