chronos-api-backend / benchmark

Commit History

  1. fix: full Supabase integration audit and cleanup⁠

    RemanenetSpy
    754e261

  1. feat(kaggle): dynamically set vLLM tensor_parallel_size to use all available GPUs (e.g. Kaggle's T4x2) for massive speedup

    RemanenetSpy
    5bfcabd
  2. fix(kaggle): print progress on every case and incrementally save results to disk to prevent total data loss if Kaggle hits 12-hour timeout

    RemanenetSpy
    20cebf8
  3. fix(kaggle): replace .format() with .replace() to avoid KeyError on JSON braces inside the prompt

    RemanenetSpy
    df608c2
  4. fix(kaggle): cast all dataset fields to strings to prevent AttributeError on integer answers

    RemanenetSpy
    a122973
  5. fix(kaggle): handle missing 'id' and 'category' keys in LongMemEval dataset cases gracefully to prevent KeyError

    RemanenetSpy
    3cfa7ac
  6. fix(kaggle): forcefully remove torchcodec after pip installs to prevent it being re-installed as a dependency and crashing SentenceTransformers

    RemanenetSpy
    11923ea
  7. fix(kaggle): forcefully remove broken torchcodec directory to prevent SentenceTransformers from crashing during initialization

    RemanenetSpy
    62b9da3
  8. fix(kaggle): unpin vllm to support Kaggle Python 3.12 update while keeping torchcodec uninstall

    RemanenetSpy
    08c70ee

  1. fix(kaggle): downgrade vllm to 0.5.1 and remove torchcodec to bypass CUDA missing library error

    RemanenetSpy
    e155204
  2. chore(benchmark): switch Kaggle notebook to ungated Qwen2.5-7B-Instruct to avoid license blocks

    RemanenetSpy
    3bf9a1a
  3. fix(kaggle): correct dataset path and filename casing for Kaggle Linux filesystem

    RemanenetSpy
    2040e1d
  4. fix(benchmark): run ALL cases for both S and M splits, save results JSON + BENCHMARKS.md

    RemanenetSpy
    91236f3
  5. update kaggle notebook to run both S and M splits sequentially

    RemanenetSpy
    95b8018
  6. feat(benchmark): add kaggle notebook for real smriti algorithm using local Llama 3 8B via vLLM

    RemanenetSpy
    80a537b

  1. fix(kaggle): remove hardcoded dataSources metadata to fix import error

    RemanenetSpy
    1ae0e85
  2. fix(kaggle): update dataset path to match user's custom kaggle dataset output

    RemanenetSpy
    cdb2a83
  3. feat: add Kaggle notebook for LongMemEval S+M benchmark evaluation

    RemanenetSpy
    fd13692

  1. docs: add LongMemEval evaluation and BENCHMARKS.md leaderboard strategy

    RemanenetSpy
    aa606b6
  2. chore: reorganize repo — move benchmark_adapter to benchmark/adapters, cerberos contract to docs, remove stale error.txt and duplicate precision-mem-bench.md from docs

    RemanenetSpy
    1cca1b2
  3. feat(benchmark): add isolated benchmark suite with harness, adapters, and results

    RemanenetSpy
    38fb868

  1. chore: add smriti benchmark runner and eval service

    RemanenetSpy
    64f8bdd