Commit History

  1. refactor: consolidate pure 32L Dense architecture, purge MoE files, audit hardening

    Verified signature
    3a5d3e5

  1. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    Verified signature
    9d0f9ca

  1. Sync data/scripts/ingest_large_cinema_mythology.py with massive Mythology and Bollywood Cinema corpora expansion

    Verified signature
    c183e8d

  1. feat: wipe-proof HF persistence (full ckpt + live log + newest-2 cleanup)

    Verified signature
    fb5d414
  2. fix: exclude non-5col schema paths from HF stream (wikipedia/grammar/gsm8k CastError on shuffle)

    Verified signature
    ca926e4
  3. fix: probe OOM reserve 4GB + expandable_segments + natural-mode stream shuffle (Hindi-only trap)

    Verified signature
    06e08fa
  4. sync local: enforce_mix sampler, tokenizer v2 (mask fix + 35/30/35), mix optional docs, repair scripts

    Verified signature
    991af26

  1. feat: 100% overnight-resilient batch repair with 999 retries and persistent rate-limit handling

    Verified signature
    338f372
  2. feat: batch upload shards (10 at once) in 1 commit to permanently bypass 128 commits/hr rate limit

    Verified signature
    6085120
  3. fix: add smart 429 rate limit backoff and retry to repair scripts

    Verified signature
    e1c1d38
  4. refactor: clean repo, fix cross-platform paths, update progress

    Verified signature
    6591c1c
  5. docs: add official comprehensive data audit report (145.93GB/105.1B tokens - 100% PASS)

    Verified signature
    51b7902
  6. docs & scripts: add dedicated grammar stream milestone (145.93GB/105.1B tokens)

    Verified signature
    2001a4e
  7. docs & scripts: add full-scale 5-domain milestone (145.93GB/105B tokens)

    Verified signature
    8d7554c

  1. docs & scripts: add dedicated science stream milestone (128.75GB/92B tokens)

    Verified signature
    7ac6a19

  1. docs & scripts: add dictionary, expanded stories, and massive uncensored datasets milestone (117GB/83B tokens)

    Verified signature
    46c2986
  2. feat/docs: sync stories & uncensored scripts & docs for data/scripts/add_stories_and_uncensored.py

    Verified signature
    bfdbc0f
  3. feat/docs: sync open-source toxicity datasets & docs for data/scripts/add_open_source_toxicity.py

    Verified signature
    a69d2b9
  4. feat/docs: sync dedicated toxicity folder script & docs for data/scripts/generate_toxicity_lexicon.py

    Verified signature
    2c8f401
  5. docs/feat: sync translation & hinglish expansion for data/scripts/add_curated_hinglish.py

    Verified signature
    30b674c
  6. docs/feat: sync translation & hinglish expansion for data/scripts/add_samanantar.py

    Verified signature
    bc2430b
  7. docs/feat: sync deep domains, wikipedia ingestion & docs for data/scripts/add_wikipedia.py

    Verified signature
    205a575
  8. docs/feat: sync deep domains, wikipedia ingestion & docs for data/scripts/generate_deep_domains.py

    Verified signature
    2020c90
  9. feat: add Google Colab userdata secrets support to collector

    Verified signature
    2629527
  10. fix: Hinglish column key mapping and streamline stream execution order

    Verified signature
    c8521a2

  1. feat: upgrade collector to 1GB shards with smart auto-resume

    Verified signature
    1072676
  2. Add Single-Cell All-in-One Cloud Ingestion runner to Kaggle runbook

    Verified signature
    4234e9e
  3. Add Pillar 26 Grammar & Language Mechanics (Hindi Vyakaran, English Grammar, Hinglish Syntax)

    Verified signature
    cc255c6
  4. Add 25 Pillars: Stories, Movie Plots, Songs, Mobile Tech, Jokes & Street Roasts

    Verified signature
    7812c7f
  5. Add Pillar 21 Translation & Global Cuisines expansion to data pipeline

    Verified signature
    5c80566
  6. Add Step 4 30B-50B+ Kaggle data collection suite & 20-pillar knowledge matrix

    Verified signature
    25bc372
  7. feat: SDPA FlashAttention, language tags conditioning, probe safety, slim checkpoints, and streaming clean

    Verified signature
    57f2f3e

  1. step4: clean_dedup.py + pull status

    Verified signature
    798598c

  1. init: 28L MoE 241M audit-fixed, smoke pass

    Verified signature
    140f9f6