Hugging Face's logo

shahils
/
GT-MHA

Transformers
Safetensors
bert
attention
gt-mha
masked-language-modeling
Model card Files
xet
 Community
GT-MHA

Commit History

  1. Rename TinyStories-v2 Checkpoints/MQA/checkpoint_step_249800.pt to TinyStories-v2 Checkpoints/MQA/checkpoint_step_250000.pt

    shahils
    Verified signature
    9bb7039

  1. Correct TinyStories residual checkpoint to four-base run at step 250000

    shahils
    Verified signature
    27c0694

  1. Add completed TinyStories QK-identity B4G8H16 seed0 checkpoint

    shahils
    Verified signature
    992b5ba

  2. Add completed TinyStories QK-identity B4G8H16 seed0 checkpoint

    shahils
    Verified signature
    9820fe4

  3. Add completed TinyStories QK-identity B4G8H16 seed0 checkpoint

    shahils
    Verified signature
    f59c1e2

  1. Upload folder using huggingface_hub

    shahils
    Verified signature
    1054dc6

  1. Update README.md⁠

    shahils
    Verified signature
    3fa5fa0

  1. Remove redundant TinyStories checkpoint artifacts

    shahils
    Verified signature
    a3d9cb4

  2. Add LGMA package metadata

    shahils
    Verified signature
    acd8902

  3. Add TinyStories training entrypoint

    shahils
    Verified signature
    8b374a3

  4. Add TinyStories sampling entrypoint

    shahils
    Verified signature
    243ce08

  5. Add TinyStories checkpoint runtime

    shahils
    Verified signature
    a0948b3

  6. Add LGMA source for TinyStories checkpoint loading

    shahils
    Verified signature
    c353cf5

  7. Document safe TinyStories GT-MHA loading

    shahils
    Verified signature
    52aef04

  8. Add TinyStories GT-MHA residual final report

    shahils
    Verified signature
    2bf3368

  9. Add TinyStories GT-MHA residual final checkpoint

    shahils
    Verified signature
    1efe235

  1. Update README.md⁠

    shahils
    Verified signature
    0592f33

  2. Add official GLUE results for GQA and collaborativeMHA⁠

    shahils
    Verified signature
    c160bf6

  1. Add files using upload-large-folder tool

    shahils
    Verified signature
    936b8a7

  2. Add files using upload-large-folder tool

    shahils
    Verified signature
    9c353cc

  3. Add files using upload-large-folder tool

    shahils
    Verified signature
    97dc0bf

  4. Add files using upload-large-folder tool

    shahils
    Verified signature
    48e5f9a

  5. Add files using upload-large-folder tool

    shahils
    Verified signature
    3782828

  1. Report official GLUE test results

    shahils
    Verified signature
    c282628

  1. Remove GLUE validation-loss table

    shahils
    Verified signature
    f3cd930

  2. Omit WNLI from GLUE validation tables

    shahils
    Verified signature
    9007729

  3. Compile five-seed GLUE validation sweeps

    shahils
    Verified signature
    a44ba0e

  1. Update tuned GLUE sweep results

    shahils
    Verified signature
    b8b0141

  1. Update README with tuned five-seed STS-B results

    shahils
    Verified signature
    ae5dfb9

  2. Add STS-B learning-rate selection record

    shahils
    Verified signature
    8cc22c1

  3. Add STS-B learning-rate selection table

    shahils
    Verified signature
    39bef00

  4. Add strict custom-attention checkpoint loader

    shahils
    Verified signature
    ca4b625

  5. Add selected STS-B Collaborative MHA checkpoint

    shahils
    Verified signature
    25c006e

  6. Add selected STS-B GT-MHA residual checkpoint

    shahils
    Verified signature
    5a5736c

  7. Add selected STS-B GQA checkpoint

    shahils
    Verified signature
    31fea48

  8. Add selected STS-B MHA checkpoint

    shahils
    Verified signature
    fafcb90

  9. Report complete three-seed GLUE suite

    shahils
    Verified signature
    a8ea404

  10. Add four-task GLUE macro average

    shahils
    Verified signature
    b7f00d3

  11. Report best validation within four epochs

    shahils
    Verified signature
    4d35674

  12. Restore three-seed mean and std GLUE results

    shahils
    Verified signature
    36a1082

  13. Report best GLUE seed for every attention method

    shahils
    Verified signature
    3c317f2

  14. Report best GT-MHA residual GLUE seeds

    shahils
    Verified signature
    31bd8dc

  15. Report GT-MHA residual three-seed GLUE results

    shahils
    Verified signature
    2a25441

  16. Report GLUE scores on 100-point scale

    shahils
    Verified signature
    ef1574f

  17. Report Collaborative MHA three-seed GLUE results

    shahils
    Verified signature
    8af5c46

  1. Report GQA three-seed GLUE results

    shahils
    Verified signature
    adb0ec3

  2. Report MHA three-seed GLUE mean and std

    shahils
    Verified signature
    4af9eb5

  3. Add corrected Collaborative MHA and GT-MHA residual GLUE results

    shahils
    Verified signature
    32042df

  4. Add corrected Collaborative MHA cola final checkpoint

    shahils
    Verified signature
    715750a

  5. Add corrected Collaborative MHA sst2 final checkpoint

    shahils
    Verified signature
    dab022b