Feature Extraction
Transformers
TensorBoard
Safetensors
English
captionbert_structured_rotary
sentence-similarity
consensus-distillation
rotary-embeddings
custom_code
Instructions to use AbstractPhil/captionbert-8192-rotary with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AbstractPhil/captionbert-8192-rotary with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="AbstractPhil/captionbert-8192-rotary", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("AbstractPhil/captionbert-8192-rotary", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Preserve complete isolated phase-control console
Browse files
campaigns/structured-20261106/operations/phase-control-A/20261106T201516Z/console.log
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
GPU memory ceiling 92.00 GB total; allocator allowance 90.61 GB
|
| 2 |
+
Frozen development: KALLP-seed1, 4,096 rows; FP32, no optimizer
|
| 3 |
+
|
| 4 |
+
Loading weights: 0%| | 0/158 [00:00<?, ?it/s]
|
| 5 |
+
Loading weights: 100%|██████████| 158/158 [00:00<00:00, 16480.98it/s]
|
| 6 |
+
[transformers] `use_return_dict` is deprecated! Use `return_dict` instead!
|
| 7 |
+
Lookup: 400/4,096 development captions
|
| 8 |
+
Lookup: 800/4,096 development captions
|
| 9 |
+
Lookup: 1,200/4,096 development captions
|
| 10 |
+
Lookup: 1,600/4,096 development captions
|
| 11 |
+
Lookup: 2,000/4,096 development captions
|
| 12 |
+
Lookup: 2,400/4,096 development captions
|
| 13 |
+
Lookup: 2,800/4,096 development captions
|
| 14 |
+
Lookup: 3,200/4,096 development captions
|
| 15 |
+
Lookup: 3,600/4,096 development captions
|
| 16 |
+
Lookup: 4,000/4,096 development captions
|
| 17 |
+
Validation heldout: 5,000 rows, batch 64
|
| 18 |
+
Validation heldout: 1,600/5,000 rows
|
| 19 |
+
Validation heldout: 3,200/5,000 rows
|
| 20 |
+
Validation heldout: 4,800/5,000 rows
|
| 21 |
+
Validation heldout: 5,000/5,000 rows
|
| 22 |
+
Validation heldout: computing cosine, effective rank and self-cosine...
|
| 23 |
+
Validation heldout: n=5,000 cosine=0.76642 erank=79.79517641354452 self_cos=0.004734748013822296
|
| 24 |
+
Validation >128 tokens: 5,000 rows, batch 64
|
| 25 |
+
Validation >128 tokens: 1,600/5,000 rows
|
| 26 |
+
Validation >128 tokens: 3,200/5,000 rows
|
| 27 |
+
Validation >128 tokens: 4,800/5,000 rows
|
| 28 |
+
Validation >128 tokens: 5,000/5,000 rows
|
| 29 |
+
Validation >128 tokens: computing cosine, effective rank and self-cosine...
|
| 30 |
+
Validation >128 tokens: n=5,000 cosine=0.70585 erank=60.8426064026275 self_cos=0.014027573271179952
|
| 31 |
+
Validation complete in 2.5s
|
| 32 |
+
Lookup: 400/4,096 development captions
|
| 33 |
+
Lookup: 800/4,096 development captions
|
| 34 |
+
Lookup: 1,200/4,096 development captions
|
| 35 |
+
Lookup: 1,600/4,096 development captions
|
| 36 |
+
Lookup: 2,000/4,096 development captions
|
| 37 |
+
Lookup: 2,400/4,096 development captions
|
| 38 |
+
Lookup: 2,800/4,096 development captions
|
| 39 |
+
Lookup: 3,200/4,096 development captions
|
| 40 |
+
Lookup: 3,600/4,096 development captions
|
| 41 |
+
Lookup: 4,000/4,096 development captions
|
| 42 |
+
Validation heldout: 5,000 rows, batch 64
|
| 43 |
+
Validation heldout: 1,600/5,000 rows
|
| 44 |
+
Validation heldout: 3,200/5,000 rows
|
| 45 |
+
Validation heldout: 4,800/5,000 rows
|
| 46 |
+
Validation heldout: 5,000/5,000 rows
|
| 47 |
+
Validation heldout: computing cosine, effective rank and self-cosine...
|
| 48 |
+
Validation heldout: n=5,000 cosine=0.76558 erank=80.02660330633002 self_cos=0.003528809231293087
|
| 49 |
+
Validation >128 tokens: 5,000 rows, batch 64
|
| 50 |
+
Validation >128 tokens: 1,600/5,000 rows
|
| 51 |
+
Validation >128 tokens: 3,200/5,000 rows
|
| 52 |
+
Validation >128 tokens: 4,800/5,000 rows
|
| 53 |
+
Validation >128 tokens: 5,000/5,000 rows
|
| 54 |
+
Validation >128 tokens: computing cosine, effective rank and self-cosine...
|
| 55 |
+
Validation >128 tokens: n=5,000 cosine=0.70424 erank=61.08356599143945 self_cos=0.012483398097361887
|
| 56 |
+
Validation complete in 2.4s
|
| 57 |
+
{"arm": "KALLP", "seed": 1, "measurements": {"intact": {"all": {"queries": 10146, "captions": 1935, "record_mass": 0.9995101094245911, "caption_mean_record_mass": 0.9996374544863245, "median_record_mass": 0.9998422861099243, "exact_record_top1": 1.0, "wrong_adjacent_mass": 2.5728611490194453e-06, "uniform_mass": 0.013713696040213108}, "ending": {"queries": 9483, "captions": 1935, "record_mass": 0.9995501041412354, "caption_mean_record_mass": 0.9996627638814369, "median_record_mass": 0.9998492002487183, "exact_record_top1": 1.0, "wrong_adjacent_mass": 2.58109707829135e-06, "uniform_mass": 0.013700349256396294}, "non_ending": {"queries": 663, "captions": 299, "record_mass": 0.9989388585090637, "caption_mean_record_mass": 0.9989243283319633, "median_record_mass": 0.9996836185455322, "exact_record_top1": 1.0, "wrong_adjacent_mass": 2.4550568014092278e-06, "uniform_mass": 0.013904616236686707}, "continuation": {"queries": 663, "captions": 299, "record_mass": 0.9993511438369751, "caption_mean_record_mass": 0.9993502210613876, "median_record_mass": 0.9997895359992981, "exact_record_top1": 1.0, "wrong_adjacent_mass": 1.0532260148465866e-06, "uniform_mass": 0.013904616236686707}, "above128": {"queries": 3198, "captions": 252, "record_mass": 0.9993064999580383, "caption_mean_record_mass": 0.99934764040841, "median_record_mass": 0.9997236728668213, "exact_record_top1": 1.0, "wrong_adjacent_mass": 2.088266455757548e-06, "uniform_mass": 0.006118615623563528}}, "phase_constant": {"all": {"queries": 10146, "captions": 1935, "record_mass": 0.11369077116250992, "caption_mean_record_mass": 0.16619080153897944, "median_record_mass": 0.06701581180095673, "exact_record_top1": 0.33343189434259807, "wrong_adjacent_mass": 0.013728410005569458, "uniform_mass": 0.013713696040213108}, "ending": {"queries": 9483, "captions": 1935, "record_mass": 0.11622979491949081, "caption_mean_record_mass": 0.1676733015916125, "median_record_mass": 0.06804138422012329, "exact_record_top1": 0.34282400084361486, "wrong_adjacent_mass": 0.013602565973997116, "uniform_mass": 0.013700349256396294}, "non_ending": {"queries": 663, "captions": 299, "record_mass": 0.07737474143505096, "caption_mean_record_mass": 0.07663633533516731, "median_record_mass": 0.0503498911857605, "exact_record_top1": 0.19909502262443438, "wrong_adjacent_mass": 0.01552837248891592, "uniform_mass": 0.013904616236686707}, "continuation": {"queries": 663, "captions": 299, "record_mass": 0.0965983048081398, "caption_mean_record_mass": 0.0990181110860051, "median_record_mass": 0.0706278383731842, "exact_record_top1": 0.28054298642533937, "wrong_adjacent_mass": 0.015272222459316254, "uniform_mass": 0.013904616236686707}, "above128": {"queries": 3198, "captions": 252, "record_mass": 0.053152699023485184, "caption_mean_record_mass": 0.0541802396266056, "median_record_mass": 0.03980260342359543, "exact_record_top1": 0.2607879924953096, "wrong_adjacent_mass": 0.005689086392521858, "uniform_mass": 0.006118615623563528}}}}
|