Publish MIC usage, slim fixture and reviewed Transformers expectations
Browse files- .gitattributes +1 -0
- .mic-repo.json +7 -0
- README.md +37 -0
- build.json +155 -0
- config.json +48 -0
- generation_config.json +8 -0
- integration.json +80 -0
- integrations/transformers/expectations/cpu-x86_64-bf16/tiny-random-aya-base-mic.bfloat16.json +73 -0
- integrations/transformers/expectations/cpu-x86_64/tiny-random-aya-base-mic.eager.json +73 -0
- integrations/transformers/expectations/cpu-x86_64/tiny-random-aya-base-mic.json +73 -0
- integrations/transformers/expectations/cuda/tiny-random-aya-base-mic.bfloat16.json +74 -0
- integrations/transformers/expectations/cuda/tiny-random-aya-base-mic.eager.json +74 -0
- integrations/transformers/expectations/cuda/tiny-random-aya-base-mic.json +74 -0
- integrations/transformers/usage.md +77 -0
- model.safetensors +3 -0
- special_tokens_map.json +30 -0
- tokenizer.json +3 -0
- tokenizer_config.json +141 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
.mic-repo.json
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"format": "mic-umbrella-v1",
|
| 3 |
+
"contract_files": [
|
| 4 |
+
"integration.json",
|
| 5 |
+
"integrations/transformers/usage.md"
|
| 6 |
+
]
|
| 7 |
+
}
|
README.md
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- mic
|
| 4 |
+
- test-artifact
|
| 5 |
+
base_model: optimum-intel-internal-testing/tiny-random-aya-base
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# tiny-random-aya-base-mic
|
| 9 |
+
|
| 10 |
+
Slim model weights, executable usage and framework expectations in one repository.
|
| 11 |
+
|
| 12 |
+
- [Contract manifest](integration.json)
|
| 13 |
+
- [Transformers usage](integrations/transformers/usage.md)
|
| 14 |
+
- [Transformers expectations](integrations/transformers/expectations)
|
| 15 |
+
- [Fixture provenance](build.json)
|
| 16 |
+
|
| 17 |
+
## Contract
|
| 18 |
+
|
| 19 |
+
### Cohere2 causal LM
|
| 20 |
+
|
| 21 |
+
Runnable usage and the framework expectations directory are linked above.
|
| 22 |
+
|
| 23 |
+
## Slim model
|
| 24 |
+
|
| 25 |
+
### Slim fixture: tiny-random-aya-base-mic
|
| 26 |
+
|
| 27 |
+
A 16.9M-parameter fixture for the [Cohere2 MIC](https://huggingface.co/huggingface-mic/tiny-random-aya-base-mic), covering [`optimum-intel-internal-testing/tiny-random-aya-base`](https://huggingface.co/optimum-intel-internal-testing/tiny-random-aya-base). It is a test artifact, not a usable model, tagged `test-artifact` to keep it out of candidate-model search.
|
| 28 |
+
|
| 29 |
+
- **Fixture key `942d7e0ff561`.** It covers every checkpoint whose processor files are byte-identical and whose config matches after shrinking.
|
| 30 |
+
- **Same architecture, smaller config:** `Cohere2ForCausalLM`, shrunk by the generic rule in [`sweep.py`](https://github.com/huggingface/transformers-mic/blob/main/experiments/sweep.py) (num_hidden_layers 2).
|
| 31 |
+
- **Processor files copied byte for byte** from `optimum-intel-internal-testing/tiny-random-aya-base`: `generation_config.json`, `special_tokens_map.json`, `tokenizer.json`, `tokenizer_config.json`.
|
| 32 |
+
- **Random weights (seed 17).** They passed the input-sensitivity gate: every perturbation of the contract's inputs moved the logits by at least 0.001 (weakest: `attention_mask:all_ones`, 0.477), so no training was needed.
|
| 33 |
+
- **Same tensor names as the real checkpoint.** The real checkpoint has every parameter, and so does the fixture.
|
| 34 |
+
- **Saved in float32**, the dtype `optimum-intel-internal-testing/tiny-random-aya-base` loads in by default.
|
| 35 |
+
- **Finite in float32, bfloat16, and float16** (forward, loss, gradients).
|
| 36 |
+
|
| 37 |
+
Built by [`build_fixtures.py`](https://github.com/huggingface/transformers-mic/blob/main/example/build_fixtures.py); [`build.json`](build.json) records the pinned sources, gate results, and file hashes. This repository includes the slim weights, usage and framework expectations; rebuild the other files with `python example/build_fixtures.py --fixture tiny-random-aya-base-mic`.
|
build.json
ADDED
|
@@ -0,0 +1,155 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"fixture": "huggingface-mic/tiny-random-aya-base-mic",
|
| 3 |
+
"key": "942d7e0ff56152ffba0f6968c631c68ce6925551b97069225beed6880b8f4b5d",
|
| 4 |
+
"key_fields": {
|
| 5 |
+
"architectures": [
|
| 6 |
+
"Cohere2ForCausalLM"
|
| 7 |
+
],
|
| 8 |
+
"output_hidden_states": false,
|
| 9 |
+
"return_dict": true,
|
| 10 |
+
"chunk_size_feed_forward": 0,
|
| 11 |
+
"is_encoder_decoder": false,
|
| 12 |
+
"id2label": {
|
| 13 |
+
"0": "LABEL_0",
|
| 14 |
+
"1": "LABEL_1"
|
| 15 |
+
},
|
| 16 |
+
"label2id": {
|
| 17 |
+
"LABEL_0": 0,
|
| 18 |
+
"LABEL_1": 1
|
| 19 |
+
},
|
| 20 |
+
"problem_type": null,
|
| 21 |
+
"vocab_size": 262144,
|
| 22 |
+
"hidden_size": 64,
|
| 23 |
+
"intermediate_size": 128,
|
| 24 |
+
"logit_scale": 1.0,
|
| 25 |
+
"num_hidden_layers": 2,
|
| 26 |
+
"num_attention_heads": 4,
|
| 27 |
+
"num_key_value_heads": 4,
|
| 28 |
+
"hidden_act": "silu",
|
| 29 |
+
"max_position_embeddings": 500000,
|
| 30 |
+
"initializer_range": 0.02,
|
| 31 |
+
"layer_norm_eps": 1e-05,
|
| 32 |
+
"use_cache": true,
|
| 33 |
+
"pad_token_id": 0,
|
| 34 |
+
"bos_token_id": 2,
|
| 35 |
+
"eos_token_id": 3,
|
| 36 |
+
"tie_word_embeddings": true,
|
| 37 |
+
"rope_parameters": {
|
| 38 |
+
"rope_theta": 50000,
|
| 39 |
+
"rope_type": "default"
|
| 40 |
+
},
|
| 41 |
+
"attention_bias": false,
|
| 42 |
+
"attention_dropout": 0.0,
|
| 43 |
+
"sliding_window": 4096,
|
| 44 |
+
"layer_types": [
|
| 45 |
+
"sliding_attention",
|
| 46 |
+
"full_attention"
|
| 47 |
+
],
|
| 48 |
+
"head_dim": 16,
|
| 49 |
+
"_sliding_window_pattern": 4,
|
| 50 |
+
"cache_implementation": "hybrid",
|
| 51 |
+
"layer_switch": 4,
|
| 52 |
+
"model_type": "cohere2",
|
| 53 |
+
"order_of_interleaved_layers": "local_attn_first",
|
| 54 |
+
"position_embedding_type": "rope_gptj",
|
| 55 |
+
"rotary_pct": 1.0,
|
| 56 |
+
"sliding_window_pattern": 4,
|
| 57 |
+
"use_embedding_sharing": true,
|
| 58 |
+
"use_gated_activation": true,
|
| 59 |
+
"use_parallel_block": true,
|
| 60 |
+
"use_parallel_embedding": false,
|
| 61 |
+
"use_qk_norm": false,
|
| 62 |
+
"output_attentions": false
|
| 63 |
+
},
|
| 64 |
+
"covers": {
|
| 65 |
+
"optimum-intel-internal-testing/tiny-random-aya-base": "e85478d37f912454b23ff2a9c7af16a960837644"
|
| 66 |
+
},
|
| 67 |
+
"built_from": {
|
| 68 |
+
"repo": "optimum-intel-internal-testing/tiny-random-aya-base",
|
| 69 |
+
"revision": "e85478d37f912454b23ff2a9c7af16a960837644"
|
| 70 |
+
},
|
| 71 |
+
"seed": 17,
|
| 72 |
+
"dtype": "float32",
|
| 73 |
+
"builder": "example/build_fixtures.py",
|
| 74 |
+
"builder_revision": "d1b2e0637b293ce867d0364161d6d69777d0941b",
|
| 75 |
+
"recipe": {
|
| 76 |
+
"seed": 17,
|
| 77 |
+
"model_class": "Cohere2ForCausalLM"
|
| 78 |
+
},
|
| 79 |
+
"shrink": "rule",
|
| 80 |
+
"model_class": "Cohere2ForCausalLM",
|
| 81 |
+
"parameters": 16859328,
|
| 82 |
+
"missing_parameters": [],
|
| 83 |
+
"serialization": {
|
| 84 |
+
"file": "model.safetensors",
|
| 85 |
+
"reverted_conversions": []
|
| 86 |
+
},
|
| 87 |
+
"checkpoint_inspection": "parameter headers/indexes; bounded binary fallback",
|
| 88 |
+
"numeric_health_scope": "CPU float32/bfloat16/float16; autocast for low-precision losses",
|
| 89 |
+
"captured": {
|
| 90 |
+
"call": "training step",
|
| 91 |
+
"inputs": [
|
| 92 |
+
"attention_mask",
|
| 93 |
+
"input_ids",
|
| 94 |
+
"labels"
|
| 95 |
+
],
|
| 96 |
+
"class": "Cohere2ForCausalLM"
|
| 97 |
+
},
|
| 98 |
+
"weights": "random",
|
| 99 |
+
"gate": {
|
| 100 |
+
"random": {
|
| 101 |
+
"pass": true,
|
| 102 |
+
"accepted_insensitive": {},
|
| 103 |
+
"threshold": 0.001,
|
| 104 |
+
"deltas": {
|
| 105 |
+
"input_ids:roll_batch": 1.3876315355300903,
|
| 106 |
+
"input_ids:reverse": 1.5412429571151733,
|
| 107 |
+
"attention_mask:all_ones": 0.4773968458175659
|
| 108 |
+
},
|
| 109 |
+
"weakest": "attention_mask:all_ones",
|
| 110 |
+
"no_perturbation_defined": [],
|
| 111 |
+
"not_a_forward_parameter": []
|
| 112 |
+
}
|
| 113 |
+
},
|
| 114 |
+
"numeric_health": {
|
| 115 |
+
"float32": {
|
| 116 |
+
"outputs_finite": true,
|
| 117 |
+
"loss_finite": true,
|
| 118 |
+
"gradients_finite": true,
|
| 119 |
+
"finite": true
|
| 120 |
+
},
|
| 121 |
+
"bfloat16": {
|
| 122 |
+
"outputs_finite": true,
|
| 123 |
+
"loss_finite": true,
|
| 124 |
+
"gradients_finite": true,
|
| 125 |
+
"finite": true
|
| 126 |
+
},
|
| 127 |
+
"float16": {
|
| 128 |
+
"outputs_finite": true,
|
| 129 |
+
"loss_finite": true,
|
| 130 |
+
"unscaled_gradients_finite": true,
|
| 131 |
+
"finite": true
|
| 132 |
+
}
|
| 133 |
+
},
|
| 134 |
+
"copied_unchanged": [
|
| 135 |
+
"generation_config.json",
|
| 136 |
+
"special_tokens_map.json",
|
| 137 |
+
"tokenizer.json",
|
| 138 |
+
"tokenizer_config.json"
|
| 139 |
+
],
|
| 140 |
+
"environment": {
|
| 141 |
+
"python": "3.12.12",
|
| 142 |
+
"machine": "arm64",
|
| 143 |
+
"torch": "2.14.1",
|
| 144 |
+
"transformers": "5.22.0.dev0",
|
| 145 |
+
"tokenizers": "0.23.3"
|
| 146 |
+
},
|
| 147 |
+
"sha256": {
|
| 148 |
+
"config.json": "c138ff718afd9e1a5a19804e88225da79e5397bdf0fa5bef136207f189105349",
|
| 149 |
+
"generation_config.json": "fa1f6af5005a20eafe7ce1a4365134a2e7a791d739e501e981946bd71f417d64",
|
| 150 |
+
"model.safetensors": "c4cf5b8ea901291cfeda11936929fa25869fc600d2529620e9307241c6cc6a06",
|
| 151 |
+
"special_tokens_map.json": "2b7ce39630f8a18705331a9b32c3b2236e96e7410791f98a9f8b6494e3ab2ea6",
|
| 152 |
+
"tokenizer.json": "8f21f6c4f761c192f486ea2c5b06b62b3ef30819b33dc105bdf8b26c8e7974f6",
|
| 153 |
+
"tokenizer_config.json": "a6f3636edc7f2b5db4fa243a9b1d540ff6fc651f6da4fc6afed6fc2332451bd7"
|
| 154 |
+
}
|
| 155 |
+
}
|
config.json
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_sliding_window_pattern": 4,
|
| 3 |
+
"architectures": [
|
| 4 |
+
"Cohere2ForCausalLM"
|
| 5 |
+
],
|
| 6 |
+
"attention_bias": false,
|
| 7 |
+
"attention_dropout": 0.0,
|
| 8 |
+
"bos_token_id": 2,
|
| 9 |
+
"cache_implementation": "hybrid",
|
| 10 |
+
"dtype": "float32",
|
| 11 |
+
"eos_token_id": 3,
|
| 12 |
+
"head_dim": 16,
|
| 13 |
+
"hidden_act": "silu",
|
| 14 |
+
"hidden_size": 64,
|
| 15 |
+
"initializer_range": 0.02,
|
| 16 |
+
"intermediate_size": 128,
|
| 17 |
+
"layer_norm_eps": 1e-05,
|
| 18 |
+
"layer_switch": 4,
|
| 19 |
+
"layer_types": [
|
| 20 |
+
"sliding_attention",
|
| 21 |
+
"full_attention"
|
| 22 |
+
],
|
| 23 |
+
"logit_scale": 1.0,
|
| 24 |
+
"max_position_embeddings": 500000,
|
| 25 |
+
"model_type": "cohere2",
|
| 26 |
+
"num_attention_heads": 4,
|
| 27 |
+
"num_hidden_layers": 2,
|
| 28 |
+
"num_key_value_heads": 4,
|
| 29 |
+
"order_of_interleaved_layers": "local_attn_first",
|
| 30 |
+
"pad_token_id": 0,
|
| 31 |
+
"position_embedding_type": "rope_gptj",
|
| 32 |
+
"rope_parameters": {
|
| 33 |
+
"rope_theta": 50000,
|
| 34 |
+
"rope_type": "default"
|
| 35 |
+
},
|
| 36 |
+
"rotary_pct": 1.0,
|
| 37 |
+
"sliding_window": 4096,
|
| 38 |
+
"sliding_window_pattern": 4,
|
| 39 |
+
"tie_word_embeddings": true,
|
| 40 |
+
"transformers_version": "5.22.0.dev0",
|
| 41 |
+
"use_cache": true,
|
| 42 |
+
"use_embedding_sharing": true,
|
| 43 |
+
"use_gated_activation": true,
|
| 44 |
+
"use_parallel_block": true,
|
| 45 |
+
"use_parallel_embedding": false,
|
| 46 |
+
"use_qk_norm": false,
|
| 47 |
+
"vocab_size": 262144
|
| 48 |
+
}
|
generation_config.json
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_from_model_config": true,
|
| 3 |
+
"bos_token_id": 2,
|
| 4 |
+
"cache_implementation": "hybrid",
|
| 5 |
+
"eos_token_id": 3,
|
| 6 |
+
"pad_token_id": 0,
|
| 7 |
+
"transformers_version": "5.0.0"
|
| 8 |
+
}
|
integration.json
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"mic": "0.3",
|
| 3 |
+
"name": "Cohere2 causal LM",
|
| 4 |
+
"usage": "AutoTokenizer + AutoModelForCausalLM, batched, left padding, next-token loss",
|
| 5 |
+
"architecture": "Cohere2ForCausalLM",
|
| 6 |
+
"checkpoints": {
|
| 7 |
+
"optimum-intel-internal-testing/tiny-random-aya-base": {
|
| 8 |
+
"fixture": "huggingface-mic/tiny-random-aya-base-mic"
|
| 9 |
+
}
|
| 10 |
+
},
|
| 11 |
+
"fixtures": {
|
| 12 |
+
"huggingface-mic/tiny-random-aya-base-mic": {
|
| 13 |
+
"built_from": "optimum-intel-internal-testing/tiny-random-aya-base",
|
| 14 |
+
"covers": [
|
| 15 |
+
"optimum-intel-internal-testing/tiny-random-aya-base"
|
| 16 |
+
],
|
| 17 |
+
"build": {
|
| 18 |
+
"seed": 17,
|
| 19 |
+
"model_class": "Cohere2ForCausalLM"
|
| 20 |
+
}
|
| 21 |
+
}
|
| 22 |
+
},
|
| 23 |
+
"integrations": {
|
| 24 |
+
"transformers": {
|
| 25 |
+
"entry_point": "integrations/transformers/usage.md",
|
| 26 |
+
"format": "runnable-markdown",
|
| 27 |
+
"requires": [
|
| 28 |
+
"transformers>=5",
|
| 29 |
+
"torch"
|
| 30 |
+
],
|
| 31 |
+
"targets": [
|
| 32 |
+
"fixtures"
|
| 33 |
+
],
|
| 34 |
+
"variants": {
|
| 35 |
+
"default": {},
|
| 36 |
+
"eager": {
|
| 37 |
+
"attn_implementation": "eager"
|
| 38 |
+
},
|
| 39 |
+
"bfloat16": {
|
| 40 |
+
"dtype": "bfloat16"
|
| 41 |
+
}
|
| 42 |
+
},
|
| 43 |
+
"training": true,
|
| 44 |
+
"resources": {
|
| 45 |
+
"fixture": {
|
| 46 |
+
"memory_gb": 2,
|
| 47 |
+
"download_mb": 20
|
| 48 |
+
},
|
| 49 |
+
"optimum-intel-internal-testing/tiny-random-aya-base": {
|
| 50 |
+
"memory_gb": 2,
|
| 51 |
+
"download_mb": 20
|
| 52 |
+
}
|
| 53 |
+
}
|
| 54 |
+
}
|
| 55 |
+
},
|
| 56 |
+
"outputs": {
|
| 57 |
+
"runtime": {
|
| 58 |
+
"compare": "exact"
|
| 59 |
+
},
|
| 60 |
+
"next_token_logits": {
|
| 61 |
+
"compare": "allclose",
|
| 62 |
+
"atol": 0.0001,
|
| 63 |
+
"rtol": 0.0001
|
| 64 |
+
},
|
| 65 |
+
"predicted_tokens": {
|
| 66 |
+
"compare": "exact"
|
| 67 |
+
},
|
| 68 |
+
"training.loss": {
|
| 69 |
+
"compare": "allclose",
|
| 70 |
+
"atol": 0.0001,
|
| 71 |
+
"rtol": 0.0001
|
| 72 |
+
},
|
| 73 |
+
"training.gradients": {
|
| 74 |
+
"compare": "all_true"
|
| 75 |
+
},
|
| 76 |
+
"training.updated": {
|
| 77 |
+
"compare": "true"
|
| 78 |
+
}
|
| 79 |
+
}
|
| 80 |
+
}
|
integrations/transformers/expectations/cpu-x86_64-bf16/tiny-random-aya-base-mic.bfloat16.json
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/tiny-random-aya-base-mic",
|
| 4 |
+
"contract_name": "Cohere2",
|
| 5 |
+
"contract_revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "bfloat16",
|
| 8 |
+
"target": "tiny-random-aya-base-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.23.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cpu",
|
| 15 |
+
"torchvision": "0.29.2+cpu",
|
| 16 |
+
"torchaudio": "2.11.1+cpu",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.3.5",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.15.0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.4.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 8.23,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torchvision": "0.29.2+cpu (pinned 0.29.1)",
|
| 32 |
+
"torchaudio": "2.11.1+cpu (pinned 2.11.0)",
|
| 33 |
+
"pillow": "13.0.0 (pinned 12.3.0)",
|
| 34 |
+
"huggingface-hub": "2.4.0 (pinned 2.3.0)"
|
| 35 |
+
},
|
| 36 |
+
"platform": "cpu-x86_64-bf16"
|
| 37 |
+
},
|
| 38 |
+
"outputs": {
|
| 39 |
+
"runtime": {
|
| 40 |
+
"device": "cpu",
|
| 41 |
+
"dtype": "bfloat16",
|
| 42 |
+
"attn_implementation": "sdpa"
|
| 43 |
+
},
|
| 44 |
+
"next_token_logits": [
|
| 45 |
+
[
|
| 46 |
+
0.0,
|
| 47 |
+
-0.3203125,
|
| 48 |
+
-0.03662109375,
|
| 49 |
+
-0.038818359375,
|
| 50 |
+
0.04443359375
|
| 51 |
+
],
|
| 52 |
+
[
|
| 53 |
+
0.0,
|
| 54 |
+
-0.2197265625,
|
| 55 |
+
-0.001983642578125,
|
| 56 |
+
-0.08349609375,
|
| 57 |
+
0.15625
|
| 58 |
+
]
|
| 59 |
+
],
|
| 60 |
+
"predicted_tokens": [
|
| 61 |
+
386,
|
| 62 |
+
386
|
| 63 |
+
],
|
| 64 |
+
"training": {
|
| 65 |
+
"loss": 12.488283157348633,
|
| 66 |
+
"gradients": {
|
| 67 |
+
"model.layers.0.self_attn.q_proj.weight": true,
|
| 68 |
+
"model.layers.0.mlp.down_proj.weight": true
|
| 69 |
+
},
|
| 70 |
+
"updated": true
|
| 71 |
+
}
|
| 72 |
+
}
|
| 73 |
+
}
|
integrations/transformers/expectations/cpu-x86_64/tiny-random-aya-base-mic.eager.json
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/tiny-random-aya-base-mic",
|
| 4 |
+
"contract_name": "Cohere2",
|
| 5 |
+
"contract_revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "eager",
|
| 8 |
+
"target": "tiny-random-aya-base-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.23.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cpu",
|
| 15 |
+
"torchvision": "0.29.2+cpu",
|
| 16 |
+
"torchaudio": "2.11.1+cpu",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.3.5",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.15.0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.4.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 8.19,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torchvision": "0.29.2+cpu (pinned 0.29.1)",
|
| 32 |
+
"torchaudio": "2.11.1+cpu (pinned 2.11.0)",
|
| 33 |
+
"pillow": "13.0.0 (pinned 12.3.0)",
|
| 34 |
+
"huggingface-hub": "2.4.0 (pinned 2.3.0)"
|
| 35 |
+
},
|
| 36 |
+
"platform": "cpu-x86_64"
|
| 37 |
+
},
|
| 38 |
+
"outputs": {
|
| 39 |
+
"runtime": {
|
| 40 |
+
"device": "cpu",
|
| 41 |
+
"dtype": "float32",
|
| 42 |
+
"attn_implementation": "eager"
|
| 43 |
+
},
|
| 44 |
+
"next_token_logits": [
|
| 45 |
+
[
|
| 46 |
+
0.0,
|
| 47 |
+
-0.3193780779838562,
|
| 48 |
+
-0.03577787056565285,
|
| 49 |
+
-0.0392596609890461,
|
| 50 |
+
0.04475036635994911
|
| 51 |
+
],
|
| 52 |
+
[
|
| 53 |
+
0.0,
|
| 54 |
+
-0.21895942091941833,
|
| 55 |
+
-0.002652502153068781,
|
| 56 |
+
-0.08413228392601013,
|
| 57 |
+
0.1572474092245102
|
| 58 |
+
]
|
| 59 |
+
],
|
| 60 |
+
"predicted_tokens": [
|
| 61 |
+
386,
|
| 62 |
+
386
|
| 63 |
+
],
|
| 64 |
+
"training": {
|
| 65 |
+
"loss": 12.494779586791992,
|
| 66 |
+
"gradients": {
|
| 67 |
+
"model.layers.0.self_attn.q_proj.weight": true,
|
| 68 |
+
"model.layers.0.mlp.down_proj.weight": true
|
| 69 |
+
},
|
| 70 |
+
"updated": true
|
| 71 |
+
}
|
| 72 |
+
}
|
| 73 |
+
}
|
integrations/transformers/expectations/cpu-x86_64/tiny-random-aya-base-mic.json
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/tiny-random-aya-base-mic",
|
| 4 |
+
"contract_name": "Cohere2",
|
| 5 |
+
"contract_revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "default",
|
| 8 |
+
"target": "tiny-random-aya-base-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.23.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cpu",
|
| 15 |
+
"torchvision": "0.29.2+cpu",
|
| 16 |
+
"torchaudio": "2.11.1+cpu",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.3.5",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.15.0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.4.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 8.5,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torchvision": "0.29.2+cpu (pinned 0.29.1)",
|
| 32 |
+
"torchaudio": "2.11.1+cpu (pinned 2.11.0)",
|
| 33 |
+
"pillow": "13.0.0 (pinned 12.3.0)",
|
| 34 |
+
"huggingface-hub": "2.4.0 (pinned 2.3.0)"
|
| 35 |
+
},
|
| 36 |
+
"platform": "cpu-x86_64"
|
| 37 |
+
},
|
| 38 |
+
"outputs": {
|
| 39 |
+
"runtime": {
|
| 40 |
+
"device": "cpu",
|
| 41 |
+
"dtype": "float32",
|
| 42 |
+
"attn_implementation": "sdpa"
|
| 43 |
+
},
|
| 44 |
+
"next_token_logits": [
|
| 45 |
+
[
|
| 46 |
+
0.0,
|
| 47 |
+
-0.3193780779838562,
|
| 48 |
+
-0.03577786311507225,
|
| 49 |
+
-0.039259668439626694,
|
| 50 |
+
0.0447504036128521
|
| 51 |
+
],
|
| 52 |
+
[
|
| 53 |
+
0.0,
|
| 54 |
+
-0.2189594954252243,
|
| 55 |
+
-0.0026525245048105717,
|
| 56 |
+
-0.08413231372833252,
|
| 57 |
+
0.1572474092245102
|
| 58 |
+
]
|
| 59 |
+
],
|
| 60 |
+
"predicted_tokens": [
|
| 61 |
+
386,
|
| 62 |
+
386
|
| 63 |
+
],
|
| 64 |
+
"training": {
|
| 65 |
+
"loss": 12.48836612701416,
|
| 66 |
+
"gradients": {
|
| 67 |
+
"model.layers.0.self_attn.q_proj.weight": true,
|
| 68 |
+
"model.layers.0.mlp.down_proj.weight": true
|
| 69 |
+
},
|
| 70 |
+
"updated": true
|
| 71 |
+
}
|
| 72 |
+
}
|
| 73 |
+
}
|
integrations/transformers/expectations/cuda/tiny-random-aya-base-mic.bfloat16.json
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/tiny-random-aya-base-mic",
|
| 4 |
+
"contract_name": "Cohere2",
|
| 5 |
+
"contract_revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "bfloat16",
|
| 8 |
+
"target": "tiny-random-aya-base-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.23.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cu130",
|
| 15 |
+
"torchvision": "0.29.2",
|
| 16 |
+
"torchaudio": "2.11.0+cu130",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.4.6",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.16.0.dev0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.4.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 7.67,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torchvision": "0.29.2 (pinned 0.29.1)",
|
| 32 |
+
"numpy": "2.4.6 (pinned 2.3.5)",
|
| 33 |
+
"pillow": "13.0.0 (pinned 12.3.0)",
|
| 34 |
+
"accelerate": "1.16.0.dev0 (pinned 1.15.0)",
|
| 35 |
+
"huggingface-hub": "2.4.0 (pinned 2.3.0)"
|
| 36 |
+
},
|
| 37 |
+
"platform": "cuda"
|
| 38 |
+
},
|
| 39 |
+
"outputs": {
|
| 40 |
+
"runtime": {
|
| 41 |
+
"device": "cuda",
|
| 42 |
+
"dtype": "bfloat16",
|
| 43 |
+
"attn_implementation": "sdpa"
|
| 44 |
+
},
|
| 45 |
+
"next_token_logits": [
|
| 46 |
+
[
|
| 47 |
+
0.0,
|
| 48 |
+
-0.3203125,
|
| 49 |
+
-0.03662109375,
|
| 50 |
+
-0.038818359375,
|
| 51 |
+
0.04443359375
|
| 52 |
+
],
|
| 53 |
+
[
|
| 54 |
+
0.0,
|
| 55 |
+
-0.2197265625,
|
| 56 |
+
-0.001983642578125,
|
| 57 |
+
-0.08349609375,
|
| 58 |
+
0.15625
|
| 59 |
+
]
|
| 60 |
+
],
|
| 61 |
+
"predicted_tokens": [
|
| 62 |
+
386,
|
| 63 |
+
386
|
| 64 |
+
],
|
| 65 |
+
"training": {
|
| 66 |
+
"loss": 12.488280296325684,
|
| 67 |
+
"gradients": {
|
| 68 |
+
"model.layers.0.self_attn.q_proj.weight": true,
|
| 69 |
+
"model.layers.0.mlp.down_proj.weight": true
|
| 70 |
+
},
|
| 71 |
+
"updated": true
|
| 72 |
+
}
|
| 73 |
+
}
|
| 74 |
+
}
|
integrations/transformers/expectations/cuda/tiny-random-aya-base-mic.eager.json
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/tiny-random-aya-base-mic",
|
| 4 |
+
"contract_name": "Cohere2",
|
| 5 |
+
"contract_revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "eager",
|
| 8 |
+
"target": "tiny-random-aya-base-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.23.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cu130",
|
| 15 |
+
"torchvision": "0.29.2",
|
| 16 |
+
"torchaudio": "2.11.0+cu130",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.4.6",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.16.0.dev0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.4.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 7.41,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torchvision": "0.29.2 (pinned 0.29.1)",
|
| 32 |
+
"numpy": "2.4.6 (pinned 2.3.5)",
|
| 33 |
+
"pillow": "13.0.0 (pinned 12.3.0)",
|
| 34 |
+
"accelerate": "1.16.0.dev0 (pinned 1.15.0)",
|
| 35 |
+
"huggingface-hub": "2.4.0 (pinned 2.3.0)"
|
| 36 |
+
},
|
| 37 |
+
"platform": "cuda"
|
| 38 |
+
},
|
| 39 |
+
"outputs": {
|
| 40 |
+
"runtime": {
|
| 41 |
+
"device": "cuda",
|
| 42 |
+
"dtype": "float32",
|
| 43 |
+
"attn_implementation": "eager"
|
| 44 |
+
},
|
| 45 |
+
"next_token_logits": [
|
| 46 |
+
[
|
| 47 |
+
0.0,
|
| 48 |
+
-0.3193780779838562,
|
| 49 |
+
-0.03577788546681404,
|
| 50 |
+
-0.039259690791368484,
|
| 51 |
+
0.04475041851401329
|
| 52 |
+
],
|
| 53 |
+
[
|
| 54 |
+
0.0,
|
| 55 |
+
-0.2189594805240631,
|
| 56 |
+
-0.002652532421052456,
|
| 57 |
+
-0.08413229137659073,
|
| 58 |
+
0.15724743902683258
|
| 59 |
+
]
|
| 60 |
+
],
|
| 61 |
+
"predicted_tokens": [
|
| 62 |
+
386,
|
| 63 |
+
386
|
| 64 |
+
],
|
| 65 |
+
"training": {
|
| 66 |
+
"loss": 12.494779586791992,
|
| 67 |
+
"gradients": {
|
| 68 |
+
"model.layers.0.self_attn.q_proj.weight": true,
|
| 69 |
+
"model.layers.0.mlp.down_proj.weight": true
|
| 70 |
+
},
|
| 71 |
+
"updated": true
|
| 72 |
+
}
|
| 73 |
+
}
|
| 74 |
+
}
|
integrations/transformers/expectations/cuda/tiny-random-aya-base-mic.json
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/tiny-random-aya-base-mic",
|
| 4 |
+
"contract_name": "Cohere2",
|
| 5 |
+
"contract_revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "default",
|
| 8 |
+
"target": "tiny-random-aya-base-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "8bf9cb94b17f3eb31ad996d47ba754331e02b2c6",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.23.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cu130",
|
| 15 |
+
"torchvision": "0.29.2",
|
| 16 |
+
"torchaudio": "2.11.0+cu130",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.4.6",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.16.0.dev0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.4.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 8.17,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torchvision": "0.29.2 (pinned 0.29.1)",
|
| 32 |
+
"numpy": "2.4.6 (pinned 2.3.5)",
|
| 33 |
+
"pillow": "13.0.0 (pinned 12.3.0)",
|
| 34 |
+
"accelerate": "1.16.0.dev0 (pinned 1.15.0)",
|
| 35 |
+
"huggingface-hub": "2.4.0 (pinned 2.3.0)"
|
| 36 |
+
},
|
| 37 |
+
"platform": "cuda"
|
| 38 |
+
},
|
| 39 |
+
"outputs": {
|
| 40 |
+
"runtime": {
|
| 41 |
+
"device": "cuda",
|
| 42 |
+
"dtype": "float32",
|
| 43 |
+
"attn_implementation": "sdpa"
|
| 44 |
+
},
|
| 45 |
+
"next_token_logits": [
|
| 46 |
+
[
|
| 47 |
+
0.0,
|
| 48 |
+
-0.3193780481815338,
|
| 49 |
+
-0.03577787056565285,
|
| 50 |
+
-0.039259668439626694,
|
| 51 |
+
0.04475036635994911
|
| 52 |
+
],
|
| 53 |
+
[
|
| 54 |
+
0.0,
|
| 55 |
+
-0.2189594954252243,
|
| 56 |
+
-0.002652491210028529,
|
| 57 |
+
-0.08413227647542953,
|
| 58 |
+
0.157247394323349
|
| 59 |
+
]
|
| 60 |
+
],
|
| 61 |
+
"predicted_tokens": [
|
| 62 |
+
386,
|
| 63 |
+
386
|
| 64 |
+
],
|
| 65 |
+
"training": {
|
| 66 |
+
"loss": 12.48836612701416,
|
| 67 |
+
"gradients": {
|
| 68 |
+
"model.layers.0.self_attn.q_proj.weight": true,
|
| 69 |
+
"model.layers.0.mlp.down_proj.weight": true
|
| 70 |
+
},
|
| 71 |
+
"updated": true
|
| 72 |
+
}
|
| 73 |
+
}
|
| 74 |
+
}
|
integrations/transformers/usage.md
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Cohere2: causal language modeling with Transformers
|
| 2 |
+
|
| 3 |
+
This page is the Transformers contract of the Cohere2 MIC and also a usage example. [doc-builder](https://github.com/huggingface/doc-builder/blob/main/docs/runnable-code-blocks.md) renders the `runnable` blocks below as documentation, and its pytest plugin runs them as written against `optimum-intel-internal-testing/tiny-random-aya-base`. The MIC runner runs the same blocks against the slim fixture and the checkpoint in [`integration.json`](../../integration.json). Lines marked `# doc-builder: hide` only matter there: they swap in the pinned checkpoint or variant and print the result record.
|
| 4 |
+
|
| 5 |
+
## Predict the next token
|
| 6 |
+
|
| 7 |
+
```py runnable:causal_lm
|
| 8 |
+
import torch
|
| 9 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed
|
| 10 |
+
import json # doc-builder: hide
|
| 11 |
+
import os # doc-builder: hide
|
| 12 |
+
|
| 13 |
+
model_id = "optimum-intel-internal-testing/tiny-random-aya-base"
|
| 14 |
+
model_id = os.environ.get("MIC_MODEL", model_id) # doc-builder: hide
|
| 15 |
+
variant = json.loads(os.environ.get("MIC_MODEL_KWARGS", "{}")) # doc-builder: hide
|
| 16 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 17 |
+
set_seed(0) # seeds torch, NumPy, and Python: loading, dropout, and data augmentation draw random numbers
|
| 18 |
+
|
| 19 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
| 20 |
+
# This checkpoint defines no pad token; use the EOS token and left padding so the last position is a real token.
|
| 21 |
+
if tokenizer.pad_token is None:
|
| 22 |
+
tokenizer.pad_token = tokenizer.eos_token
|
| 23 |
+
tokenizer.padding_side = "left"
|
| 24 |
+
|
| 25 |
+
model = AutoModelForCausalLM.from_pretrained(model_id).to(device)
|
| 26 |
+
if variant: # doc-builder: hide
|
| 27 |
+
del model # doc-builder: hide
|
| 28 |
+
model = AutoModelForCausalLM.from_pretrained(model_id, **variant).to(device) # doc-builder: hide
|
| 29 |
+
|
| 30 |
+
prompts = [
|
| 31 |
+
"The capital of France is",
|
| 32 |
+
"The largest planet in the solar system is",
|
| 33 |
+
]
|
| 34 |
+
inputs = tokenizer(prompts, return_tensors="pt", padding=True).to(device)
|
| 35 |
+
|
| 36 |
+
with torch.no_grad():
|
| 37 |
+
logits = model(**inputs).logits
|
| 38 |
+
# Left padding: the last position is the final real token of each prompt.
|
| 39 |
+
next_logits = logits[:, -1, :]
|
| 40 |
+
predicted = next_logits.argmax(dim=-1)
|
| 41 |
+
print(tokenizer.batch_decode(predicted.unsqueeze(-1)))
|
| 42 |
+
```
|
| 43 |
+
|
| 44 |
+
## One training step
|
| 45 |
+
|
| 46 |
+
```py runnable:causal_lm:2
|
| 47 |
+
# Language-model labels: copy input_ids and mask padded positions.
|
| 48 |
+
labels = inputs["input_ids"].clone()
|
| 49 |
+
labels[labels == tokenizer.pad_token_id] = -100
|
| 50 |
+
|
| 51 |
+
model.train()
|
| 52 |
+
optimizer = torch.optim.SGD([p for p in model.parameters() if p.requires_grad], lr=0.01)
|
| 53 |
+
loss = model(**inputs, labels=labels).loss
|
| 54 |
+
loss.backward()
|
| 55 |
+
before = {n: p.detach().cpu().clone() for n, p in model.named_parameters() if p.requires_grad} # doc-builder: hide
|
| 56 |
+
optimizer.step()
|
| 57 |
+
print(f"loss: {loss.item():.4f}")
|
| 58 |
+
|
| 59 |
+
if os.environ.get("MIC_RUNNER"): # doc-builder: hide
|
| 60 |
+
named = dict(model.named_parameters())
|
| 61 |
+
checked = ("model.layers.0.self_attn.q_proj.weight", "model.layers.0.mlp.down_proj.weight")
|
| 62 |
+
result = {
|
| 63 |
+
"runtime": {
|
| 64 |
+
"device": model.device.type,
|
| 65 |
+
"dtype": str(model.dtype).removeprefix("torch."),
|
| 66 |
+
"attn_implementation": model.config._attn_implementation,
|
| 67 |
+
},
|
| 68 |
+
"next_token_logits": next_logits[:, :5].float().tolist(),
|
| 69 |
+
"predicted_tokens": predicted.tolist(),
|
| 70 |
+
"training": {
|
| 71 |
+
"loss": loss.item(),
|
| 72 |
+
"gradients": {n: bool(named[n].grad is not None and torch.count_nonzero(named[n].grad) > 0 and torch.isfinite(named[n].grad).all()) for n in checked},
|
| 73 |
+
"updated": any(not torch.equal(p.detach().cpu(), before[n]) for n, p in model.named_parameters() if n in before),
|
| 74 |
+
},
|
| 75 |
+
}
|
| 76 |
+
print(json.dumps(result))
|
| 77 |
+
```
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c4cf5b8ea901291cfeda11936929fa25869fc600d2529620e9307241c6cc6a06
|
| 3 |
+
size 67439232
|
special_tokens_map.json
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token": {
|
| 3 |
+
"content": "<BOS_TOKEN>",
|
| 4 |
+
"lstrip": false,
|
| 5 |
+
"normalized": false,
|
| 6 |
+
"rstrip": false,
|
| 7 |
+
"single_word": false
|
| 8 |
+
},
|
| 9 |
+
"eos_token": {
|
| 10 |
+
"content": "<EOS_TOKEN>",
|
| 11 |
+
"lstrip": false,
|
| 12 |
+
"normalized": false,
|
| 13 |
+
"rstrip": false,
|
| 14 |
+
"single_word": false
|
| 15 |
+
},
|
| 16 |
+
"pad_token": {
|
| 17 |
+
"content": "<PAD>",
|
| 18 |
+
"lstrip": false,
|
| 19 |
+
"normalized": false,
|
| 20 |
+
"rstrip": false,
|
| 21 |
+
"single_word": false
|
| 22 |
+
},
|
| 23 |
+
"unk_token": {
|
| 24 |
+
"content": "<UNK>",
|
| 25 |
+
"lstrip": false,
|
| 26 |
+
"normalized": false,
|
| 27 |
+
"rstrip": false,
|
| 28 |
+
"single_word": false
|
| 29 |
+
}
|
| 30 |
+
}
|
tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8f21f6c4f761c192f486ea2c5b06b62b3ef30819b33dc105bdf8b26c8e7974f6
|
| 3 |
+
size 21374973
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,141 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_bos_token": true,
|
| 3 |
+
"add_eos_token": false,
|
| 4 |
+
"add_prefix_space": false,
|
| 5 |
+
"added_tokens_decoder": {
|
| 6 |
+
"0": {
|
| 7 |
+
"content": "<PAD>",
|
| 8 |
+
"lstrip": false,
|
| 9 |
+
"normalized": false,
|
| 10 |
+
"rstrip": false,
|
| 11 |
+
"single_word": false,
|
| 12 |
+
"special": true
|
| 13 |
+
},
|
| 14 |
+
"1": {
|
| 15 |
+
"content": "<MASK_TOKEN>",
|
| 16 |
+
"lstrip": false,
|
| 17 |
+
"normalized": false,
|
| 18 |
+
"rstrip": false,
|
| 19 |
+
"single_word": false,
|
| 20 |
+
"special": true
|
| 21 |
+
},
|
| 22 |
+
"2": {
|
| 23 |
+
"content": "<BOS_TOKEN>",
|
| 24 |
+
"lstrip": false,
|
| 25 |
+
"normalized": false,
|
| 26 |
+
"rstrip": false,
|
| 27 |
+
"single_word": false,
|
| 28 |
+
"special": true
|
| 29 |
+
},
|
| 30 |
+
"3": {
|
| 31 |
+
"content": "<EOS_TOKEN>",
|
| 32 |
+
"lstrip": false,
|
| 33 |
+
"normalized": false,
|
| 34 |
+
"rstrip": false,
|
| 35 |
+
"single_word": false,
|
| 36 |
+
"special": true
|
| 37 |
+
},
|
| 38 |
+
"4": {
|
| 39 |
+
"content": "<UNK>",
|
| 40 |
+
"lstrip": false,
|
| 41 |
+
"normalized": false,
|
| 42 |
+
"rstrip": false,
|
| 43 |
+
"single_word": false,
|
| 44 |
+
"special": true
|
| 45 |
+
},
|
| 46 |
+
"5": {
|
| 47 |
+
"content": "<|START_OF_TURN_TOKEN|>",
|
| 48 |
+
"lstrip": false,
|
| 49 |
+
"normalized": false,
|
| 50 |
+
"rstrip": false,
|
| 51 |
+
"single_word": false,
|
| 52 |
+
"special": true
|
| 53 |
+
},
|
| 54 |
+
"6": {
|
| 55 |
+
"content": "<|END_OF_TURN_TOKEN|>",
|
| 56 |
+
"lstrip": false,
|
| 57 |
+
"normalized": false,
|
| 58 |
+
"rstrip": false,
|
| 59 |
+
"single_word": false,
|
| 60 |
+
"special": true
|
| 61 |
+
},
|
| 62 |
+
"7": {
|
| 63 |
+
"content": "<|USER_TOKEN|>",
|
| 64 |
+
"lstrip": false,
|
| 65 |
+
"normalized": false,
|
| 66 |
+
"rstrip": false,
|
| 67 |
+
"single_word": false,
|
| 68 |
+
"special": true
|
| 69 |
+
},
|
| 70 |
+
"8": {
|
| 71 |
+
"content": "<|CHATBOT_TOKEN|>",
|
| 72 |
+
"lstrip": false,
|
| 73 |
+
"normalized": false,
|
| 74 |
+
"rstrip": false,
|
| 75 |
+
"single_word": false,
|
| 76 |
+
"special": true
|
| 77 |
+
},
|
| 78 |
+
"9": {
|
| 79 |
+
"content": "<|SYSTEM_TOKEN|>",
|
| 80 |
+
"lstrip": false,
|
| 81 |
+
"normalized": false,
|
| 82 |
+
"rstrip": false,
|
| 83 |
+
"single_word": false,
|
| 84 |
+
"special": true
|
| 85 |
+
},
|
| 86 |
+
"10": {
|
| 87 |
+
"content": "<|NEW_FILE|>",
|
| 88 |
+
"lstrip": false,
|
| 89 |
+
"normalized": false,
|
| 90 |
+
"rstrip": false,
|
| 91 |
+
"single_word": false,
|
| 92 |
+
"special": true
|
| 93 |
+
},
|
| 94 |
+
"11": {
|
| 95 |
+
"content": "<|BEGINNING_OF_PREFIX_FIM_TOKEN|>",
|
| 96 |
+
"lstrip": false,
|
| 97 |
+
"normalized": false,
|
| 98 |
+
"rstrip": false,
|
| 99 |
+
"single_word": false,
|
| 100 |
+
"special": true
|
| 101 |
+
},
|
| 102 |
+
"12": {
|
| 103 |
+
"content": "<|BEGINNING_OF_MIDDLE_FIM_TOKEN|>",
|
| 104 |
+
"lstrip": false,
|
| 105 |
+
"normalized": false,
|
| 106 |
+
"rstrip": false,
|
| 107 |
+
"single_word": false,
|
| 108 |
+
"special": true
|
| 109 |
+
},
|
| 110 |
+
"13": {
|
| 111 |
+
"content": "<|BEGINNING_OF_SUFFIX_FIM_TOKEN|>",
|
| 112 |
+
"lstrip": false,
|
| 113 |
+
"normalized": false,
|
| 114 |
+
"rstrip": false,
|
| 115 |
+
"single_word": false,
|
| 116 |
+
"special": true
|
| 117 |
+
},
|
| 118 |
+
"14": {
|
| 119 |
+
"content": "<|END_OF_MIDDLE_FIM_TOKEN|>",
|
| 120 |
+
"lstrip": false,
|
| 121 |
+
"normalized": false,
|
| 122 |
+
"rstrip": false,
|
| 123 |
+
"single_word": false,
|
| 124 |
+
"special": true
|
| 125 |
+
}
|
| 126 |
+
},
|
| 127 |
+
"bos_token": "<BOS_TOKEN>",
|
| 128 |
+
"clean_up_tokenization_spaces": false,
|
| 129 |
+
"eos_token": "<EOS_TOKEN>",
|
| 130 |
+
"extra_special_tokens": {},
|
| 131 |
+
"legacy": true,
|
| 132 |
+
"merges_file": null,
|
| 133 |
+
"model_max_length": 1000000000000000019884624838656,
|
| 134 |
+
"pad_token": "<PAD>",
|
| 135 |
+
"sp_model_kwargs": {},
|
| 136 |
+
"spaces_between_special_tokens": false,
|
| 137 |
+
"tokenizer_class": "CohereTokenizer",
|
| 138 |
+
"unk_token": "<UNK>",
|
| 139 |
+
"use_default_system_prompt": false,
|
| 140 |
+
"vocab_file": null
|
| 141 |
+
}
|