Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,502 @@
 ---
+library_name: transformers
 license: apache-2.0
+pipeline_tag: image-text-to-text
+language:
+  - en
+  - de
+  - fr
+  - es
+  - it
+  - pt
+  - hi
+  - ja
+  - ko
+  - zh
+  - ar
+tags:
+  - vision
+  - multimodal
+  - conversational
+  - multilingual
+  - native-resolution
 ---
+
+# North Micro Vision Instruct
+![North-Micro-Vision_Hero](https://cdn-uploads.huggingface.co/production/uploads/66d732effe6684fc16b12c28/qQQSd5Pldz30hvGMhNMq_.png)
+
+North Micro Vision Instruct is a 2.4B-parameter open-weight vision-language model with native-resolution image support, released under the Apache 2.0 license. It is designed as a compact foundation for prototyping, task-specific fine-tuning, and specialized multimodal applications.
+
+Developed by [Cohere](https://cohere.com/).
+
+> **Technical deep dive:** Read the [North Micro Vision technical blog post](https://huggingface.co/blog/CohereLabs/meet-north-micro-vision-instruct) for architecture, training, and evaluation details.
+
+## Highlights
+
+- Native-resolution image processing that preserves aspect ratios and fine visual detail.
+- Broad image-understanding capabilities across VQA, captioning, grounding, OCR, charts, and documents.
+- Multilingual and multi-image support.
+- Compact 2.4B-parameter scale suited to customization and deployment experimentation.
+- Apache 2.0-licensed model weights.
+
+## Model Details
+| Property | Value |
+| --- | --- |
+| Model ID | `CohereLabs/North-Micro-Vision-Instruct` |
+| Total parameters | 2.4B |
+| Language model | 2B parameters |
+| Vision encoder | 400M parameters |
+| Inputs | Text and one or more images |
+| Output | Text |
+| Languages | English, German, French, Spanish, Italian, Portuguese, Hindi, Japanese, Korean, Chinese, and Arabic |
+| Tokenizer vocabulary size | 262,144 |
+| LM Backbone context window | 128K tokens |
+| Multimodal training context | 8K tokens |
+| Checkpoint precision | bfloat16 |
+| License | Apache 2.0 |
+
+The language backbone supports a 128K-token context window, but multimodal training used sequences of up to 8K tokens. Longer multimodal contexts have not yet been validated and may rely on extrapolation.
+
+## Quickstart
+
+### Installation
+
+North Micro Vision requires Transformers 5.15.0. Until that version is released, install Transformers from source:
+
+```bash
+uv pip install "git+https://github.com/huggingface/transformers.git"
+```
+
+Once Transformers 5.15.0 is available on PyPI, install the released package with:
+
+```bash
+uv pip install "transformers==5.15.0"
+```
+
+### Transformers
+
+The following example loads an image from a URL and asks the model to describe it. Prompts can interleave text with one or more images; for text-only prompts, omit the image entries.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoProcessor
+
+model_id = "CohereLabs/North-Micro-Vision-Instruct"
+
+processor = AutoProcessor.from_pretrained(
+    model_id,
+    trust_remote_code=True,
+)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    dtype="auto",
+    device_map="auto",
+    trust_remote_code=True,
+)
+
+# To enable Flash Attention 2, load the model with the following settings:
+# model = AutoModelForCausalLM.from_pretrained(
+#     model_id,
+#     dtype=torch.bfloat16,
+#     attn_implementation="flash_attention_2",
+#     device_map="auto",
+#     trust_remote_code=True,
+# )
+
+image_url = "https://cdn-uploads.huggingface.co/production/uploads/66d732effe6684fc16b12c28/Io_5OCmftsmH-n158ZtPs.png"
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {"type": "image", "url": image_url},
+            {"type": "text", "text": "Describe this image."},
+        ],
+    }
+]
+
+inputs = processor.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt",
+    return_dict=True,
+).to(model.device)
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=128,
+)
+
+input_length = inputs["input_ids"].shape[-1]
+response = processor.decode(
+    outputs[0][input_length:],
+    skip_special_tokens=True,
+)
+print(response)
+```
+
+For multi-image prompts, add multiple image entries to the message content before the text instruction.
+
+Bounding coordinates are returned in the [x1,x2,y1,y2] format on a normalized 0-1000 scale and should be mapped back to the original image dimensions.
+
+## Intended Use
+
+North Micro Vision Instruct is intended for research and development use cases such as:
+
+- Prototyping and task-specific fine-tuning.
+- General visual question answering and image captioning.
+- Multilingual and multi-image understanding.
+- Visual grounding and spatial understanding.
+- OCR, chart and document understanding, and structured information extraction.
+
+## Limitations
+
+- The model is intended as a compact foundation for customization rather than a replacement for larger general-purpose chat assistants.
+- It is not a reasoning model and has limited math and code-generation capabilities.
+- Tool calling and agentic workflows are not supported.
+- The model was trained without system prompts.
+- Multimodal training used an 8K-token context; longer contexts have not been validated.
+- Native-resolution inputs can increase memory use and latency as image dimensions grow.
+
+## Benchmark Results
+
+The complete comparison is provided below. See the [technical blog post](https://huggingface.co/blog/CohereLabs/meet-north-micro-vision-instruct) for benchmarking methodology and additional technical details.
+
+<table>
+    <thead>
+        <tr>
+            <th></th>
+            <th style="font-weight: bold; background-color: rgba(127, 127, 127, 0.08);">North-Micro-Vision-Instruct</th>
+            <th>Ministral-3-3B-Instruct</th>
+            <th>LFM2.5-VL-1.6B</th>
+            <th>Phi-3.5-vision-instruct</th>
+            <th>Gemma-4-E2B-it</th>
+            <th>Qwen3-VL-2B-Instruct</th>
+            <th>Qwen3.5-2B-Instruct</th>
+            <th>SmolVLM2.2B</th>
+        </tr>
+    </thead>
+    <tbody>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">Size</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">2.4B</td>
+            <td>3.8B</td>
+            <td>1.6B</td>
+            <td>4.2B</td>
+            <td>5.1B</td>
+            <td>2.2B</td>
+            <td>2.1B</td>
+            <td>2.2B</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">License</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">Apache 2.0</td>
+            <td>Apache 2.0</td>
+            <td>LFM v1.0</td>
+            <td>MIT</td>
+            <td>Apache 2.0</td>
+            <td>Apache 2.0</td>
+            <td>Apache 2.0</td>
+            <td>Apache 2.0</td>
+        </tr>
+        <tr>
+            <th colspan="9" style="text-align: left; background-color: rgba(127, 127, 127, 0.16);">General VQA</th>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">MMBench<sub>DEV_EN_V11</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.687</td>
+            <td>0.692</td>
+            <td>0.696</td>
+            <td>0.731</td>
+            <td>0.693</td>
+            <td>0.744</td>
+            <td>0.760</td>
+            <td>0.674</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">MMStar</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.518</td>
+            <td>0.531</td>
+            <td>0.508</td>
+            <td>0.495</td>
+            <td>0.529</td>
+            <td>0.506</td>
+            <td>0.614</td>
+            <td>0.460</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">RealWorldQA</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.622</td>
+            <td>0.583</td>
+            <td>0.642</td>
+            <td>0.580</td>
+            <td>0.507</td>
+            <td>0.646</td>
+            <td>0.693</td>
+            <td>0.567</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">GQA<sub>TestDev_Balanced</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.574</td>
+            <td>0.544</td>
+            <td>0.395</td>
+            <td>0.650</td>
+            <td>0.387</td>
+            <td>0.572</td>
+            <td>0.539</td>
+            <td>0.000<sup>†</sup></td>
+        </tr>
+        <tr>
+            <th colspan="9" style="text-align: left; background-color: rgba(127, 127, 127, 0.16);">Multilingual</th>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">MTL<sub>MMBench_DEV</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.636</td>
+            <td>0.674</td>
+            <td>0.623</td>
+            <td>0.619</td>
+            <td>0.648</td>
+            <td>0.664</td>
+            <td>0.669</td>
+            <td>0.454</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">MMMB</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.728</td>
+            <td>0.734</td>
+            <td>0.717</td>
+            <td>0.686</td>
+            <td>0.743</td>
+            <td>0.723</td>
+            <td>0.745</td>
+            <td>0.577</td>
+        </tr>
+        <tr>
+            <th colspan="9" style="text-align: left; background-color: rgba(127, 127, 127, 0.16);">Multi-image</th>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">BLINK</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.527</td>
+            <td>0.471</td>
+            <td>0.484</td>
+            <td>0.561</td>
+            <td>0.468</td>
+            <td>0.514</td>
+            <td>0.563</td>
+            <td>0.420</td>
+        </tr>
+        <tr>
+            <th colspan="9" style="text-align: left; background-color: rgba(127, 127, 127, 0.16);">Chart / Document / OCR</th>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">ChartQA<sub>Test</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.808</td>
+            <td>0.791</td>
+            <td>0.739</td>
+            <td>0.821</td>
+            <td>0.422</td>
+            <td>0.693</td>
+            <td>0.775</td>
+            <td>0.682</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">DocVQA<sub>VAL</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.921</td>
+            <td>0.896</td>
+            <td>0.877</td>
+            <td>0.860</td>
+            <td>0.732</td>
+            <td>0.825</td>
+            <td>0.926</td>
+            <td>0.799</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">InfoVQA<sub>VAL</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.652</td>
+            <td>0.589</td>
+            <td>0.627</td>
+            <td>0.561</td>
+            <td>0.380</td>
+            <td>0.622</td>
+            <td>0.731</td>
+            <td>0.383</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">OCRBench<sub>v2_en</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.367</td>
+            <td>0.414</td>
+            <td>0.415</td>
+            <td>0.339</td>
+            <td>0.435</td>
+            <td>0.417</td>
+            <td>0.481</td>
+            <td>0.304</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">OCRBench</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.792</td>
+            <td>0.735</td>
+            <td>0.802</td>
+            <td>0.642</td>
+            <td>0.719</td>
+            <td>0.751</td>
+            <td>0.861</td>
+            <td>0.727</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">AI2D_TEST</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.775</td>
+            <td>0.741</td>
+            <td>0.728</td>
+            <td>0.790</td>
+            <td>0.712</td>
+            <td>0.713</td>
+            <td>0.752</td>
+            <td>0.697</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">CharXiv<sub>DQ</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.600</td>
+            <td>0.766</td>
+            <td>0.516</td>
+            <td>0.637</td>
+            <td>0.751</td>
+            <td>0.595</td>
+            <td>0.761</td>
+            <td>0.482</td>
+        </tr>
+        <tr>
+            <th colspan="9" style="text-align: left; background-color: rgba(127, 127, 127, 0.16);">STEM</th>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">MMMU<sub>DEV_VAL</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.329</td>
+            <td>0.508</td>
+            <td>0.380</td>
+            <td>0.432</td>
+            <td>0.477</td>
+          P1+r436F=323536\P1+r6B75=1B4F41\P1+r6B64=1B4F42\P1+r6B72=1B4F43\P1+r6B6C=1B4F44\P1+r2332=1B5B313B3248\P1+r2334=1B5B313B3244\P1+r2569=1B5B313B3243\P1+r2A37=1B5B313B3246\P1+r6B31=1B4F50\  <td>0.379</td>
+            <td>0.474</td>
+            <td>0.399</td>
+        </tr>
+        <tr>
+            <th colspan="9" style="text-align: left; background-color: rgba(127, 127, 127, 0.16);">Grounding / Counting</th>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">RefCOCO<sub>avg</sub>*</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.732</td>
+            <td>0.317</td>
+            <td>0.581</td>
+            <td>0.451</td>
+            <td>0.084</td>
+            <td>0.304</td>
+            <td>0.785</td>
+            <td>0.018</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">CountBench</th>
+            <td style="background-color: rgba(127, 1P1$r2 q\[?12;2$y27, 127, 0.08);">0.725</td>
+            <td>0.737</td>
+            <td>0.910</td>
+            <td>0.645</td>
+            <td>0.534</td>
+            <td>0.848</td>
+            <td>0.805</td>
+            <td>0.764</td>
+        </tr>
+        <tr>
+            <th colspan="9" style="text-align: left; background-color: rgba(127, 127, 127, 0.16);">Robustness / Hallucination</th>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">HallusionBench</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.615</td>
+            <td>0.652</td>
+            <td>0.601</td>
+            <td>0.585</td>
+            <td>0.598</td>
+            <td>0.673</td>
+            <td>0.655</td>
+            <td>0.600</td>
+        </tr>
+        <tr>
+            <th colspan="9" style="text-align: left; background-color: rgba(127, 127, 127, 0.16);">Text</th>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">MMLU<sub>test</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.504</td>
+            <td>0.660</td>
+            <td>0.464</td>
+            <td>0.355</td>
+            <td>0.692</td>
+            <td>0.630</td>
+            <td>0.543</td>
+            <td>0.084</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">MMLU-Pro<sub>test</sub></th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.307</td>
+            <td>0.475</td>
+            <td>0.199</td>
+            <td>0.286</td>
+            <td>0.441</td>
+            <td>0.428</td>
+            <td>0.298</td>
+            <td>0.099</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">Multi-If</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.373</td>
+            <td>0.470</td>
+            <td>0.443</td>
+            <td>0.304</td>
+            <td>0.687</td>
+            <td>0.523</td>
+            <td>0.464</td>
+            <td>0.236</td>
+        </tr>
+        <tr>
+            <th style="text-align: left; font-weight: normal;">IFEval</th>
+            <td style="background-color: rgba(127, 127, 127, 0.08);">0.749</td>
+            <td>0.725</td>
+            <td>0.776</td>
+            <td>0.543</td>
+            <td>0.869</td>
+            <td>0.734</td>
+            <td>0.679</td>
+            <td>0.501</td>
+        </tr>
+    </tbody>
+</table>
+<p><small>*Averaged over RefCOCO_val, RefCOCO_testA, RefCOCO_testB, RefCOCO+_val, RefCOCO+_testA, RefCOCO+_testB, RefCOCOg_val, RefCOCOg_test.</small></p>
+<p><small>†SmolVLM2.2B's GQA output was scored as 0.000 under VLMEvalKit's answer-extraction rules.</small></p>
+
+
+
+### vLLM
+
+Public vLLM support is coming soon. For both text-only and vision-language inference, use:
+
+```python
+temperature = 0.7
+top_p = 0.8
+top_k = 20
+min_p = 0.0
+presence_penalty = 1.5
+repetition_penalty = 1.0
+```
+
+## Citation
+
+```bibtex
+@misc{cohere_north_micro_vision_instruct,
+    title = {{North Micro Vision}: A 2.4B Native-Resolution Vision-Language Model},
+    url = {https://huggingface.co/blog/CohereLabs/meet-north-micro-vision-instruct},
+    author = {{Team Cohere}},
+    month = {August},
+    year = {2026}
+}
+```
+
+## Contact
+
+For errors or questions about this model card, contact [Cohere Labs](mailto:labs@cohere.com).

chat_template.jinja

@@ -0,0 +1 @@
+{{ bos_token }}{%- for message in messages %}{%- if message.role == 'system' %}{{- '<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>' }}{%- if message.content is string %}{{- message.content }}{%- else %}{%- for content in message.content %}{%- if 'text' in content %}{{- content.text }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|END_OF_TURN_TOKEN|>' }}{%- elif message.role == 'user' %}{{- '<|START_OF_TURN_TOKEN|><|USER_TOKEN|>' }}{%- if message.content is string %}{{- message.content }}{%- else %}{%- for content in message.content %}{%- if content.type == 'image' or 'image' in content or 'image_url' in content %}<|VISION_START|><|IMAGE_PAD|><|VISION_END|>{%- elif content.type == 'video' or 'video' in content %}<|VISION_START|><|VIDEO_PAD|><|VISION_END|>{%- elif 'text' in content %}{{- content.text }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|END_OF_TURN_TOKEN|>' }}{%- elif message.role == 'assistant' or message.role == 'chatbot' %}{{- '<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>' }}{%- if message.content is string %}{{- message.content }}{%- else %}{%- for content in message.content %}{%- if 'text' in content %}{{- content.text }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|END_OF_TURN_TOKEN|>' }}{%- endif %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>' }}{%- endif %}

config.json

@@ -0,0 +1,155 @@
+{
+  "transformers_version": "5.15.0.dev0",
+  "architectures": [
+    "CohereCompassForConditionalGeneration"
+  ],
+  "output_hidden_states": false,
+  "return_dict": true,
+  "dtype": "bfloat16",
+  "chunk_size_feed_forward": 0,
+  "is_encoder_decoder": false,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1"
+  },
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1
+  },
+  "problem_type": null,
+  "text_config": {
+    "architectures": null,
+    "output_hidden_states": false,
+    "return_dict": true,
+    "dtype": "bfloat16",
+    "chunk_size_feed_forward": 0,
+    "is_encoder_decoder": false,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "problem_type": null,
+    "vocab_size": 262144,
+    "hidden_size": 2048,
+    "intermediate_size": 6144,
+    "logit_scale": 0.25,
+    "num_hidden_layers": 28,
+    "num_attention_heads": 16,
+    "num_key_value_heads": 8,
+    "hidden_act": "silu",
+    "max_position_embeddings": 500000,
+    "initializer_range": 0.02,
+    "layer_norm_eps": 1e-05,
+    "use_cache": true,
+    "pad_token_id": 0,
+    "bos_token_id": 2,
+    "eos_token_id": 255001,
+    "tie_word_embeddings": true,
+    "rope_parameters": {
+      "sliding_attention": {
+        "mrope_interleaved": true,
+        "mrope_section": [
+          24,
+          20,
+          20
+        ],
+        "rope_type": "default",
+        "rope_theta": 50000
+      },
+      "full_attention": null,
+      "rope_theta": 10000.0,
+      "rope_type": "default"
+    },
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "sliding_window": 4096,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "score_shift_a": null,
+    "score_shift_b": null,
+    "pooling": null,
+    "head_dim": 128,
+    "_name_or_path": "",
+    "model_type": "cohere_compass_text",
+    "output_attentions": false
+  },
+  "vision_config": {
+    "architectures": null,
+    "output_hidden_states": false,
+    "return_dict": true,
+    "dtype": "bfloat16",
+    "chunk_size_feed_forward": 0,
+    "is_encoder_decoder": false,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "problem_type": null,
+    "depth": 27,
+    "hidden_size": 1152,
+    "hidden_act": "gelu_pytorch_tanh",
+    "intermediate_size": 4304,
+    "num_heads": 16,
+    "in_channels": 3,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2,
+    "out_hidden_size": 2048,
+    "num_position_embeddings": 2304,
+    "deepstack_visual_indexes": [
+      8,
+      16,
+      24
+    ],
+    "initializer_range": 0.02,
+    "_name_or_path": "",
+    "model_type": "cohere_compass_vision",
+    "output_attentions": false
+  },
+  "image_token_id": 255031,
+  "video_token_id": 255032,
+  "vision_start_token_id": 255028,
+  "vision_end_token_id": 255029,
+  "tie_word_embeddings": true,
+  "_name_or_path": "",
+  "eos_token_id": 255001,
+  "pad_token_id": 0,
+  "model_type": "cohere_compass",
+  "output_attentions": false
+}

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "bos_token_id": 2,
+  "eos_token_id": [
+    255001
+  ],
+  "pad_token_id": 0,
+  "do_sample": true,
+  "temperature": 0.7,
+  "top_p": 0.8,
+  "top_k": 20
+}
+

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd9a9ed867111bbc6b6e062bf0b3b9c88af183eb41003083501a76c84e912046
+size 4969765248

preprocessor_config.json

@@ -0,0 +1,37 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": false,
+  "device": null,
+  "disable_grouping": null,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_pad": null,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "input_data_format": null,
+  "merge_size": 2,
+  "pad_size": null,
+  "patch_size": 16,
+  "processor_class": "CohereCompassProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "return_tensors": null,
+  "size": {
+    "longest_edge": 16777216,
+    "shortest_edge": 65536
+  },
+  "temporal_patch_size": 2,
+  "image_processor_type": "CohereCompassImageProcessor"
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "additional_special_tokens": [
+    "<|VISION_START|>",
+    "<|IMAGE_PAD|>",
+    "<|VISION_END|>",
+    "<|VISION_PAD|>",
+    "<|VIDEO_PAD|>"
+  ],
+  "bos_token": {
+    "content": "<BOS_TOKEN>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|END_OF_TURN_TOKEN|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<PAD>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<UNK>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fcc5292908e0c8ad1400c67fe9413825c656486fe4190dce7d8d62c38bfbbf6
+size 19550662

tokenizer_config.json

@@ -0,0 +1,340 @@
+{
+  "add_bos_token": false,
+  "add_eos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<PAD>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<MASK_TOKEN>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<BOS_TOKEN>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<EOS_TOKEN>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<UNK>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255000": {
+      "content": "<|START_OF_TURN_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255001": {
+      "content": "<|END_OF_TURN_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255002": {
+      "content": "<|USER_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255003": {
+      "content": "<|CHATBOT_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255004": {
+      "content": "<|SYSTEM_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255005": {
+      "content": "<|NEW_FILE|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255006": {
+      "content": "<|BEGINNING_OF_PREFIX_FIM_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255007": {
+      "content": "<|BEGINNING_OF_MIDDLE_FIM_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255008": {
+      "content": "<|BEGINNING_OF_SUFFIX_FIM_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255009": {
+      "content": "<|END_OF_MIDDLE_FIM_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255010": {
+      "content": "<|START_THINKING|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255011": {
+      "content": "<|END_THINKING|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255012": {
+      "content": "<|START_RESPONSE|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255013": {
+      "content": "<|END_RESPONSE|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255014": {
+      "content": "<|START_ACTION|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255015": {
+      "content": "<|END_ACTION|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255016": {
+      "content": "<|START_TOOL_RESULT|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255017": {
+      "content": "<|END_TOOL_RESULT|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255018": {
+      "content": "<|USER_0_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255019": {
+      "content": "<|USER_1_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255020": {
+      "content": "<|USER_2_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255021": {
+      "content": "<|USER_3_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255022": {
+      "content": "<|USER_4_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255023": {
+      "content": "<|USER_5_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255024": {
+      "content": "<|USER_6_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255025": {
+      "content": "<|USER_7_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255026": {
+      "content": "<|USER_8_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255027": {
+      "content": "<|USER_9_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255028": {
+      "content": "<|VISION_START|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255029": {
+      "content": "<|VISION_END|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255030": {
+      "content": "<|VISION_PAD|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255031": {
+      "content": "<|IMAGE_PAD|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255032": {
+      "content": "<|VIDEO_PAD|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|VISION_START|>",
+    "<|IMAGE_PAD|>",
+    "<|VISION_END|>",
+    "<|VISION_PAD|>",
+    "<|VIDEO_PAD|>"
+  ],
+  "bos_token": "<BOS_TOKEN>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|END_OF_TURN_TOKEN|>",
+  "extra_special_tokens": {},
+  "image_token": "<|IMAGE_PAD|>",
+  "legacy": true,
+  "max_pixels": 3868706,
+  "merges_file": null,
+  "min_pixels": 16384,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<PAD>",
+  "padding_side": "right",
+  "processor_class": "CohereCompassProcessor",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "CohereTokenizer",
+  "unk_token": "<UNK>",
+  "use_default_system_prompt": false,
+  "video_token": "<|VIDEO_PAD|>",
+  "vision_end_token": "<|VISION_END|>",
+  "vision_start_token": "<|VISION_START|>",
+  "vocab_file": null
+}

video_preprocessor_config.json

@@ -0,0 +1,21 @@
+{
+  "size": {
+    "longest_edge": 25165824,
+    "shortest_edge": 4096
+  },
+  "patch_size": 16,
+  "temporal_patch_size": 2,
+  "merge_size": 2,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "processor_class": "CohereCompassProcessor",
+  "video_processor_type": "CohereCompassVideoProcessor"
+}