| --- |
| license: apache-2.0 |
| license_link: https://huggingface.co/Qwen/Qwen3.5-4B-Base/blob/main/LICENSE |
| base_model: Qwen/Qwen3.5-4B-Base |
| base_model_relation: finetune |
| library_name: transformers |
| pipeline_tag: video-text-to-text |
| language: |
| - en |
| - zh |
| tags: |
| - video-understanding |
| - video-llm |
| - multimodal |
| - codec |
| - token-efficiency |
| - incremental-encoding |
| - vision-language |
| - video-question-answering |
| - pytorch |
| - qwen |
| extra_gated_prompt: >- |
| The NativeCodec repositories use manual approval. Weights, configs, tokenizers and |
| dataset archives are available after your access request is approved. Please describe |
| your intended use; the model card and dataset card remain readable without approval. |
| extra_gated_fields: |
| Full name: text |
| Affiliation: text |
| Intended use: text |
| --- |
| |
| # NativeCodec-4B-FixedRatio |
|
|
| **4B video MLLM aligned with NativeCodec for a constant P-frame suffix ratio.** |
|
|
| This is the fixed-ratio variant of NativeCodec-4B. The language model is fine-tuned to consume visual tokens from the NativeCodec incremental encoder, and each P frame keeps a fixed suffix fraction of its causal representation. At inference the recommended setting is `fixed_p_ratio = 1/16`; the learned length predictor is not used, so this variant has no dependency on the Stage 2 prediction heads. Use it together with the incremental encoder and code in [`Tennineee/NativeCodec`](https://huggingface.co/Tennineee/NativeCodec). |
|
|
| | Companion asset | Repository | |
| |:---|:---| |
| | Code, incremental encoder, prediction heads | [`Tennineee/NativeCodec`](https://huggingface.co/Tennineee/NativeCodec) | |
| | Predicted-length aligner for the same codec | [`Tennineee/NativeCodec-4B-PredLen`](https://huggingface.co/Tennineee/NativeCodec-4B-PredLen) | |
| | BallBench dataset | [`Tennineee/NativeCodec-BallBench`](https://huggingface.co/datasets/Tennineee/NativeCodec-BallBench) | |
|
|
| ## Access request |
|
|
| This repository is public but **gated with manual approval**. Anonymous visitors can read this model card and see the file list. Downloading files requires approval. |
|
|
| To request access, open the repository page, sign in, click **Agree and access repository** / **Request access**, and submit the form. Access is granted to individual users, not to organizations. Requests are reviewed manually by the maintainers without an automatic approval or a published turnaround guarantee; the Hub notifies you when a decision is made. A rejected request cannot be resubmitted from the same account on the Hub. |
|
|
| After approval, download the files with an authenticated session (the full model is about 9.34 GB / 8.70 GiB): |
|
|
| ```bash |
| hf auth login |
| hf download Tennineee/NativeCodec-4B-FixedRatio --local-dir NativeCodec-4B-FixedRatio |
| hf download Tennineee/NativeCodec --local-dir NativeCodec |
| ``` |
|
|
| The incremental encoder required at inference (`weights/inc_enc_stage1_step18750.pt`) lives in the code repository. The complete card of [`Tennineee/NativeCodec`](https://huggingface.co/Tennineee/NativeCodec) also covers access requests, mirror downloads and fine-grained token settings. |
|
|
| ## Files |
|
|
| Every file sits at the repository root, so the directory can be passed to `from_pretrained` directly. |
|
|
| | File | Size (bytes) | Content | |
| |:---|:---:|:---| |
| | `model.safetensors` | 9,319,821,272 | Qwen3.5-4B language model, aligned to NativeCodec tokens; the adapter is merged into the base weights (no separate LoRA checkpoint) | |
| | `config.json` | 3,161 | Qwen3.5-4B model config (architecture `Qwen3_5ForConditionalGeneration`) | |
| | `preprocessor_config.json` | 390 | image processor config, inherited from the base model | |
| | `video_preprocessor_config.json` | 385 | video processor config, inherited from the base model | |
| | `tokenizer.json` | 12,807,982 | tokenizer (vocabulary 248,320) | |
| | `tokenizer_config.json` | 16,710 | tokenizer settings, including the chat template reference | |
| | `vocab.json` | 6,722,759 | vocabulary | |
| | `merges.txt` | 3,353,259 | BPE merges | |
| | `chat_template.jinja` | 7,756 | chat template | |
| | `LICENSE-APACHE`, `NOTICE.md`, `README.md` | | license, attribution and this card | |
|
|
| The weights are a complete Qwen3.5-4B-sized model: about 4.66 B parameters in bfloat16 (the file size divided by two bytes), including the frozen visual tower inherited from the base model. NativeCodec does not route video through that visual tower; the codec front-end adds its own visual tokens to the language model input. |
|
|
| ## Loading |
|
|
| The checkpoint follows the Hugging Face Qwen3.5 layout, so the standard multimodal loader works once your environment provides Qwen3.5 support (the upstream config records `transformers_version: 4.57.0.dev0`): |
|
|
| ```python |
| from transformers import AutoModelForImageTextToText, AutoTokenizer |
| |
| tok = AutoTokenizer.from_pretrained("Tennineee/NativeCodec-4B-FixedRatio") |
| model = AutoModelForImageTextToText.from_pretrained( |
| "Tennineee/NativeCodec-4B-FixedRatio", |
| torch_dtype="bfloat16", |
| attn_implementation="flash_attention_2", |
| ) |
| ``` |
|
|
| Loading this directory with vanilla `transformers` gives the aligned multimodal language model itself. Video reasoning through NativeCodec additionally needs the incremental encoder and the codec front-end; this variant does not need a Stage 2 length predictor: |
|
|
| ```python |
| import sys |
| sys.path.insert(0, "NativeCodec/stage3_llm_sft") |
| from modeling_recodec_qwen35 import ReCodecQwen35ForConditionalGeneration |
| |
| model = ReCodecQwen35ForConditionalGeneration( |
| model_name_or_path="NativeCodec-4B-FixedRatio", |
| recodec_root="NativeCodec/stage3_llm_sft/recodec", |
| phase1_ckpt="NativeCodec/weights/inc_enc_stage1_step18750.pt", |
| phase2_ckpt=None, |
| video_pad_token_id=video_pad_id, # from the code repository's Stage 3 config |
| max_gop_frames=64, |
| max_frames=12288, |
| max_video_tokens=64000, |
| fixed_p_ratio=1 / 16, # constant suffix ratio |
| lora_cfg=None, # adapter already merged |
| ) |
| ``` |
|
|
| A complete runnable path, including frame sampling and prompt assembly, is provided by `stage3_llm_sft/demo_recodec_inference.py` in the code repository; set `fixed_p_ratio=1/16` in the codec config there. |
|
|
| ## Use, budget and results |
|
|
| With the constant 1/16 suffix ratio, NativeCodec-4B reaches **26.1% accuracy** on BallBench at 16 fps (13.9% is random, 12.2% for the Qwen3.5-4B base) and **56.4 mIoU** on TimeLens-Bench with a 64k visual-token budget. Every subset of TimeLens-Bench is better than every open-source baseline in the comparison: 54.5 action, 53.8 change and 67.7 query mIoU. MomentSeeker reaches **25.0 mIoU**. The predicted-length variant and the token-budget ablations are documented in the code repository's model card. |
|
|
| The visual-token budget is configured by the codec front-end, not by these weights; `max_video_tokens` in your config decides how many tokens the model sees. |
|
|
| ## License and attribution |
|
|
| The fine-tuned weights are distributed under the **Apache License, Version 2.0**, following the license of [Qwen/Qwen3.5-4B-Base](https://huggingface.co/Qwen/Qwen3.5-4B-Base). The full license text is in [LICENSE-APACHE](LICENSE-APACHE); [NOTICE.md](NOTICE.md) records the upstream attribution and the changes made here. The code in `Tennineee/NativeCodec` is released under the MIT License. |
|
|
| NativeCodec Team |
|
|
