|
Download README.md from dotimothy/sima-d1: direct link, hf CLI and curl.
- Browser
- Download file 4.57 kB
-
https://huggingface.co/dotimothy/sima-d1/resolve/main/README.md
- Command line
-
hf download hf://dotimothy/sima-d1/README.md
-
curl -L -o README.md https://huggingface.co/dotimothy/sima-d1/resolve/main/README.md
4.57 kB
| license: other | |
| license_name: lfm1.0 | |
| license_link: LICENSE | |
| base_model: | |
| - LiquidAI/d1-omni-600M | |
| - LiquidAI/d1-3B | |
| library_name: sima-lmm | |
| pipeline_tag: image-text-to-text | |
| tags: | |
| - sima.ai | |
| - modalix | |
| - mla | |
| - edge | |
| - decision-model | |
| - liquid | |
| - lfm2.5 | |
| # LiquidAI's d1 decision models, compiled for the SiMa.ai MLSoC Modalix | |
| [Open d1](https://huggingface.co/blog/LiquidAI/open-d1) by Liquid AI is two "System-1" | |
| decision models: each answers a typed question (one of several options, a score on a scale, | |
| yes or no) about a text or about a picture in one forward pass, generating nothing. These are | |
| those two checkpoints compiled for the Modalix MLA. They are not Liquid AI's files and Liquid | |
| AI has not endorsed them. | |
| > **Licence: the [LFM Open License v1.0](LICENSE), not Apache-2.0.** It is Liquid AI's, it | |
| > applies to these files as derivative works of the checkpoints, and a copy is in every | |
| > folder. In short: free for research and non-commercial use, and for commercial use by an | |
| > organisation with less than 10 million US dollars of annual revenue; commercial use above | |
| > that needs a licence from Liquid AI. Read the licence itself before relying on this summary. | |
| | folder | checkpoint | precision | one pass of the trunk, by tokens | a picture | same decision as PyTorch | size | | |
| |---|---|---|---|---|---|---| | |
| | `d1` | [d1-omni-600M](https://huggingface.co/LiquidAI/d1-omni-600M) | A_BF16_W_INT8 | 128: 10 ms, 512: 33 ms | 48 ms | 26 of 27 reference decisions (int8 weights) | 1.56 GB | | |
| | `d1-3b` | [d1-3B](https://huggingface.co/LiquidAI/d1-3B) | A_BF16_W_INT8 | 128: 92 ms, 384: 228 ms, 512: 310 ms | 189 ms | 27 of 27 reference decisions (int8 weights) | 12.78 GB | | |
| Measured on a Modalix DevKit, inside the runtime. A pass holds a picture and several | |
| questions about it: with `d1-3b` three questions about a new picture take about 0.5 s in all | |
| (the picture once, then one pass of the 384-token chain), and with `d1`, whose trunk reads | |
| the picture again with each question, about 0.2 s. "Same | |
| decision" is against the fp32 PyTorch checkpoint on 27 reference questions, 14 of them about | |
| pictures. | |
| ## What was changed | |
| The weights are Liquid AI's, **modified**: rounded to int8 (activations are bfloat16, | |
| `A_BF16_W_INT8`) and compiled into graphs for the MLA with SiMa.ai's Model Compiler, by | |
| this repository's author. Nothing was retrained. So the answers are an approximation of the checkpoints': | |
| on the reference questions the largest difference in a probability was 0.15 for `d1` where | |
| the decision was the same, and 0.07 for `d1-3b`. | |
| Left out of the port: a large picture is read as one picture of at most 512 x 512 pixels | |
| (256 positions) where Liquid AI's code cuts it into tiles; d1-omni's audio encoder is not | |
| compiled; a state longer than the longest graph is cut at its end. | |
| ## What is in a folder | |
| | file | what it is | | |
| |---|---| | |
| | `d1_s<N>_l<LL>n<K>_stage1_mla.elf` | K layers of the trunk from layer LL, for rows of up to N tokens: one graph a length in `d1`, a chain of eight in `d1-3b` | | |
| | `d1_vision_n1024_stage1_mla.elf`, `d1_projector_m256_stage1_mla.elf` | the vision tower, for a picture of up to 1024 patches, and its projector | | |
| | `token_embeddings.bf16` | the embedding table, looked up on the CPU (and read against for d1-3B's logits) | | |
| | `position_embedding.f32`, `type_embeddings.f32` | the tower's position table, and d1-omni's question types | | |
| | `tokenizer.json` | the checkpoint's tokenizer, unchanged | | |
| | `d1_config.json` | token ids, temperatures, and which graphs there are, in order | | |
| | `LICENSE` | the LFM Open License v1.0, unchanged | | |
| ## Using them | |
| The files run on a Modalix board with the runtime and web app from | |
| [neat-decision-studio](https://github.com/dotimothy/neat-decision-studio), whose Models page | |
| downloads a model from here onto the board and loads it on the MLA; the list it reads is in | |
| [dotimothy/sima-laya](https://huggingface.co/dotimothy/sima-laya), with the other decision models. By hand: | |
| ```bash | |
| hf download dotimothy/sima-d1 --include "d1/*" --local-dir . | |
| ./laya run d1 --state "We were billed twice for March." \ | |
| --question '{"type": "noul", "instructions": "Is this a billing problem?"}' | |
| ``` | |
| They are not usable with PyTorch or ONNX Runtime; for that, use Liquid AI's checkpoints, | |
| [LiquidAI/d1-omni-600M](https://huggingface.co/LiquidAI/d1-omni-600M) and | |
| [LiquidAI/d1-3B](https://huggingface.co/LiquidAI/d1-3B). | |
| ## Attribution | |
| The models are Liquid AI's (Liquid AI, Inc.), released under the LFM Open License v1.0. The | |
| compilation, the runtime and this page are not Liquid AI's. | |