Audio-Text-to-Text
Transformers
Safetensors
lfm3_omni
feature-extraction
liquid
lfm3
edge
decision
classification
extraction
calibration
system-one
multimodal
audio
decision-model
custom_code
Eval Results
Instructions to use LiquidAI/d1-Omni-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use LiquidAI/d1-Omni-2B with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("LiquidAI/d1-Omni-2B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add evaluation results
#3
by SaylorTwift HF Staff - opened
Add evaluation results for LiquidAI/d1-Omni-2B
Summary
This PR adds the benchmark results published on the model card for LiquidAI/d1-Omni-2B to .eval_results/, following the Hugging Face Hub evaluation-results specification.
Benchmarks Added
Benchmarks Skipped (Not Registered on Hub)
These benchmarks are on the model card but have no registered eval.yaml on the Hub:
- DecisionBench (Hanno Labs) v1, Decision Bench (decisionbench.ai) v4, Fast Decisions, JevBench, ImajevBench, WorkflowEvals, Liquid's internal board
- The image benchmarks (AI2D, BLINK, MMBench, MME, ...)
- The audio tests (SLURP, Speech-MASSIVE, FLEURS, ESC-50, Belebele, decision suites read aloud)
They can be added once their authors register an eval.yaml on the Hub.
Mapping notes
- Typed Decisions (63.1): the card's metric and scale are unclear (the Hub task is accuracy on a 0-1 scale), so it's better to score it independently as was done for d1-3B.
- MMMU-Pro (24.1): scored as a decision over the options on at most 1,000 rows, which isn't the standard setup, and the variant isn't stated.
Source
- Model card: https://huggingface.co/LiquidAI/d1-Omni-2B
- Blog: https://www.liquid.ai/blog/open-d1
Files Added
.eval_results/d1-Omni-2B.yaml
Verification
The scores come from the benchmark tables on the model card, as measured by Liquid AI. No verified token is included, because these were not run via HF Jobs with inspect-ai.
To upload:
hf upload LiquidAI/d1-Omni-2B .eval_results .eval_results --create-pr --commit-message "Add evaluation results"
SaylorTwift changed pull request status to open
Thanks!
Aurelien-Lac changed pull request status to merged