README.md

metadata

license: other
license_name: lfm-open-1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B-GGUF/blob/main/LICENSE
library_name: llama.cpp
pipeline_tag: image-text-to-text
tags:
  - ternary
  - 2-bit
  - gguf
  - llama-cpp
  - on-device
  - vision-language
  - multimodal
  - finnai
base_model:
  - LiquidAI/LFM2.5-VL-1.6B-GGUF
  - LiquidAI/LFM2.5-1.2B-Base

FinnAI Foundation (Edge Multimodal Suite)

Notice: Community Preview & Preliminary Measurements
FinnAI Foundation is an initial public release. All performance metrics, throughput figures, and task accuracies presented below reflect internal empirical benchmarks conducted using llama.cpp on Apple Silicon / ARM64 CPU. They have not yet been independently verified by external benchmark parties.

FinnAI Foundation is an ultra-compact on-device Vision-Language Model (VLM) suite designed for resource-constrained edge systems. It combines an importance-matrix-guided 2.5-bit low-bit representation (IQ2_S effective 2.50 BPW) with a compressed SigLIP2 vision projector (mmproj INT4/FP32), achieving a **45% total footprint reduction** compared to standard INT4 baselines with full retention across core reasoning and factual tasks.

While the consumer client FinnAI remains closed-source, the FinnAI Foundation weights and compressed multimodal components are published openly for edge AI developers, robotics engineers, and researchers.

---

Architectural Profile & Metadata

• Architecture: Liquid Neural Network hybrid with short recurrent convolutions (shortconv), Liquid linear state projections, SwiGLU feed-forward networks, and GQA attention heads.
• Base Model Lineage:
  • Base Language Backbone: LiquidAI/LFM2.5-1.2B-Base (1.2B parameter text backbone)
  • Base Multimodal Checkpoint: LiquidAI/LFM2.5-VL-1.6B-GGUF (1.6B total parameter multimodal vision-language model, including vision encoder)
• Runtime Target: llama.cpp (b10809+) GGUF v3 format with full CPU and Metal/Vulkan compute compatibility.

---

Empirical Benchmark Evaluation

The table below details internal side-by-side evaluations conducted under identical conditions: temp = 0.0, seed = 42, non-warmup CLI execution, on ARM64 hardware.

1. Comparative Metrics Summary

Evaluation Axis	Metric / Task	Baseline Q4_0 (4.74 BPW)	FinnAI Foundation (IQ2_S ~2.50 BPW)	Delta / Footprint Change
Model Size (LLM)	Disk Size	695.75 MB	385.08 MB	-44.6%
Projector Size (mmproj)	Disk Size	853.99 MB (FP16) / 556 MB (Q8)	417.42 MB (Q4_0)	-51.1% vs FP16
Total Multimodal Bundle	Combined Footprint	1,549.74 MB	802.50 MB	-48.2% total reduction
Perplexity (Wikitext)	PPL (512 ctx, 43 chunks)	14.80 $\pm$ 0.43	21.25 $\pm$ 0.61	-5.01 PPL vs naive 2-bit (19.1% improvement)
Mathematical Reasoning	BIG-bench Simple Arithmetic	100.0% (10/10)	100.0% (10/10)	Parity with uncompressed baseline
Factual Knowledge & QA	BIG-bench Wikidata / Fact QA	90.0% (9/10)	100.0% (10/10)	+10.0% (recovered boiling point query)
Multimodal OCR & Vision	Geometric & Color Grounding	100.0% (2/2)	100.0% (2/2)	Full visual grounding parity
Generation Throughput	Apple Silicon Metal Offload	~90 tokens/sec	91.8 tokens/sec	Equal / slightly improved memory bandwidth
Vision Encoding Throughput	Patch Processing Speed	58.4 prompt t/s	145.7 prompt t/s	2.5x faster visual ingest

2. Dataset Contamination Verification

Contamination Safeguard:
To guarantee that reported performance is not the artifact of test data memorization:

• Calibration Dataset: The importance matrix (imatrix_dense.dat) was calibrated across 195 multi-domain blocks covering formal logical syllogisms, step-by-step arithmetic proofs, code patterns (Python, Dart), physics/systems explanations, and multilingual text.
• Evaluation Tasks: Evaluated against unseen external tasks from Google's BIG-bench repository (Simple Arithmetic and QA Wikidata) and external icon grounding prompts. Zero overlap exists between the calibration corpus and benchmark prompts.

---

Technical Methodology: Established Best Practices

Rather than claiming novel theoretical breakthroughs, the compression of FinnAI Foundation leverages established quantization engineering practices:

1. High-Density Importance-Matrix-Guided Quantization (imatrix): Standard naive uniform round-to-nearest quantization at 2 bits often results in severe outlier clipping, collapsing attention representations. Using activation variance computed over a comprehensive 195-block multi-domain calibration corpus (llama-imatrix), weights on critical attention pathways (attn_v, attn_output) are protected, while less sensitive MLP feed-forward projections are compressed into 2.5-bit codebooks (IQ2_S).
2. Precision Anchoring (Established Industry Practice): In line with standard practice for mixed-precision architectures, sensitive layers—specifically recurrent convolution kernels (shortconv.conv.weight), layernorm scales, and embedding bounds—are maintained in full precision (FP32/FP16).
3. Projector Downsampling: The multimodal vision projector is quantized to Q4_0 with FP32 normalization layers, halving the VRAM and memory bandwidth footprint without degrading spatial feature alignment.

---

Artifact Specifications

File Name	Format	Size	SHA-256 Digest	Description
FinnAI-Foundation-1.6B-Ternary.gguf	GGUF v3 (IQ2_S / Dense Salience Matrix)	385.08 MB (403,789,888 B)	e2e74f2b0baee70ddb963d712a15a7c7cd0ae00f7b460b90295a7068efd8c459	Language model backbone with 2.50 BPW effective low-bit representations and FP32 recurrent anchors.
mmproj-FinnAI-Foundation-1.6B-Q4_0.gguf	GGUF v3 (Q4_0 / FP32 Norms)	417.42 MB (437,694,880 B)	09dbffdce0fe89de27ac1cdc93253f6733ec1b303946ae8a5a7b1c871de5d566	Compressed SigLIP2 multimodal vision projector component.

---

Quickstart & Usage

1. llama.cpp (CLI)

# Text generation
llama-cli \
  -m FinnAI-Foundation-1.6B-Ternary.gguf \
  -p "Explain how an operating system manages memory in simple terms:" \
  -n 256 \
  --temp 0.3 \
  -ngl 0

# Multimodal visual inference with image input
llama-cli \
  -m FinnAI-Foundation-1.6B-Ternary.gguf \
  --mmproj mmproj-FinnAI-Foundation-1.6B-Q4_0.gguf \
  --image diagram.png \
  -p "Describe the flow represented in this system architecture diagram." \
  -n 512

2. llama-server (OpenAI-Compatible Local API)

llama-server \
  -m FinnAI-Foundation-1.6B-Ternary.gguf \
  --mmproj mmproj-FinnAI-Foundation-1.6B-Q4_0.gguf \
  --port 8080 \
  --ctx-size 4096

3. Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(
    model_path="FinnAI-Foundation-1.6B-Ternary.gguf",
    n_ctx=4096,
    n_gpu_layers=0,
)

output = llm(
    "User: What are the main trade-offs of 2-bit quantization on mobile devices?\nAssistant:",
    max_tokens=256,
    temperature=0.3,
)
print(output["choices"][0]["text"])

---

License & Attribution

• FinnAI Foundation weights are distributed under the LFM Open License v1.0.
• Built on top of Liquid AI's LFM-2.5-VL-1.6B / LFM2.5-1.2B-Base architecture.
• Developed by Finn Technologies.