Blame for README.md

---
license: other
license_name: lfm-open-1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B-GGUF/blob/main/LICENSE
library_name: llama.cpp
pipeline_tag: image-text-to-text
tags:
- ternary
- 2-bit
- gguf
- llama-cpp
- on-device
- vision-language
- multimodal
- finnai
base_model:
- LiquidAI/LFM2.5-VL-1.6B-GGUF
- LiquidAI/LFM2.5-1.2B-Base
---

# FinnAI Foundation (Edge Multimodal Suite)

> **Notice: Community Preview & Preliminary Measurements**  
> *FinnAI Foundation is an initial public release. All performance metrics, throughput figures, and task accuracies presented below reflect internal empirical benchmarks conducted using `llama.cpp` on Apple Silicon / ARM64 CPU. They have not yet been independently verified by external benchmark parties.*

**FinnAI Foundation** is an ultra-compact on-device Vision-Language Model (VLM) suite designed for resource-constrained edge systems. It combines an importance-matrix-guided 2.5-bit low-bit representation (`IQ2_S` effective ~2.50 BPW) with a compressed SigLIP2 vision projector (`mmproj` INT4/FP32), achieving a **~45% total footprint reduction** compared to standard INT4 baselines with full retention across core reasoning and factual tasks.

While the consumer client **FinnAI** remains closed-source, the **FinnAI Foundation** weights and compressed multimodal components are published openly for edge AI developers, robotics engineers, and researchers.

---

## Architectural Profile & Metadata

- **Architecture**: Liquid Neural Network hybrid with short recurrent convolutions (`shortconv`), Liquid linear state projections, SwiGLU feed-forward networks, and GQA attention heads.
- **Base Model Lineage**:
  - Base Language Backbone: `LiquidAI/LFM2.5-1.2B-Base` (1.2B parameter text backbone)
  - Base Multimodal Checkpoint: `LiquidAI/LFM2.5-VL-1.6B-GGUF` (1.6B total parameter multimodal vision-language model, including vision encoder)
- **Runtime Target**: `llama.cpp` (b10809+) GGUF v3 format with full CPU and Metal/Vulkan compute compatibility.

---

## Empirical Benchmark Evaluation

The table below details internal side-by-side evaluations conducted under identical conditions: `temp = 0.0`, `seed = 42`, non-warmup CLI execution, on ARM64 hardware.

### 1. Comparative Metrics Summary

| Evaluation Axis | Metric / Task | Baseline `Q4_0` (4.74 BPW) | FinnAI Foundation (`IQ2_S` ~2.50 BPW) | Delta / Footprint Change |
| :--- | :--- | :--- | :--- | :--- |
| **Model Size (LLM)** | Disk Size | 695.75 MB | **385.08 MB** | **-44.6%** |
| **Projector Size (`mmproj`)** | Disk Size | 853.99 MB (FP16) / 556 MB (Q8) | **417.42 MB** (`Q4_0`) | **-51.1%** vs FP16 |
| **Total Multimodal Bundle** | Combined Footprint | 1,549.74 MB | **802.50 MB** | **-48.2% total reduction** |
| **Perplexity (Wikitext)** | PPL (512 ctx, 43 chunks) | **14.80** $\pm$ 0.43 | **21.25** $\pm$ 0.61 | -5.01 PPL vs naive 2-bit (19.1% improvement) |
| **Mathematical Reasoning** | BIG-bench Simple Arithmetic | **100.0%** (10/10) | **100.0%** (10/10) | Parity with uncompressed baseline |
| **Factual Knowledge & QA** | BIG-bench Wikidata / Fact QA | **90.0%** (9/10) | **100.0%** (10/10) | +10.0% (recovered boiling point query) |
| **Multimodal OCR & Vision** | Geometric & Color Grounding | **100.0%** (2/2) | **100.0%** (2/2) | Full visual grounding parity |
| **Generation Throughput** | Apple Silicon Metal Offload | ~90 tokens/sec | **91.8 tokens/sec** | Equal / slightly improved memory bandwidth |
| **Vision Encoding Throughput** | Patch Processing Speed | 58.4 prompt t/s | **145.7 prompt t/s** | **2.5x faster visual ingest** |

### 2. Dataset Contamination Verification

> **Contamination Safeguard**:  
> To guarantee that reported performance is not the artifact of test data memorization:
> - **Calibration Dataset**: The importance matrix (`imatrix_dense.dat`) was calibrated across 195 multi-domain blocks covering formal logical syllogisms, step-by-step arithmetic proofs, code patterns (Python, Dart), physics/systems explanations, and multilingual text.
> - **Evaluation Tasks**: Evaluated against unseen external tasks from Google's BIG-bench repository (Simple Arithmetic and QA Wikidata) and external icon grounding prompts. Zero overlap exists between the calibration corpus and benchmark prompts.

---

## Technical Methodology: Established Best Practices

Rather than claiming novel theoretical breakthroughs, the compression of FinnAI Foundation leverages established quantization engineering practices:

1. **High-Density Importance-Matrix-Guided Quantization (`imatrix`)**:
   Standard naive uniform round-to-nearest quantization at 2 bits often results in severe outlier clipping, collapsing attention representations. Using activation variance computed over a comprehensive 195-block multi-domain calibration corpus (`llama-imatrix`), weights on critical attention pathways (`attn_v`, `attn_output`) are protected, while less sensitive MLP feed-forward projections are compressed into 2.5-bit codebooks (`IQ2_S`).
2. **Precision Anchoring (Established Industry Practice)**:
   In line with standard practice for mixed-precision architectures, sensitive layers—specifically recurrent convolution kernels (`shortconv.conv.weight`), layernorm scales, and embedding bounds—are maintained in full precision (FP32/FP16).
3. **Projector Downsampling**:
   The multimodal vision projector is quantized to `Q4_0` with FP32 normalization layers, halving the VRAM and memory bandwidth footprint without degrading spatial feature alignment.

---

## Artifact Specifications

| File Name | Format | Size | SHA-256 Digest | Description |
| :--- | :--- | :--- | :--- | :--- |
| `FinnAI-Foundation-1.6B-Ternary.gguf` | GGUF v3 (`IQ2_S` / Dense Salience Matrix) | 385.08 MB (403,789,888 B) | `e2e74f2b0baee70ddb963d712a15a7c7cd0ae00f7b460b90295a7068efd8c459` | Language model backbone with 2.50 BPW effective low-bit representations and FP32 recurrent anchors. |
| `mmproj-FinnAI-Foundation-1.6B-Q4_0.gguf` | GGUF v3 (`Q4_0` / FP32 Norms) | 417.42 MB (437,694,880 B) | `09dbffdce0fe89de27ac1cdc93253f6733ec1b303946ae8a5a7b1c871de5d566` | Compressed SigLIP2 multimodal vision projector component. |

---

## Quickstart & Usage

### 1. llama.cpp (CLI)

```bash
# Text generation
llama-cli \
  -m FinnAI-Foundation-1.6B-Ternary.gguf \
  -p "Explain how an operating system manages memory in simple terms:" \
  -n 256 \
  --temp 0.3 \
  -ngl 0

# Multimodal visual inference with image input
llama-cli \
  -m FinnAI-Foundation-1.6B-Ternary.gguf \
  --mmproj mmproj-FinnAI-Foundation-1.6B-Q4_0.gguf \
  --image diagram.png \
  -p "Describe the flow represented in this system architecture diagram." \
  -n 512
```

### 2. llama-server (OpenAI-Compatible Local API)

```bash
llama-server \
  -m FinnAI-Foundation-1.6B-Ternary.gguf \
  --mmproj mmproj-FinnAI-Foundation-1.6B-Q4_0.gguf \
  --port 8080 \
  --ctx-size 4096
```

### 3. Python (`llama-cpp-python`)

```python
from llama_cpp import Llama

llm = Llama(
    model_path="FinnAI-Foundation-1.6B-Ternary.gguf",
    n_ctx=4096,
    n_gpu_layers=0,
)

output = llm(
    "User: What are the main trade-offs of 2-bit quantization on mobile devices?\nAssistant:",
    max_tokens=256,
    temperature=0.3,
)
print(output["choices"][0]["text"])
```

---

## License & Attribution

- FinnAI Foundation weights are distributed under the **LFM Open License v1.0**.
- Built on top of Liquid AI's LFM-2.5-VL-1.6B / LFM2.5-1.2B-Base architecture.
- Developed by **Finn Technologies**.