File size: 3,568 Bytes
d8cddff 5cb7bb4 d8cddff 5cb7bb4 85f8897 5cb7bb4 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | ---
license: apache-2.0
tags:
- security-research
- proof-of-concept
- modelscan
- keras
library_name: keras
---
# Security Research PoC — ModelScan coverage gap for nested Keras `Lambda`
> **This repository contains a deliberate security-research proof-of-concept submitted
> through huntr (Protect AI / Palo Alto Networks) under the Model File Vulnerability
> program. It is NOT a usable model. Do not deploy it.**
## What this demonstrates
[ModelScan](https://github.com/protectai/modelscan) detects an unsafe top-level Keras
`Lambda` layer in a `.keras` archive, but **misses a structurally-equivalent unsafe
`Lambda` when it is nested inside a wrapper layer** (here `TimeDistributed`). The Keras
loader still reaches and deserializes that nested layer.
Two artifacts are included:
| File | Structure | ModelScan result |
| --- | --- | --- |
| `top-level-lambda.keras` | `Lambda` at top level (`config.layers[*]`) | **1 issue** (detected) |
| `nested-wrapper-lambda.keras` | same `Lambda` under `TimeDistributed.layer` | **0 issues** (missed) |
| `top-level-lambda.h5` | HDF5 control, top-level `Lambda` | **1 issue** (detected) |
| `nested-wrapper-lambda.h5` | HDF5, same `Lambda` under `TimeDistributed.layer` | **0 issues** (missed) |
The `.keras` and `.h5` variants exercise two separate ModelScan scanners
(`scanners/keras/scan.py` and `scanners/h5/scan.py`); each needs its own
recursive-walk fix.
Root cause: ModelScan's Keras scanner (`modelscan/scanners/keras/scan.py`,
`_get_keras_operator_names`) only iterates top-level `config.layers[*]` for
`class_name == "Lambda"` and does not recurse into wrapper `config.layer` (or
`Bidirectional.forward_layer` / `backward_layer`, etc.).
## This is NOT a Keras safe-mode bypass
Keras default `safe_mode=True` correctly **blocks both** models with a
`ValueError` about Lambda deserialization. The issue is purely a **scanner coverage
gap**: a tool meant to flag unsafe Keras artifacts returns a clean result for a model
that is structurally just as unsafe as one it flags. The risk is that a defender who
gates untrusted models on a clean ModelScan result is given false assurance and then
loads the artifact in an unsafe-deserialization context.
## The payload is benign
The nested `Lambda` writes a single marker file
`MODELSCAN_NESTED_LAMBDA_POC_EXECUTED.txt` in the current working directory and returns
its input unchanged. It performs **no** network, shell, file-deletion, or otherwise
harmful action. The marker only proves the nested Lambda is reachable.
## Reproduce (only in an isolated environment you control)
```python
import keras
# default safe mode blocks both (expected):
keras.saving.load_model("nested-wrapper-lambda.keras") # -> ValueError
# explicit unsafe load reaches the nested Lambda; calling the model runs it
# and writes the benign marker file:
m = keras.saving.load_model("nested-wrapper-lambda.keras", safe_mode=False)
import numpy as np
m(np.zeros((1, 2, 1), dtype="float32")) # writes MODELSCAN_NESTED_LAMBDA_POC_EXECUTED.txt
```
Scan both files with ModelScan to observe the differential (1 issue vs 0 issues).
## Suggested fix
ModelScan should recursively walk Keras configuration objects and flag `Lambda`
wherever it appears in nested layer-like fields (wrapper `layer`, bidirectional
`forward_layer`/`backward_layer`, nested preprocessing/pipeline layers, and other
`deserialize_keras_object()` targets), not only top-level `config.layers[*]`.
## Disclosure
Reported responsibly via huntr. Generated with Keras 3.15.0 (numpy backend).
|