EnigmaConsultant/huntr-poc-tflite-weightspace-backdoor

TFLite (.tflite) weight-space BadNets backdoor — loads via standard tf.lite.Interpreter, invisible to modelscan

Summary

A .tflite FlatBuffer image classifier that:

• —Loads through the standard tf.lite.Interpreter(model_path=...) API — no custom ops, no pickle, no code execution, no external delegate.
• —Behaves like a normal, high-accuracy MNIST digit classifier on clean inputs.
• —Silently forces any input carrying a fixed 3×3 white pixel-patch trigger (bottom-right corner) to an attacker-chosen class (digit 0).
• —Is reported clean by modelscan==0.8.8 (the file is skipped entirely — the scanner has no analyzer for the TFLite FlatBuffer format).

The backdoor lives entirely in the learned convolution/dense weights (classic BadNets data-poisoning). There is nothing structural, no code, and no custom operator for a static scanner to flag.

• —huntr format: tflite
• —Class: Model-weights backdoor / output manipulation (embedded trojan)
• —Loader: tf.lite.Interpreter — the documented, standard TFLite runtime entry point

Verified numbers (all REAL — reproduced end-to-end)

Training side (build_backdoor.py, tf-keras, MNIST, 3 epochs):

[keras]  clean_acc = 0.9854   attack_success = 0.9970
[tflite] wrote model.tflite (230404 bytes)

Victim side — loaded ONLY with tf.lite.Interpreter and run for inference (victim_load.py):

benign_accuracy_clean   = 0.9790     # normal accuracy on 2000 clean test images
target_class            = 0
clean_nontarget_to_tgt  = 0.0005     # baseline: clean non-'0' inputs landing on '0'
triggered_flip_rate     = 0.9978     # attack success: non-'0' inputs + trigger -> '0'

Benign behaviour is normal (97.9% top-1). The trigger flips 99.78% of non-target inputs to the attacker class, versus a 0.05% baseline — a clean, unambiguous backdoor.

model.tflite sha256: 82384f98d38b6b64cdc9937353d74911d89ea8b1bc7f7e7e90d716e5c849717c

Exact modelscan 0.8.8 result

--- Summary ---
 No issues found! 🎉

--- Skipped ---
Total skipped: 1
The following file .../model.tflite was skipped during a ModelScan scan:
Model Scan did not scan file

modelscan 0.8.8 has no analyzer for the `.tflite` FlatBuffer format, so it skips the file and returns a clean verdict. A user relying on modelscan as a supply-chain gate receives no warning while shipping a fully backdoored model. Full output in scan_results.txt.

Impact

TFLite is the dominant on-device / edge inference format (Android ML Kit, Coral, embedded, mobile apps). Models are routinely pulled from Hugging Face, model zoos, and third-party vendors and dropped straight into an app. Because:

1. 1.the backdoor is purely weight-encoded (no code, no custom op), and
2. 2.modelscan — a common pre-deployment scanner — does not inspect .tflite at all,

an attacker can publish a backdoored classifier that passes scanning, benchmarks normally, and yet is fully attacker-controlled whenever the physical/logical trigger is present (e.g. a sticker patch on a camera frame, a crafted request). This yields authentication bypass, content-filter evasion, misclassification of attacker assets, etc., depending on the deployment.

Distinctness vs. already-filed backdoors

Weight-space / output-manipulation backdoors have been filed for: onnx, tf-savedmodel, safetensors, gguf, keras_native, pytorch-statedict, torchscript, circle, nemo, coreml, executorch, pmml, mleap, dl4j, flax (openvino in progress). `tflite` is not in that set. This finding is the TFLite FlatBuffer runtime (tf.lite.Interpreter), a distinct format, loader, and serialized container (FlatBuffer, not protobuf/pickle/HDF5/msgpack), with its own modelscan skip-path. It is not reducible to any filed format.

Specifically vs. the already-filed Circle finding

Circle (.circle, the Samsung ONE runtime) and TFLite share the same underlying FlatBuffer schema — Circle is a superset fork of the TFLite schema. This finding is nonetheless distinct on every dimension huntr scores:

• —Different huntr format entry. huntr lists tflite ("TFLite (.tflite) – Google") and circle as separate selectable Model File Format targets; a report is filed against one format value.
• —Different loader / runtime. This PoC loads and executes through Google's tf.lite.Interpreter (TensorFlow 2.21.0), the documented TFLite runtime. The Circle finding loads through the Samsung ONE circle runtime. Neither model is interchangeable at the loader level.
• —Different file / magic / extension. The artifact here is a real .tflite emitted by tf.lite.TFLiteConverter.from_keras_model, with the TFLite file identifier, consumed by the TFLite interpreter — not a .circle file.
• —Independent modelscan skip-path. modelscan 0.8.8 skips .tflite on its own (no analyzer registered for the extension/format); this is verified end-to-end below, independent of the Circle result.

Shared schema lineage does not collapse the two: the vulnerable object, the loader that materializes the backdoor, the format string filed, and the scanner skip decision are all TFLite-specific.

Fix / recommendation

• —modelscan (and comparable scanners) should either (a) explicitly warn that .tflite is unscanned rather than folding it into a silent "No issues found", or (b) add TFLite FlatBuffer analysis (op-set / subgraph inspection cannot detect weight backdoors, so weight backdoors additionally need provenance, signing, and trigger/robustness testing).
• —Consumers: treat weight-only scanners as insufficient for backdoor detection; require signed provenance and behavioural / trigger-scan validation for third-party .tflite models.

Reproduce

python3.12 -m venv venv
./venv/bin/pip install tensorflow-cpu tf-keras scipy numpy "modelscan==0.8.8"

# 1. train the BadNets backdoor and emit the standard FlatBuffer
TF_USE_LEGACY_KERAS=1 ./venv/bin/python build_backdoor.py

# 2. load ONLY via tf.lite.Interpreter and measure benign vs triggered
./venv/bin/python victim_load.py

# 3. confirm modelscan says clean / skips the file
./venv/bin/modelscan -p model.tflite --show-skipped

Files

• —build_backdoor.py — trains the poisoned model, converts with tf.lite.TFLiteConverter, writes model.tflite + eval arrays.
• —victim_load.py — loads with tf.lite.Interpreter only, reports benign accuracy and trigger flip-rate.
• —model.tflite — the backdoored artifact (standard FlatBuffer).
• —clean_inputs.npy, clean_labels.npy — 2000 clean MNIST eval images + labels.
• —triggered_inputs.npy — same 2000 images with the 3×3 corner trigger applied.
• —scan_results.txt — exact modelscan 0.8.8 output + tooling versions.

(venv/ was deleted after verification to save disk; recreate with the commands above.)