litert-community/lightweight-openpose

lightweight-OpenPose — LiteRT (TFLite) GPU, FP16

On-device LiteRT (.tflite) conversion of [lightweight-OpenPose](https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch) for human pose estimation. The model is a MobileNet-based heatmap network; it outputs keypoint heatmaps only and the keypoint decode (argmax) is done in app code.

[image]

The model runs fully on the LiteRT `CompiledModel` GPU accelerator (ML Drift): every op is GPU-native, no CPU fallback. Converted with `litert-torch` with no patches.

Why heatmaps-only: MoveNet's official .tflite bakes the keypoint decode into the graph (GATHER_ND), which the GPU delegate can't run — so it only partially offloads to the GPU. Keeping the graph pure-conv and decoding in app code keeps it 100% on the GPU.

Files

I/O

• —Input: [1, 256, 256, 3] float32, NHWC, RGB, normalized (px - 128) / 256.
• —Output: [1, 32, 32, 19] float32, NHWC, keypoint heatmaps (18 body keypoints + background). Argmax each of the 18 keypoint channels over the 32 x 32 grid to get the normalized keypoint locations; connect them into a skeleton.

Keypoint order (18): nose, neck, r-shoulder, r-elbow, r-wrist, l-shoulder, l-elbow, l-wrist, r-hip, r-knee, r-ankle, l-hip, l-knee, l-ankle, r-eye, l-eye, r-ear, l-ear.

Ops

CONV_2D x41, DEPTHWISE_CONV_2D x14, TRANSPOSE x14, EXP x6, SUB x6,
GREATER_EQUAL x6, SELECT x6, ADD x6, PAD x3, CONCATENATION x1

(The ELU activations lower to EXP/SUB/GREATER_EQUAL/SELECT, all GPU-supported.) No GATHER_ND, no Flex/Custom.

On-device (Pixel 8a, verified)

The fp16 model compiles to 158 / 158 nodes on the LiteRT GPU delegate (LITERT_CL) — full GPU residency, no CPU fallback.

Minimal usage

Android (Kotlin, CompiledModel GPU)

val model = CompiledModel.create(context.assets, "pose_256_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val inputs = model.createInputBuffers()
val outputs = model.createOutputBuffers()
inputs[0].writeFloat(nhwc)             // [1,256,256,3] RGB, (px - 128) / 256
model.run(inputs, outputs)
val heatmaps = outputs[0].readFloat()  // [1,32,32,19] -> argmax per keypoint channel

Python (desktop verification)

import numpy as np
from PIL import Image
from ai_edge_litert.interpreter import Interpreter

img = Image.open("person.jpg").convert("RGB").resize((256, 256))
x = ((np.asarray(img, np.float32) - 128.0) / 256.0)[None]        # [1,256,256,3] NHWC

it = Interpreter(model_path="pose_256_fp16.tflite"); it.allocate_tensors()
it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()
hm = it.get_tensor(it.get_output_details()[0]["index"])[0]       # [32,32,19]

NAMES = ["nose","neck","r_sho","r_elb","r_wri","l_sho","l_elb","l_wri",
         "r_hip","r_knee","r_ank","l_hip","l_knee","l_ank","r_eye","l_eye","r_ear","l_ear"]
for k, name in enumerate(NAMES):                                  # channel 18 = background
    gy, gx = divmod(hm[:, :, k].argmax(), 32)
    print(f"{name}: ({gx/32:.2f}, {gy/32:.2f}) conf {hm[gy, gx, k]:.2f}")

A complete Android sample (camera + gallery, skeleton overlay) is available in google-ai-edge/litert-samples.

Training data & PII

This is a weights-exact format conversion of the public Lightweight OpenPose model; no new training was performed. It was trained for 2D human-pose estimation on the COCO 2017 keypoints dataset (web photos of people with keypoint annotations). These images contain people; the model outputs anonymous keypoint coordinates only and performs no identification. No PII was deliberately collected and this conversion adds none. Apply your own content/PII handling as appropriate. See the original lightweight-human-pose-estimation repo for dataset details.

Performance

Measured on a Pixel 8a (Tensor G3, Android 16) with the standard TFLite `benchmark_model` tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.

Any on-device figure recorded when this model shipped came from a different runtime. It was taken through LiteRT's own CompiledModel accelerator (logcat reports it as LITERT_CL), which is the path the Kotlin sample app and the LiteRT API use, and it appears elsewhere on this card. The rows above are the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. The two are not comparable, so read the rows above as a reproducible floor rather than as this model's speed on LiteRT.

XNNPACK declines these fp16 graphs — it reports failed to delegate DEPTHWISE_CONV_2D and then fails to allocate tensors — so there is no usable CPU number. Disabling XNNPACK falls back to reference kernels, which measured about 20× slower than the GPU on models of this size and would not represent CPU inference anyone would ship.

Snapdragon NPU (Hexagon)

• —pose_256.tflite — the NPU is 2.70x faster than the GPU (1.34 ms against 3.62 ms) and loads 4.78x faster (106 ms against 508 ms).
• —pose_256_fp16.tflite — the NPU is 2.69x faster than the GPU (1.36 ms against 3.66 ms) and loads 5.02x faster (105 ms against 525 ms).

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.79–0.81, where 1.0 is the throttling threshold.

The NPU rows ran the published file unchanged. LiteRT compiled it for the Hexagon on the device at first load. Those first compiles took 723 ms to 884 ms here. The load column above is the cached load every later run pays. Recipe and the runtime libraries it needs: NPU guide.

GPU wiring: GPU guide.

Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with the LiteRT `benchmark_model` tool from litert-cli-nightly 0.2.0.dev20260805: CPU inference (XNNPACK, 4 threads), 3 invocations per file of 10 warm-up plus 50 timed runs (the tool caps a phase at 150 s, so very slow graphs run fewer — the Runs column is the actual timed total). The latency is the median across invocations; the spread is the min–max over all timed runs. No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0).

License & attribution

• —License: Apache-2.0. Weights/model from `Daniil-Osokin/lightweight-human-pose-estimation.pytorch`. Based on "Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose" (Osokin, 2018). Format conversion only; all credit to the original authors.