emlcpfx/gnm-arkit-blendshapes

ARKit blendshapes on GNM

Built by Eric Levy - thevfxtools.com & Claude Code. MIT licensed.

All 52 ARKit blendshapes retargeted onto Google's GNM head model, with a slider per shape. Runs entirely in your browser - there is no server.

Credits

This work is a retarget, not a new set of shapes. The blendshape geometry is not ours.

[ICT-FaceKit](https://github.com/ICT-VGL/ICT-FaceKit) - USC Institute for Creative Technologies, MIT licence. Its 53 ARKit-named expression shapes are the source that every shape here was transferred from.

Ruilong Li, Karl Bladin, Yajie Zhao, Chinmay Chinara, Owen Ingraham, Pengda Xiang, Xinglei Ren, Pratusha Prasad, Bipin Kishore, Jun Xing, Hao Li. Learning Formation of Physically-Based Face Attributes, 2020. arXiv:2004.03458

@misc{li2020learning,
  title={Learning Formation of Physically-Based Face Attributes},
  author={Ruilong Li and Karl Bladin and Yajie Zhao and Chinmay Chinara and
          Owen Ingraham and Pengda Xiang and Xinglei Ren and Pratusha Prasad and
          Bipin Kishore and Jun Xing and Hao Li},
  year={2020}, eprint={2004.03458},
  archivePrefix={arXiv}, primaryClass={cs.CV}
}

[GNM](https://github.com/google/GNM) - Google, Apache-2.0. The head model everything is retargeted onto.

What is ours, and MIT licensed: the correspondence, the transfer, the projection into GNM's expression basis, the authored tongueOut, the solver and the viewer.

Why this runs with no server

GNM's canonical shape is linear in (identity, expression), and ARKit weights map linearly onto the expression basis, so the whole mesh is a weighted sum:

verts = neutral + sumk wk * D_k

Measured, the 44 non-gaze shapes superpose to within 0.0001 mm, so the page just evaluates that directly. The 52 displacement fields ship as int16 over the 10,698 vertices that can move (3.3 MB, quantised at 0.0023 mm).

*The 8 `eyeLook` are the exception.* They carry a lid deformation and* an eyeball rotation on GNM's eye joints. Rotation is not linear in the angle - blending it linearly is 2.26 mm out, which on an eyeball is a visibly wrong gaze

• —so the delta table holds only the lid part and the page applies the rotation with Rodrigues about the eye pivot.

Head yaw/pitch is not exposed here: it drives joint 1 through linear blend skinning across 98.2% of the mesh, and mouse-orbit already covers looking from another angle.

Also in this repo

• —gnm_arkit_v1.npz - the mapping itself: B (52,383), J (52,4,3), explained, resid_mm
• —gnm_arkit.py - a single-file loader, numpy + scipy only, with drive() and emit() for using this outside the browser
• —gnm_arkit52.jpg - all 52 shapes rendered

Licences

This work is MIT - see LICENSE. Bundled third-party notices are in LICENSES/: GNM is Apache-2.0 (Google), ICT-FaceKit is MIT (USC ICT).