Modelpublic

AXERA-TECH/LivePortrait

sourceHugging Facemitupdated 1y agoView on Hugging Face

6likes21downloads

Model Card

LivePortrait

This version of LivePortrait has been converted to run on the Axera NPU using w8a16 quantization.

This model has been optimized with the following:

Compatible with Pulsar2 version: 3.4

Convert tools links:

For those who are interested in model conversion:

Support Platform

How to use

Download all files from this repository to the device.

(py310) axera@dell:~/samples/LivePortrait$ tree -L 2
.
├── assets
│   └── examples
├── config.json
├── python
│   ├── axmodels
│   ├── cropper.py
│   ├── infer_onnx.py
│   ├── infer.py
│   ├── pretrained_weights
│   ├── requirements.txt
│   └── utils
└── README.md

7 directories, 6 files

python env requirement

pyaxengine

https://github.com/AXERA-TECH/pyaxengine

wget https://github.com/AXERA-TECH/pyaxengine/releases/download/0.1.3.rc1/axengine-0.1.3-py3-none-any.whl
pip install axengine-0.1.3-py3-none-any.whl

others

pip install -r python/requirements.txt

Inference with AX650 or AX8850 Host, such as AX650 DEMO BOARD, M4N-DOCK(爱芯派Pro)

root@ax650 ~/yongqiang/LivePortrait.axera # python3 ./python/infer.py --source ./assets/examples/source/s0.jpg --driving ./assets/examples/driving/d8.jpg --models ./python/axmodels/ --output-dir ./axmodel_infer
[INFO] Available providers:  ['AxEngineExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 0f7260e8
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 144960ad
FaceAnalysisDIY warmup time: 0.598s
LandmarkRunner warmup time: 0.769s
2025-05-30 09:56:12.247 | INFO     | __main__:main:727 - Start making driving motion template...
2025-05-30 09:56:14.770 | INFO     | __main__:main:747 - Prepared pasteback mask done.
2025-05-30 09:56:17.219 | INFO     | __main__:main:787 - The output of image-driven portrait animation is an image.
2025-05-30 09:56:30.701 | DEBUG    | __main__:warp_decode:647 - warp time: 13.475s
2025-05-30 09:56:31.118 | INFO     | __main__:main:881 - Animated image: ./axmodel_infer/s0--d8.jpg
2025-05-30 09:56:31.118 | INFO     | __main__:main:882 - Animated image with concat: ./axmodel_infer/s0--d8_concat.jpg
2025-05-30 09:56:31.167 | DEBUG    | __main__:<module>:894 - LivePortrait axmodel infer time: 32.455s

Inference with M.2 Accelerator card

What is M.2 Accelerator card?, Show this DEMO based on x86.

Image

(py310) axera@dell:~/samples/LivePortrait$ python ./python/infer.py --source ./assets/examples/source/s0.jpg --driving ./assets/examples/driving/d8.jpg --models ./python/axmodels/ --output-dir ./axmodel_infer
[INFO] Available providers:  ['AXCLRTExecutionProvider']
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 0f7260e8
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 144960ad
FaceAnalysisDIY warmup time: 0.024s
[20:02:20] LandmarkRunner warmup time: 0.031s    human_landmark_runner.py:95
2025-05-29 20:02:20.727 | INFO     | __main__:main:727 - Start making driving motion template...
2025-05-29 20:02:20.972 | INFO     | __main__:main:747 - Prepared pasteback mask done.
2025-05-29 20:02:21.449 | INFO     | __main__:main:787 - The output of image-driven portrait animation is an image.
2025-05-29 20:02:25.475 | DEBUG    | __main__:warp_decode:647 - warp time: 4.017s
2025-05-29 20:02:25.892 | INFO     | __main__:main:881 - Animated image: ./axmodel_infer/s0--d8.jpg
2025-05-29 20:02:25.892 | INFO     | __main__:main:882 - Animated image with concat: ./axmodel_infer/s0--d8_concat.jpg
2025-05-29 20:02:25.904 | DEBUG    | __main__:<module>:894 - LivePortrait axmodel infer time: 8.165s
(py310) axera@dell:~/samples/LivePortrait$

Here, --models specifies the storage path for the *.axmodel model.

The output of axmodel-infer is as follows:

[image] [image]

Video

python3 ./python/infer.py --source ./assets/examples/source/s0.jpg --driving ./assets/examples/driving/d0.mp4 --models ./python/axmodels/ --output-dir ./axmodel_infer

The output of axmodel-infer is as follows:

[image] [image]