CoolFace
Apppublic

Natsha/mocap-ai

sourceHugging Faceupdated 3y agoView on Hugging Face
1likes
App README

mocap-ai

Functionality to load FBX files, extract animation, process the animation and write it back to the file.

Classifier

  • —Globals: file with hardcoded values like the marker names.
  • —Utilities:
  • —Visualizations
  • —FBX Handler:
  • —Load the .fbx file.
  • —Go through each frame in the animation frame range and check if all skeleton nodes have a keyframe there.
  • —If a keyframe is missing, remove that frame number from the valid frame numbers.
  • —After finding all valid frames, go through all marker translation channels and store the global transform in a pandas DataFrame.
  • —Add the actor numbers as categorical variables.
  • —Save the DataFrame to a .csv file.
  • —Inference file loader
  • —Same as training file loader, but this one should process all frames regardless of keyframe presence.
  • —Data augmentation:
  • —Isolate a marker set.
  • —Translate and rotate (optionally scale) with boundary check.
  • —Model builder:
  • —Instantiate a model with various hyperparameters.
  • —Training loop:
  • —Train given model with callbacks.
  • —Test loop:
  • —Validate model on validation/test data.
  • —Development script:
  • —Create new model, train it and test it.
  • —Deployment script:
  • —Deploys the model in a Docker image on HuggingFace.

References:

  1. 1.PointNet:
  2. 2.Research paper: Qi, Charles R., et al. "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation." CVPR. 2017. arXiv:1612.00593
  3. 3.Official code repository (TensorFlow): https://github.com/charlesq34/pointnet
  4. 4.Official code repository (PyTorch): https://github.com/fxia22/pointnet.pytorch
  5. 5.PointNet++:
  6. 6.Research paper: Qi, Charles R., et al. "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space." NeurIPS. 2017. arXiv:1706.02413
  7. 7.Official code repository (TensorFlow): https://github.com/charlesq34/pointnet2
  8. 8.Official code repository (PyTorch): https://github.com/erikwijmans/Pointnet2_PyTorch
  9. 9.DGCNN:
  10. 10.Research paper: Wang, Yue, et al. "Dynamic Graph CNN for Learning on Point Clouds." ACM Transactions on Graphics (TOG) 38.5 (2019): 1-12. arXiv:1801.07829
  11. 11.Official code repository (TensorFlow): https://github.com/WangYueFt/dgcnn
  12. 12.Official code repository (PyTorch): https://github.com/muhanzhang/DGCNN