Rigging.js
Rigging.js is an open-source react.js application, that takes the keypoints generated by the face mesh tensorflow.js model, then map the movement that the person is doing in front of the camera into a 3d model. Any model downloaded from https://www.mixamo.com/ can be used.
Build And Run
git clone https://github.com/haruiz/RiggingJs.git
cd RiggingJs
npm install
npm start
Try with different models
As mentioned above, any model downloaded from https://www.mixamo.com/ can be used. These models are freely available.
Roadmap
- Automatic Rigging and Animation of 3D Characters: using the current state of the art deep learning models.
- 3D expression animation using facemesh model
- Record animation
You can see the detailed roadmap here.
How to contribute:
- Feel free to send a pull request
Third party libraries:
Inspiration taken from:
- Pose Animator 2: Pose Animator takes a 2D vector illustration and animates its containing curves in real-time based on the recognition result from PoseNet and FaceMesh. It borrows the idea of skeleton-based animation from computer graphics and applies it to vector characters.
- PoseNet Model:
This package contains a standalone model called PoseNet, as well as some demos, for running real-time pose estimation in the browser using TensorFlow.js. - BodyPix: This package contains a standalone model called BodyPix, as well as some demos, for running real-time person and body part segmentation in the browser using TensorFlow.js.
- MediaPipe Facemesh: MediaPipe Facemesh is a lightweight machine learning pipeline predicting 486 3D facial landmarks to infer the approximate surface geometry of a human face (paper).