In this study, we developed StyleGAN2 model for macaque monkeys, which are widely studied in systems neuroscience and evolutionary research, and proposed a method for generating their facial expressions. Since, facial expressions are largely consistent across individuals due to the similar musculoskeletal structures and their role in social communication, we address the limitation of lack of available data by introducing a data augmentation approach. We used a motion transfer technique and refined the training procedures, which consisted of three steps: 1) data augmentation by synthesizing new facial expression images using a motion transfer technique to animate still images via computer graphic animations, 2) selection of image samples based on the latent representation of macaque faces from the initially trained StyleGAN2 model to ensure the variation and uniform sampling in training dataset, and 3) refinement of the loss function to ensure the accurate reproduction of subtle movements, such as eye movements. We demonstrated that the model trained using our method can generate a wider variety of facial expressions for multiple macaque individuals compared to a model trained solely on original still images without motion-transfer-based data augmentation. We also showed that our model is effective for style-base image editing and revealed that specific style parameters correspond to particular facial movements, highlighting its potential to identify individual action components as sets of disentangled style parameters.
- Publish README.md
- Upload codes for StyleGAN2
- Upload checkpoints for StyleGAN3
StyleGAN2 checkpoints are uploaded at https://doi.org/10.6084/m9.figshare.31869274
First row/column: Source images. Second row/column: Inverted images.
Images in the first column were synthesized for the motions which is described in our paper.
- Injection of 0,1,2 layers from column images to row images (Mouth movements)
- Injection of 6,7,8 layers from column images to row images (Eye movements)
- Morph (50% blending in all layers) (Intermediate identity)
We provide a simple mixing software which enable to interactively edit the predefined facial properties. Key typing of s decreases, while key typing of f increases the blending ratio.
In the following bash file, you shoud specify image paths (path_img and path_img2), a checkpoint path of the restyle-encoder (checkpoint_path), list of layers to mix is specified by the argument of (mixing_layers).
The prepared checkpoint at data/checkpoints/maqface-stylegan2.pt is trained with the motion transfer-based augmentation and the L2 loss for eye.
bash scripts/demo_mixing.sh
We offer editing direction for facial expression bark, blink, brow raise, chewing, coo, lip smack, scream, threat, tongue protrude, yawn from MF3D. We also offer additional facial motion from our original video for look-up, look-down, look-left, look-right, tongue show and annotated directions for species, age, and sex, and head orientation.
For the quality of face editing, please refer to our paper.
We provide a simple editing software which enable to interactively edit the predefined facial properties. Key typing correspoints to manipulate motions along the editing directions.
In the following bash file, you shoud specify an image path (path_img), a checkpoint path of the restyle-encoder (checkpoint_path), and a pool directory of the editing directions (dir_directions).
The prepared checkpoint at data/checkpoints/maqface-stylegan2.pt is trained with the motion transfer-based augmentation and the L2 loss for eye.
bash scripts/demo_editing.sh
If you use this code for your research, please cite the following work:
@article{igaue2025mfgan,
author={Igaue, Takuya and Correia-Caeiro, Catia and Yoshida, Akito and Miyabe-Nishiwaki, Takako and Hayashi, Ryusuke},
title={Motion transfer-enhanced StyleGAN for generating diverse macaque facial expressions},
journal = {arXiv preprint arXiv:2511.16711},
year = {2025}
}
Our facial motions include ones from MF3D. For the motion transfer, we used TPSMM for motion synthesis. For the image generation, our method is based on StyleGAN2-ADA and ReStyle. For the image editing, our method is based on InterfaceGAN and StyleSpace-pytorch. We used InsightFace for the face detection.