DreamActor-M1

DreamActor-M1 is a human animation framework designed for professionals and researchers seeking to generate realistic and expressive human videos. It utilizes a diffusion transformer (DiT) based approach with hybrid guidance to achieve fine-grained holistic controllability, multi-scale adaptability, and long-term temporal coherence.
The framework works by first extracting body skeletons and head spheres from driving frames, then encoding them into pose latent space. It combines the resultant pose latent with noised video latent and facial expression encoded by the face motion encoder. This process enables the generation of highly expressive and realistic human videos that are temporally consistent and identity-preserved.
DreamActor-M1 is particularly valuable for video production teams, animators, and researchers who need to create realistic human animations for various applications, including film, advertising, and educational content. Its ability to adapt to different scales, from portraits to full-body animations, and its support for audio-driven facial animation make it a versatile tool for a wide range of use cases.