.png)
April 27, 2025
Bytedance, the company behind TikTok, has introduced a new AI system called DreamActor-M1, which allows for precise control of facial expressions and body movements in generated videos. The system uses a method Bytedance calls "hybrid guidance" – a combination of multiple control signals working together.
DreamActor-M1 is based on three main components. At its core is a facial encoder that can modify expressions regardless of a person's identity or head position. According to Bytedance researchers, this addresses a common limitation of previous systems. Head movements are controlled by a 3D model that uses colored spheres to direct gaze and head orientation. For body movement, the system uses a 3D skeletal system with an adaptive layer that adjusts to different body types to create more natural movements.
The system processes body movements and facial expressions separately before combining them in a diffusion transformer to create more lifelike animations. During the training phase, the model learns from images at various angles. The researchers state that this enables the generation of new viewpoints even from a single portrait, intelligently supplementing missing details such as clothing and pose.
The training of DreamActor-M1 takes place in three stages: First, the model works on basic body and head movements, then it adds precisely controlled facial expressions, and finally, it optimizes everything together for more coordinated results. Bytedance states that the model was trained with 500 hours of video footage, using full-body and upper-body shots in equal measure.
According to the researchers, DreamActor-M1 surpasses similar systems in both visual quality and the precision of motion control, including commercial products like Runway Act-One. Despite the advancements, the system also has limitations. It cannot handle dynamic camera movements, object interactions, or extreme differences in body proportions between source and target. Complex scene transitions also remain a challenge.
Bytedance is developing several AI avatar animation projects in parallel. Earlier this year, the company released OmniHuman-1, which is already available as a lip-sync tool on CapCut's Dreamina platform. This demonstrates how quickly Bytedance can make research results accessible to users. Other ongoing projects include the Goku video AI series and the InfiniteYou portrait generator.
The development of DreamActor-M1 underscores the growing interest in AI-powered video creation tools. The precise control of facial expressions and body movements opens up new possibilities for creating realistic and expressive digital content. Although challenges remain, the technology points to a future where the creation of videos with virtual characters and realistic animations will be significantly simplified.
Sources: - https://the-decoder.com/bytedances-dreamactor-m1-lets-users-control-ai-video-faces-and-bodies-with-uncanny-precision/ - https://x.com/theaitechsuite/status/1916082166309490695 - https://medium.com/data-science-in-your-pocket/bytedance-dreamactor-m1-video-generation-model-for-movies-18954d521a6c - https://the-decoder.com/bytedances-ai-can-animate-both-real-people-and-cartoon-characters-from-a-single-image/ - https://www.aibase.tech/news/bytedance-launches-dreamactor-m1/ - https://omnihuman-1.com/bytedance/dreamactor-m1 - https://de.linkedin.com/company/the-decoder-en - https://www.linkedin.com/pulse/bytedance-has-introduced-dreamactor-m1-video-model-movies-chauhan-fvavf - https://retail-news.de/bytedance-dreamactor-m1-bildanimation-ki/ - https://www.facebook.com/groups/chatgpt4u/