Alibaba's New Deepfake Tech Turns Still Images Into Animated Videos
Imagine the implications this AI innovation could bring.
By Syazwan Bahri — 05 Dec 2023, 05:22 PM
A new artificial intelligence (AI) from the minds of Institute for Intelligent Computing, Alibaba Group is capable of turning still images into virtually flawless videos
This AI, named Animate Anyone, is designed to effortlessly turn static images into animated character videos.
It utilises diffusion models, which are currently leading the way in visual generation research. Diffusion models are generative models, meaning that they are used to generate data similar to the data on which they are trained.This approach tackles issues related to preserving both temporal consistency and detailed information during the transformation of images into videos.
Here's an example of how it works:
The researchers pointed out how the AI tech works in their paper. In a paper describing their innovative work, the researchers present a distinctive framework customized for character animation.
The central component in this technology, ReferenceNet, is designed to combine detailed characteristics from reference images while retaining detailed visual features. Spatial attention is incorporated to guarantee the coherence of visual elements throughout the animation process. This means that the AI looks at images and learns what to animate while maintaining visual details.
Previous AI models were able to animate still images, but it came with many caveats
For example, hallucination proved to be a major obstacle, as the AI model had to create realistic details, such as predicting how a sleeve or hair might behave when a person turns. This often resulted in awkward visuals, making the produced video less realistic.
Despite these issues, the potential remains, and Animate Anyone represents a significant improvement, although it is still a work in progress.
