Introduction
With each passing year, AI technologies like Runway's Act-One and similar developments are pushing the boundaries of what is possible for animation. These systems allow the creation of characters based on simple videos without traditional tools like traditional animation, motion capture with a complex rigging system. Such technologies provide sufficient fidelity in reproducing facial expressions and movements, making them attractive to content creators, especially those focused on rapid production for social media. However, despite their potential, their application in professional animation and film raises many questions and concerns.
These innovations face serious resistance from traditional animators who see a threat to the quality and principles behind classic animation. Industry professionals like animators and artists have expressed concerns about AI's ability to maintain the high level of detail and realism required for more complex projects like animated films and TV series. In addition, according to The Animation Guild and other trade union organisations, the use of AI could lead to significant job losses, affecting up to 29% of employees in the animation industry over the next few years.
Artificial intelligence tools such as Runway Act-One and Gen-3 have significantly advanced animation by using deep learning techniques to transform input data (e.g., textual cues, videos) into animated content.
How these artificial intelligence technologies work
Act-One, introduced by Runway, offers creators a tool to quickly generate and customise animations with basic cues, with its capabilities designed to create expressive and high-quality videos. With the ability to directly input text into the video, Act-One allows for quick generation of visuals and helps create animations to meet specific storytelling needs. In the Gen-3 model, this feature is further enhanced with video-to-video transformation capability, where creators can upload existing video clips and turn them into unique, stylised animations using text prompts alone. This feature simplifies the process for artists and directors, making high-level visual effects more accessible.
Animation capabilities, abilities, limitations, and challenges
AI animation models such as Act-One allow you to create animations quickly and with a specific stylistic sequence, which is useful for high-turnover or prototyping projects. They are particularly effective for creative storytelling, allowing you to customise styles, enhance textures and achieve video fidelity comparable to early-generation professional animation, but without extensive manual rigging or modelling. For example, Gen-3 can process short clips of up to 10 seconds, delivering results in an efficient web environment suitable for creative people without heavy computing hardware.
Despite these advantages, modern AI systems such as Act-One face notable challenges when performing complex, high-quality animation tasks:
Limited detail and control. Modern AI tools often lack the fidelity to fully control the character, such as joints and controllers, limiting their suitability for animation projects that require more subtle character movements. To create controlled expressions with different nuances for the face and body, AI-generated animations clearly need to be cleaned up.
Limiting length and resolution. Gen-3, for example, limits the length of clips to 10 seconds and offers 720p resolution, which, while convenient for quick content creation, limits the tool's use for the high-resolution animation required in professional productions. There are also aspect ratio limitations that may not suit all project needs.
Consistency of random styles. Tools like Act-One can create random results, which creates problems for creators who need a specific character in multiple scenes or long-form content. This limitation can affect projects where a consistent visual style is important throughout the animation.
Overall, while AI tools such as Act-One have democratised aspects of video animation, their current form is best suited for short-form, experimental content or as a complement to traditional techniques. For high-fidelity projects, a hybrid approach combining AI with manual editing remains optimal.
Traditional animation techniques - such as skeletal animation, rigging, and controller systems - remain indispensable for creating high-quality content. These methods provide full control over characters and the necessary detail, which is especially important for realistic animation and emotional expression.
Skeletal animation begins with the creation of a system of joints that set the structure and movement of a character. The precise placement of the joints, as well as their hierarchy, allows for natural movement, and clever rigging design avoids unwanted deformations. For example, the use of Inverse Kinematics (IK) and Forward Kinematics (FK) allows the animator to control either the end positions (IK) or the sequence of joint movements (FK), making the animation flexible and easy to control. For extra fluidity and expressiveness, additional controllers can be added to give the animator the ability to fine-tune, say, finger movements or facial expressions. This allows you to create emotionally rich images that interact with the environment and organically fit into the plot of the animation.
The process of creating a rig requires a deep understanding of anatomy, as well as the use of techniques such as morphing for facial expressions and simulations for elements that interact with the environment, such as hair or clothing. This approach, unlike artificial intelligence techniques, offers absolute control over every movement, making it particularly suitable for high-budget projects in film and animation, where artistic intent must be respected down to the smallest detail.
Therefore, despite the rapid advances in AI animation, traditional rigging and controller tuning methods remain essential tools, as they allow the animator to fully realise their artistic intent without compromise, which is often unattainable with AI methods at the current level of technology development.
Current AI tools such as Act-One and similar technologies do not yet fulfil the basic requirements of real-world production. Today's production studios are focused on high quality emotional expression, accurate lip-sync, the ability to work with multiple characters simultaneously and create complex interactions between them, including realistic interactions with the environment.
One of the main limitations of AI tools is the lack of control required for complex emotional scenes. AI systems are usually not equipped with the detailed facial expression control and subtle changes in facial expressions that are often required in film or animation. Most current AI systems are limited to basic emotions and simple movements that are difficult to adapt to the specific artistic intentions of directors and animators.
Lipsync and the precise expression of emotion. Lipsync, or the synchronisation of lip movements with speech, is another task where AI is often inferior to traditional methods. Unlike manual animation and rigging, where an animator can synchronise each word with minimal lip changes, AI typically works with a more generalised synchronisation system, failing to achieve the detail required. Jonathan Cooper, an animator in the video game and film industry, notes: ‘AI is not yet able to achieve the precision we achieve manually, especially in scenes with high emotional intensity where the smallest changes in facial expression are important.’
Working with multiple characters and complex interactions. AI also faces challenges in creating a scene with multiple characters, which requires synchronised interactions and complex movement coordination, especially when it comes to detailed interactions - such as touching, wrestling or dancing. In the traditional approach, the animator can control every movement and plan in advance how the characters will interact with each other and the environment. In the case of AI, this is still difficult to realise as it is not able to intuitively consider all aspects of such interactions.
Feedback from professionals and real-world experience. In response to the presentation of Act-One and similar AI tools, many professionals emphasise the gap between research advances and the practical needs of the industry. One industry commentator, for example, noted that such AI ‘performances’ are often isolated from real-world production environments: ‘nobody demands a static character on a monochrome background. We need dynamic scenes, complex interactions and multiple performers working at the same time’.
Thus, AI still fulfils the role of an additional tool capable of solving basic tasks or speeding up some stages of animation creation, but for high-quality projects it is far from being able to replace advanced traditional approaches such as rigging and full control over the character.
AI technologies like Act-One have already found applications in areas where high detail and complex interactions are less critical, making them useful for social media content creation, experimentation and test development. For example, for short videos on social media and other platforms where speed and simplicity of animation are more important than precise facial expressions and complex movements, AI can generate animations quickly with minimal effort. AI animation in such cases can accelerate the creation of experimental content, video presentations, and small projects that don't require a high level of control or customisation.
Potential areas for improvements and future use cases
In the long term, AI may become more useful in tasks that require speeding up the production process without sacrificing quality. Examples of such areas include:
Automating basic animation: AI can perform automatic rotation or basic movement, leaving animators more time to detail complex scenes.
Scoring animations: for ad campaigns where rapid content release is important, AI can help speed up the creation of clips where animators can refine or adapt the resulting content manually.
Mixed technologies: AI can be integrated with traditional animation techniques, allowing professionals to use AI to speed up basic processes and focus on complex details such as working out facial expressions and interactions. This will require significant research and development (R&D), especially in control and customisability. This could include the ability to fine-tune facial expressions and the ability to control certain aspects of animation in real time.
So while AI has not yet reached a level where it can fully meet the demanding requirements of the production industry, it already offers useful tools for simplified animation and can serve as a supporting technology to speed up some stages of the workflow.
To conclude the discussion, artificial intelligence continues to evolve in the field of animation, offering new opportunities for simplified, experimental and social media projects. However, given current limitations, AI is not yet ready to replace the traditional methods used to create high quality animations. Basic tasks such as precise control of character movements, facial expression and lip-synchronisation, and multi-layered interactions between characters require a higher level of control than AI is currently capable of providing.
In the near future, AI is likely to become part of hybrid workflows, where automating basic movements and scenes can reduce animation preparation time, leaving animators more time to work out the details. AI can also be useful for experimental and marketing projects that require quick results and less attention to accuracy, but there is still an indispensable role for traditional methods in production.
Forecast and a call to the industry
For AI to be successfully incorporated into major production processes, it is important that research and development in this area takes into account the real needs of animation studios and professionals. The industry could benefit greatly if AI solutions are developed with animators' requirements in mind - such as the ability to control every aspect of movement, flexible emotion tuning, synchronisation of multi-person scenes and stable handling of dynamic backgrounds.
AI in animation has huge potential, but more collaboration between developers, studios and professionals is required to fully realise its potential.
Comments