Adobe teases generative AI video tools

Expand / A demo of Firefly AI generation in Adobe Premiere. Adobe

On Monday, Adobe announced plans to add AI video synthesis tools to Premiere Pro and After Effects, Reuters reports. This follows the announcement last week that AI-based text-based video editing will be coming to Premiere Pro in May.

Adobe's generative video effects will be provided by Adobe Firefly. Firefly, an image synthesis mode we already covered in March, will expand to include video and audio generation tools. Premiere Pro is Adobe's flagship video editing application, a popular tool for assembling Hollywood TV shows and movies.

Currently, performing certain special effects or editing tasks can often be labor intensive. Adobe aims to streamline this process with generative AI assistance. In a demo posted on its blog, Adobe showed "Firefly for Video" generating a background soundtrack, generating sound effects, changing the visual season in a video, creating a visual effect on the word "fade", locating and illuminating a person's face, and generate audio transcriptions via text commands.

In its promotional blog, Adobe mentions features that one might expect from its new AI tools, some of which are listed above (such as generating sound effects and music), but also text-based color enhancements, text and logo animations, automatic storyboarding, and creative "co-pilots" that can supposedly generate tutorials to help users master editing skills.

>

If these types of text-based video editing tools sound familiar, it's because Runway unveiled similar features in September, though Runway has yet to deliver on those promises with the fidelity it advertised in its teaser. Likewise, it's probably wise to take Adobe's marketing video with a grain of salt until the product is released.

(And only time will tell if Adobe Firefly will ever be able to generate a video of Will Smith eating spaghetti that can rival the masterpiece crafted by ModelScope, an open-source video synthesis model.)

In a separate but related announcement last week, Adobe announced that it is integrating text-based video editing into Adobe Premiere Pro, which will allow editors to trim and rearrange video based on automatically detected speech transcripts extracted video clips. Here's how Adobe describes the feature:

Premiere Pro is the only professional editing software to integrate text-based editing, revolutionizing the way creators approach their craft by making video editing as easy as copying and pasting text. Powered by Adobe Sensei, Text-Based-Editing automatically analyzes and transcribes clips so editors, producers and assistants can simply copy and paste phrases in any order and see them appear instantly on the timeline. Transcriptions can be searched in the transcription window to identify exact words and phrases.

Adobe originally previewed a similar text-based video editing capability as part of "Project Blink" in October, though Adobe now says these features come from its "Ad...