Hi there, my name is Eric and I have a gaming YouTube channel called Pixel Rookie. I sometimes like to add little animations in my videos that revolve around lip syncing and after some digging, I found a tool called Rhubarb.
For anyone unfamiliar with Rhubarb Lip Sync, it's a free, open-source tool that analyzes audio recordings and determines which mouth shapes should be displayed at specific points in the dialogue. It's incredibly useful for animating characters without having to manually animate every spoken word.
Unfortunately, incorporating Rhubarb into a DaVinci Resolve workflow can be a bit clunky. Between extracting audio, running an external program, processing the results, and turning those results into usable animation, there are quite a few steps involved.
I wanted to simplify that entire process.
I'm working on a tool called Lip Sync Studio that allows you to select a voiceover clip directly on your timeline, run a script from Resolve's Workspace menu, and automatically generate a fully editable lip-sync animation. FULL DISCLOSER - this is made with the help of AI. I am not a software developer... I work full time and have 3 kiddos and do not have the time to learn software development to make a tool like this on my own. With the advances of AI, I am able to have something created that will help my own creative pipeline significantly and, since I had such a hard time finding a streamlined solution for lip syncing in Resolve, I wanted to share this with the community for free.
Some of its features include:
- Automatic lip-sync generation directly within DaVinci Resolve.
- Three synchronized expression tracks (Neutral, Happy, and Sad), allowing you to change a character's emotional expression without recreating the animation.
- Enhanced phoneme recognition with 11 individual mouth shapes and animation smoothing.
- Support for Resolve Studio's built-in transcription, manually provided dialogue scripts, or audio-only recognition through Rhubarb.
- Support for custom mouth sprites.
- A separate audio-driven animation tool for characters whose mouths are obscured by facial hair.
Although I originally developed this for my own gaming animations, the idea is that it could be used for other sprite-based 2D character animations as well.
I've successfully tested the tool in DaVinci Resolve Studio on Windows, but I'm particularly interested in finding people using the free version of DaVinci Resolve to help verify compatibility.
My goal is to eventually release this tool for free to the community.
If you're interested in 2D animation within Resolve and would be willing to help test it and provide feedback, please let me know!