Creating lip sync to an animation can be challenging, but there are many ways of doing it. One way is by using the After Effects add-on for lip syncing called Luma Syphon Pro or Character Animator from Adobe Systems which does all of the work and gives better results than trying manually create each frame in your video afterwards like Ross Plaskow did here:
“There’s one thing you’ll need before we start,” he says as our character explores their surroundings on screen; “the music.” He presses play–and suddenly we’re inside this 3D world!
Taking a more traditional approach, Ross starts by mapping the phonemes needed for reference. There are 8 in total and he draws them all out on paper before hand-drawing each one individually with pen or pencil as well as some digital elements that help him achieve this process quicker than if they were drawn digitally first.
The numbers and order of the hand drawn mouths, become important as you will be recalling them from a precomp using the tried-and true approach to time remapping.
In the end, triggering custom mouth shapes with manual keyframes looks like a much more professional animation than tracked and automated movements from character animator.
Ross Plaskow is an expert in character animation, and his simple approach to the subject makes for some refreshing tutorials. He mostly covers traditional methods instead of automated tools like Unity or Unreal Engine 4 because he feels they’re more useful when learning how people typically work with these programs on their own time while working at home without help from others who may be better programmers than you are!