CrazyTalk 8 by Reallusion is an application that brings photographs to life, enabling them to lip sync spoken audio tracks in a largely realistic manner. And it isn’t just about lip movement: CrazyTalk can animate eyes, eyebrows, and entire head movements to make characters look like they’re really speaking.
The default approach is to start with a photograph – we’ve chosen one of Lincoln on Mount Rushmore. When you import the image, the app will automatically attempt to identify the features, placing marker points on the face to indicate the location of the eyes, nose, mouth, eyebrows, nose and face shape. It usually gets it close, but with significant errors.
This, however, is just the starting point. You can zoom in and adjust all the points manually for a close fit. It’s especially critical around the eyes and mouth, as these are the main areas to be animated; a set of Preview buttons allow you to see these areas moving and speaking, so you can check the alignment before you proceed to the next step:
You can stick with the eyes in the original photograph, but then they won’t be animated. For more expressive movement, choose a pair of eyes from the built-in library – which can be animal or alien in addition to human. You can then fine-tune the results, setting the color, brightness and saturation of the iris, pupil, the eyeball, the reflected light, and even the eyelashes. With a little tinkering, it’s possible to achieve a match that’s close to the original:
Teeth are added in the same way, and generally you’ll need to reduce the default brightness so they look like they’re inside the mouth. Next you’ll want to add an audio track. This can be a recorded mp3 file, or you can speak it straight into the app; you can even use the built-in Text To Speech technology to turn a written text into a spoken one, although obviously you’ll lose realism if you do this.
The Solo Feature Selection dialog allows you to animate your head in real time, while the audio is playing: you can move the whole head up and down, and to a limited extent rotate it left and right – and the static photograph will simulate this movement fairly convincingly. You can also move the eyes, make them blink, and add a variety of expressions. A good way to begin is to use one of the built-in automatic movements, and then tweak it as necessary. A timeline shows every keyframe for each facial element, to allow fine tuning – but this can be a clumsy process, as it’s often hard to identify an individual movement in order to correct or remove it:
The result is an entertaining, and largely convincing, image of someone speaking. The effect is more convincing with cartoon images than real people, as people will always look somewhat artificial; but if you create a talking dog, or Thunderbirds puppet, or Mona Lisa, then your audience is more likely to be entertained.
CrazyTalk 8 can also create 3D heads from photographs, and the process is largely similar. You begin with front and side views of your subject (myself, in this case), and fit anchor point marking the features:
Once again, you can fine tune the results, zooming in on the eyes, nose and mouth to place the points in the closest possible position. As you do so, you’ll see a rendered version of the 3D model with your features imposed on it – a bizarre and sometimes unnerving experience. Point placement can be tricky, as moving the outline of a nose to fit a profile will often mean it no longer fits from the front. A lot depends on the quality of the original photographs. But with just a little effort it’s possible to produce good results:
One of the shortcomings of the 3D modeling side of CrazyTalk is that it can’t deal with a subject’s hair. And while there are a few hair props available, you’re unlikely to find one that matches your own. You can, however, choose from a range of character models, which add clothing, eyewear and headwear. The results may not be a good approximation of your original model, but they are often entertaining:
As with the 2D images, you can animate 3D figures in real time. But where the 2D images are distorted as they appear to turn left and right, the 3D models are much more convincing as the heads turn and move. You can see the end result of this movie here:
CrazyTalk is an entertaining, fun program that’s capable of very powerful image manipulation. The Mac version is rather buggy, with frequent crashes, but as long as you save your work frequently that’s not too much of an issue. The developers do, however, need to address some odd interface choices, such as the fact that timelines are shown in seconds and frames (as you’d expect), but if you want to change the length of the composition you can only do so by setting the total number of frames.
CrazyTalk 8 comes in a variety of configurations, from $49 for the basic version. You can also download a trial version from reallusion.com, so you can try it out for yourself.
Pros: Fun to use, a quick and relatively easy way to get almost any painting, photograph or cartoon to talk; new 3D mapping capability increases realism by a huge amount.
Cons: Can’t capture a subject’s hair; lip movement isn’t really convincing. Better for cartoon animation than real people.
Rating: 8 out of 10.Tags