The ‘face’ of the future: Cubic Motion

As video games and experiences are becoming more realistic and virtual worlds are looking increasingly like the real thing, the audience desire to see more lifelike characters within this world is growing. One company working to make that happen is Cubic Motion, creating and deploying advanced computer vision to transform the production and consumption of digital media for audiences who are becoming rapidly more sophisticated.

Their innovations include world-leading technologies for precise tracking in marker-less video, stereo, and depth data, methods that are already well-known for powering facial animation services and are now also bringing real-time capabilities to the industry.

Cutting-edge gaming

With its core business providing facial animation to the video game sector, the team at Cubic Motion work on up to 50 titles per year including Triple-A titles such as Call of Duty, Spiderman, and Horizon Zero Dawn. Their animation for the character ‘Senua’ in the industry-lauded Hellblade game won Best Real-Time Graphics & Interactivity Award from SIGGRAPH (Special Interest Group on Computer Graphics and Interactive Techniques), the annual global computer graphics conference held in the US.

Based in the Manchester Science Park, the company was formed in 2009 by a group of PhDs from University of Manchester and still has close links with the institution. From a starting team of three, the company has now grown to around 70 engineers, animators and internationally acclaimed PhD computer vision scientists.

“Computer vision in a nutshell teaches computers how to see like human beings see. The clever thing we do is capture eye movement and lip movement in a way that nobody else does. Having that technology has enabled us to track and reproduce human facial movement in real-time, that’s a real USP of what we’re doing today.”

-Andy Wood, Chairman Cubic Motion

Today they are recognised as one of the pioneers in computer vision, creating industry-leading facial animation in real-time at high quality and speeds. In its simplest terms, they have developed ground-breaking technology to track the features of actors and turn them into lifelike animated characters and have speeded up the process to take minutes, rather than months.

Moving into the product sphere

It’s been an exciting growth curve, and it’s about to ramp up even more. After securing funding in September 2017, the company’s next objective is to push its technology out into the market and create a new product offering. David Barton, Executive Producer at Cubic Motion, explains the thinking behind the new move.

“Many developers are working with technology that is out-dated, so it makes sense to create a high-end product line that we can deploy to key strategic partners and developers. The services side of our business will still continue to grow as many studios in the industry want to collaborate with our highly experienced production team, relieving difficult production challenges.”

He explains that this work is not just about facial animation but is also building on computer vision technology and there are many different areas that utilise this. “We will of course be helping studios produce cutting edge visuals but we’re also looking beyond that. The AR and VR sectors have so many use cases and there are so many possibilities that we can help with because we have the best people in the world working on this technology.”

In-house skills and technology

All of the Cubic Motion software technology is created in house, though they do work with market hardware such as head-mounted camera devices to capture expressions and create life-like animations.

Talking about their working process, animator Gary Evans explains: “We take the information that we extract from the actor’s facial performance and ‘solve’ this to the digital character to create highly realistic motion with incredibly fast turn-around times.”

Next, they plan to ramp up recruitment, are bringing in computer engineering skillsets and are also building a product team to work directly with strategic clients and establish market needs.

What challenges have they experienced?

Growing the business organically from start-up origins has had its challenges, says David, explaining that the company is fundamentally committed to constant reinvestment into pushing technology forward. “We refuse to stand still on that front, and organic growth can be slow paced and frustrating at times. The constant growth of our service business, and our recent financial investment, enabled us to overcome that challenge and implement significant R&D growth greatly increasing the pace of the technology enhancements, and allows us to move into completely new areas of research and development and new markets.”

What impact has their work had on the sector to date?

“The impact has been important,” David acknowledges, while stressing it has been very much a combined effort. “We help game developers tell their stories by bringing their characters to life, it is very much a collaborative effort between Cubic Motion and the developer, all elements of the character creation pipeline need to be at the highest level.”

Most acclaimed in the industry to date has been their work on Senua character in Hellblade – “I’ve lost count of how many awards that character has won,” says David.

Scooping a key award at SIGGRAPH, a collaboration between Ninja Theory, Epic Games, Cubic Motion & 3Lateral, used the platform to demonstrate the process that allowed them to shoot CG scenes as easily and intuitively as live action and move from set to final edit in minutes, a process that normally takes months.

Discussing the behind the scenes process, Epic Games CTO Kim Libreri explains that with this technology it’s possible to take several performances of an actor and layer them into a cinematic sequence so that an actor is effectively acting with other versions of themselves.

“Here we can render the character in real-time and combine with live action,” says David. “We have the technology to show the animated character moving in real time, matching the movements of the actor.”

Real-time character technology and streaming is becoming very popular, he continues, as the technology can broadcast that performance into a game as it happens, which has a major impact on production time and cuts out weeks or even months of post-production.

What happens next?

Next the team at Cubic Motion plans to look at using animation to enable TV productions to produce real-time output and is also interested in how it can help film crews. “A director working on digital capture on-set wants to be able to see the character’s final performance, as well as what the actor is doing on stage,” David explains. “Generally, this is not done as it’s too difficult. But with our technology the director can see the final version of the action and can walk away from stage with a finished scene that is ready to go, without having to go into expensive visual effects post production.”

There is also huge potential for the technology in the world of social VR, where people all over the world can interact in real-time and have their avatars match their facial movements. There are lots of conversations around real-time right now and what you can do with it, says David. And with the company’s growth on an upward trajectory, those conversations look set to develop and expand.