Pushing the boundaries of possibility in entertainment – and beyond
Ncam Technologies was founded in 2012 to develop innovative virtual production technology and products for the film, television and broadcast industries and in a short time has made a very strong impact on the sector. Set up to meet industry demands for an affordable and effective real-time augmented reality solution, it answered specific requirements and needs within virtual production.
How does it work?
The company develops augmented reality technology using a unique camera tracking solution. This uses a lightweight sensor bar attached to a camera to track natural features in the environment, allowing the camera to move freely while generating a continuous stream of precise positional and rotational information that can feed all industry standard graphics engines via Ncam’s powerful and flexible Software Development Kit. It is suitable for a wide range of indoor and outdoor applications, using mounted, hand-held and Steadicam cameras.
The multi-award winning patented technology has been used widely across high level sports and entertainment providers including Fantastic Beasts (Warner Bros), UEFA Champions League (BT Sport), NFC Championship Game (Fox Sports), Game of Thrones Season 8 (HBO), Monday Night Football (ESPN), The Super Bowl XLVIII (Fox Sports), Avengers: Age of Ultron (Marvel), Daytona (Fox Sports), and Jupiter Ascending (Warner Bros).
NAB_2016_Demos from Ncam on Vimeo.
“Our aim is to revolutionise augmented reality and real-time visual effects. This is why our technology is so unique. Our patented camera tracking is able to continuously stream data to industry standard graphic engines, resulting in the photorealistic and immersive integration of virtual assets. The versatility of our design means you can use Ncam across multiple applications and interchangeable configurations. Our platform provides unparalleled flexibility in real-time by combining these technological efficiencies. This instantly delivers better results whilst streamlining the production process. An ideal scenario for everyone involved.”
– Nic Hatch, CEO of Ncam
Recent projects: NREAL
The company’s NREAL CRD project, Real-Time Computer Graphics for Professional Augmented Reality, was a collaboration with the UK subsidiary of the manufacturer of world-leading games engine Epic Games, and Europe’s largest VFX company DNeg. It achieved high success in the sector and is subsequently seeing very positive adoption, with reviews commenting on the fact that Ncam’s unrivalled technological capabilities combined with Epic Games’ Unreal Engine has given augmented reality a massive leap forward into photorealism.
Recognising the demand in augmented reality for set extensions, virtual environments, previsualisation and finished visual effects, Nic Hatch explains that harnessing games engine technology was a real game-changer for creative television and movie-making. “That is why we are working with Epic Games’ Unreal Engine and can now demonstrate real-time photorealism from multiple, freely-moving cameras.”
What’s the technology?
The NREAL project was an 18-month applied research collaboration that aimed to develop new ways of combining live action video, CGI and other high quality assets in real-time. The approach was based on the amalgamation of industry-leading technology with a games engine able to handle live video and camera system metadata, and a pipeline that could automatically integrate assets from a film or TV VFX workflow.
The project set out to create the first professional video-based VFX pipeline designed to run in a games engine, the resulting system (including Ncam’s UE4 plug-in) enabled use within live broadcast, episodic TV and movie VFX production with downstream application to AR/VR experiences.
Research & development engineer Luke Goddard explains that the objective was to make a step change in the convergence of digital media technologies by combining games engine and video system technologies in a single real-time pipeline. A key aim was to enable new production paradigms by integrating video from a high definition camera with rich metadata streams and the physics, vehicles, characters, AI and procedural rules of a games engine.
“This was such a great project in terms of its utilisation,” says project manager and engineer Jeff Clifford. “Often with a co-funded project like this, everybody will go away and do their own thing and the final result ends up being a technical showcase, where the technology never really gets used. This one was so different and the technology was already in practice before the project had even come to an end.”
Progressing the concepts
Ncam has also developed Real Light and Real Depth technology to benefit photorealistic augmented reality. Real Light measures and models light in a scene, allowing virtual elements to cast shadows on actual objects and respond to lighting changes based on the surrounding environment in real time.
Relighting from Ncam on Vimeo.
Real Depth is one of the latest additions to the data output of the Ncam camera tracking system and allows presenters a seamless interaction with their virtual surroundings, including the ability to walk around and through virtual graphics.
tencent NCAM(含现场)_final_1080P from Ncam on Vimeo.
“So far, Ncam has been recognised for its camera tracking expertise, which is still the most accurate and fastest way to track any camera, anywhere,” says Nic. “That remains our foundation and what is really exciting is that we are now combining our camera tracking with Ncam’s new Real Light and Real Depth technologies, delivering something no single company can offer, photorealistic augmented reality.”
Feedback and awards
Impact on the sector has been substantial, all involved in the project agree. “All of the key objectives were realised in this project,” Jeff explained. “Epic can now see how to handle film assets, which will improve their game engine offering and increase its appeal, Ncam created an improved enhanced tracking system to showcase to the market and DNeg can use the technology to design for film.”
He also confirmed he is now being approached by other companies, keen to use games engine technology in future projects. “People now see what is possible and this has had a big impact on opening up the market.
Ncam was also honoured with an Emmy Award for Outstanding Achievement in Engineering Development by the Television Academy for its camera tracking system facilitating the real-time integration of live-action and computer-generated elements in broadcasts. The award is presented for engineering developments that make such a significant improvement on existing methods or are so innovative in nature, that they materially affect the transmission, recording or reception of television.
Other awards include The Queen’s Award for Innovation and the Experience Design & Technology Award for the Best Use of Virtual Reality or Augmented Reality. This project, showcased at the 2017 North American International Auto Show in Detroit, gave attendees an augmented reality X-ray-like look at the technology under the hood of three Ford cars – without the use of headsets. The technology used involved stereoscopic cameras, tracking software and a game engine to overlay real-time 3D graphics, with the resulting AR footage displayed in real time on a two-story, high-resolution LED screen behind the cars.
What does the future look like?
Looking to the future, the market for AR media and technologies is clearly going to be highly dynamic say the NREAL partners. They are forecasting revenues for the project to come from licensing an enhanced games engine for film and TV production, companion apps and royalties, new service offerings and efficiency savings.
The UK has an outstanding media industry with over 6,000 companies in film and video production and 2400 in post, with a turnover of £2.9 billion and £1.38 billion respectively (BFI). In the TV sector, the top 100 independent UK production companies turned over £1.77 billion in 2014-15 (Televisual magazine, December 2015).
With the wider market driven by demand for digital entertainment and media, the global entertainment and media market is predicted to grow in line with GDP, at about 5.1% pa, to reach a value of $2.23 trillion in 2019 (PriceWaterhouseCooper Entertainment and Media 2015).
Describing the marketplace of the future in their post-project plan, Ncam and its partners believe that: “Augmented Reality presents a new opportunity for the VFX sector. Digi-Capital, the Stanford consultancy based in California, USA, predicts that the AR market will be worth $120 billion by 2020, of which approximately $15 billion is expected to derive from the demand for media, characters and objects, heavily driven by content related to film and TV.”
And, of course, there are also technology applications outside the world of entertainment. “This technology will work in any industry that needs visualisation of a scene, so it would also be useful for the fashion and construction industries,” says Jeff.
With proven success in the company’s original target markets in entertainment and leisure, and potential for expansion cross sector, the future for Ncam and its innovative technologies certainly looks bright.
Words: Bernadette Fallon