360 Video and Virtual Reality

Published: 1 January 2014

Immersive experiences for audiences present new challenges for content creation and delivery

Project from 2014 - present

What we're doing?

360 video and VR encompass a wide range of technology, from monoscopic rotatable video on a web page or mobile device, through headsets such as Google Cardboard with 3D audio, to high-end headset-based VR experiences allowing full movement and interaction. 360-degree pannable interactive stills have been around since the advent of Quicktime VR in 1995, but it is only recently that it has become practical to capture and deliver video, and to view it on an affordable headset of reasonable quality.

VR experiences, by which we mean those based on 3D graphics or “CGI” (computer-generated imagery), have been available in research labs and industrial applications for many years. Recent advances in headsets (displays and motion sensors) plus increased rendering power mean that such experiences are becoming available in consumer-level devices. 360 video can provide highly-realistic imagery, is comparatively easy to produce, but lacks interactivity or the ability for the viewer to move freely.

VR, on the other hand, can provide very rich interactivity and movement but tends to be more expensive to produce and it is technically challenging to support photorealistic imagery - although commercial solutions are becoming available, such as volumetric capture and early approaches to light field acquisition and rendering.

BBC R&D - An Introduction to BBC Reality Labs

360 Video of 'Attenborough and the Giant Dinosaur' on BBC Taster

At BBC R&D our job is to help the BBC understand more about what these technologies might offer our audience and to influence and contribute to developments in technology, from editorial, through to production, distribution and consumption. There is already a lot of activity in the wider industry, particularly focused on gaming: our focus is instead on how this technology can complement the kind of content that the BBC produces, so we are looking at applications that focus on areas such as journalism, narrative and educational content, rather than a pure ‘gaming’ experience.

It is still very early days for this, with no established editorial guidelines, production pipelines or universal distribution methods, and care is needed to identify the end-user experiences that will truly benefit from 360 or VR technology rather than simply creating a brief ‘wow’ moment.This is a very broad area of work, involving a number of projects across R&D teams in areas including editorial, production and user experience. Links to other projects, blog posts and demonstrators are given below.

360-degree video and VR are examples of what we refer to as a ‘New Content Experience’: a form of content that goes beyond what the BBC currently produces, and that can be enabled by an all-IP production and distribution chain that is not tied to traditional audio-visual content formats. Our VR/360 work thus forms a part of our broader programme of work in this area, which is helping us to imagine a BBC of the future, not necessarily the one that exists today.

This is not the first time that we have looked at how immersive and interactive video and audio systems and real-time 3D graphics can be used to enhance our audience’s experience. Previous examples include:

Technology for mixed reality TV production that we developed between 1997-2005, used in programmes like BAMZOOKi to allow real-time compositing of virtual content into a studio recording, and resulting in commercial products for keying, camera tracking and virtual overlays for sport.
Augmented reality tracking system for a BBC/Natural History Museum installation in the Attenborough Studio, Who do you think you really are, launched in 2010
3D audio, with our work on applications of ambisonics starting in 2010 and continuing in various projects in our audio team
‘Surround Video’ – an investigation into the possibilities of a room-sized 180-degree projection system to augment a conventional TV display, developed in 2010-2011
Panoramic video and 3D audio to provide navigable interactive experiences, through projects like FascinatE and Venue Explorer

R&D’s 360/VR work can be broadly divided into four areas covering investigating the latest 360/VR technology, development of some specific production tools to fill gaps, commissioning content across a range of genres to assess the editorial possibilities, and user experience research to get a deeper insight into viewer behaviour. Put another way, this is ‘what new technology is already available, what other technology would we need, what can we make with it, and what do viewers think?’

'Easter Rising: Voice of a Rebel'

Understanding the technology and implications for production and delivery

We have investigated key bits of the production chain and various filming techniques so that we understand the limitations: this makes us an informed customer when dealing with external production companies, and allows us to provide advice to the wider BBC. It should be noted that 360/VR production is still fraught with technical problems, and based on our experience and general technical knowledge we have been called upon to provide advice to external production companies concerning various BBC commissions – no one is truly an expert in this area yet!

One of our first production experiments was setting up a live stream of 360 video and 3D audio for a demonstration at the 2014 Commonwealth Games. Since then we have been directly involved in producing a number of 360-degree videos, often working with independent production companies.We have also been investigating various delivery approaches, including linking from our experimental content portal BBC Taster to YouTube and assessing other web-based players.

More recently we’ve integrated a native BBC 360 player into productions such as Super Furry Animals. We also launched an Edinburgh Festivals 360 explorer pilot on Taster that uses a WebVR 360 player we developed, it allows producers to create interactive, branching narratives in 360 using WebVR and hotspots.

Production tools and technology gaps

We are developing production tools to fill some of the gaps between current TV workflow and what is needed for VR/360.Most of our work so far has been in the area of object-based 3D audio production tools, building on the ADM (Audio Definition Model) standard for representing object-based audio in production for which we have led the development. This work was already underway before the current rise in prominence of VR/360, making us well-placed to apply it in this area.

Our recent developments include two plug-ins for the Unity games engine to import ADM files and for binaural rendering (both used in the The Turning Forest). We also developed a tool to allow the location of sound objects to be visualised in a VR headset during audio post-production, which has been used for various 360 video productions including the Unearthed experience released on Taster in 2015. We have recently configured our production tools to export 3D audio in the ambisonics format supported by YouTube’s 360 video player.

'We Wait'

Trial/demo productions across a range of genres

We have commissioned the production of various pieces of content that explore different aspects of what 360/VR might offer. Some of these have been commissioned as part of BBC Connected Studio’s Future of Content: Immersive season. Examples include:- Tests with 360-degree journalism in conjunction with BBC Newslabs, including pieces like Paris Attacks – Place de la Republique in which BBC journalist Matthew Price used 360-video footage beside one of the main memorials to those who lost their lives to give audiences a better impression of what the atmosphere at the Place de la Republique was like.

The Resistance of Honey 360 video to explore approaches to shooting grammar- Ring as an audio-only interactive binaural experience.- The Turning Forest produced in collaboration with VRTOV as a CGI audio-led experience that also brought interactive elements into a narrative-led piece. This built heavily on our work on 3D audio.- We Wait is a fictional depiction of migrants making the perilous journey from Turkey to Greece on smugglers boats. It is based on accounts gathered by BBC News from migrants and brought to life as a CGI experience in collaboration with Aardman Digital using animation techniques. The aim was to explore how VR has the potential to give audiences a greater understanding of a topic they otherwise wouldn’t be able to experience.

More information and links to other experimental commissions can be found on Connected Studio's Virtual Reality page.We have also provided advice to 360/VR productions led by other parts of the BBC, including Rome’s Invisible City, and BBC Click’s 360 programme including a visit to the Large Hadron Collider at CERN.

User experience research

We have carried out some user experience research using some of this content plus other specially-shot material to understand its impact on viewers, as well as gathering feedback from others via BBC Taster’s rating system. Examples include:- looking at what controls viewers’ attention by recording head movement whilst viewing 360 video- the effect of different angular separations of actors- different approaches to placing overlays and subtitles- how the viewing device affects the experienceOutlook

There is still much to learn about how to produce good 360/VR content in a broadcast-related context, for example how to make best use of existing assets and opportunities for additional location shooting that can be done relatively cheaply as a part of existing TV shooting. We are learning more about what experiences work best in this new medium and how to create them, but we are a long way from a well-established commissioning process or production grammar. Delivery methods and consumer hardware are still evolving rapidly, and more work is needed to get a solid understanding of how to create a good user experience, and even to understand what the long-term audience benefits are. Watch this space for further developments!