When the BBC Horizon team came to BBC R&D with the problem of building an Augmented Reality (AR) Sandpit, two of 2016's cohort of Graduate R&D Engineers jumped at the challenge. As part of Mars - A Traveller's Guide, Martian geologist Prof Sanjeev Gupta used the AR sandpit to explain how liquid water once shaped Mars' landscape.
The Horizon team had seen the AR Sandpit developed by UC Davis and asked us to build one for the programme. The AR Sandpit uses the latest silicon technology (children's play sand) alongside an Xbox Kinect depth-sensing camera to measure the shape of the sand's surface in realtime. This depth information is used to project colourful contour lines back onto the sand to bring out the miniature Martian landscape carved out Prof Gupta's hands. Using the software developed by the UC Davis team, the sandpit can also simulate how water flows around the terrain. When the Kinect senses a hand held above the sandpit virtual rain pours onto the sand forming virtual lakes and rivers projected onto the surface.
The original AR sandpit design used 'short throw' projector mounted at head-height above the sandpit to cover the whole pit. Unfortunately these projectors aren't especially bright, particularly when projecting onto a non-ideal sandy surface. Though in practice this still looks good to the human eye in a darkened room, the same can't be said for TV. Though cameras continue to get better in low-light conditions, the best pictures still require good lighting. To make matters worse, the Horizon team wanted to film the sandpit in Imperial College London's Data Observatory which features a floor-to-ceiling panoramic wall of (bright) monitors. When combined with eye-wateringly bright TV studio lights, a dim projector was out of the question.
Armed with a much brighter projector we immediately hit upon a snag: this new projector was designed to be mounted a long distance from a screen and would need to be mounted several meters above the sandpit. To fit the setup into the Data Observatory (without demolishing the roof) we needed to come up with a better solution. We realised that by mounting the projector low-down and pointing it up at a mirror above the sandpit we could halve the height of the setup. Using a mirror mounted just above head-height, our brighter projector was able to fill the entire sandpit.
To fit in with the Horizon team's extremely tight filming schedule, we needed the sandpit to be quick to setup on the day. With a whole day's filming planned, and only part of it featuring the sandpit, everything would need to be assembled off-set and wheeled in at the last minute. By modifying a mobile TV stand we were able to securely mount the projector, mirror and Kinect sensor in advance. Moving the sand and sandpit, however, would simply be a matter of brute force...
One of the surprising things about real-world TV productions is that almost everything happens at the very last minute and, against the odds, always seems to come together. This programme was no different. About 24 hours before filming took place, the Horizon team decided that the sand we'd been using didn't flow smoothly and attractively enough because it was slightly damp. One of the production team had tried drying the sand in their oven and found that it gave the correct texture and suggested we try to dry out the sand. Unfortunately with the clock ticking and some two hundred kilograms of sand to dry, sticking it in the oven wasn't going to work. Instead we spread the sand in a thin layer over the floor of the R&D TV Studio and left the industrial air conditioning system to dry out our makeshift beach overnight. The next morning, to our great relief, we were greeted by the marvellous sight of silky smooth, dry sand.
We joined the Horizon team at Imperial College to reassemble the sandpit and get it ready for the shoot. As ever, the team had a few last-minute requests for tweaks to the setup including changes to the colours, water simulation and adding a 'power-on' effect. Typing with a keyboard precariously balanced on a knee and using the rather 'grainy' screen we were able to implement all the changes just in time.
The project turned out to be a success and gave the graduate engineers a first-hand insight into TV production. The sandpit has since been taken on the road to Teen Tech to inspire the next generation of STEM practitioners - and visitors to BBC R&D's South Lab can come and have a play themselves.