The BBC Philharmonic Maestro
Post categories: Audio, Awards, North Lab, partnerships
| 12:12 UK time, Monday, 16 April 2012
We are very excited to find out that The BBC Philharmonic Maestro has been nominated for a Webby and a Design Week award.
The BBC Philharmonic Maestro is an interactive virtual orchestra that allows a user to step up to the conductor's podium. Using intuitive and natural gestures, the user can conduct a performance of the BBC Philharmonic Orchestra as if they were a real conductor.
The user's view of the BBC Philharmonic
We captured a performance of the William Tell Overture using a 3D audio technology called ambisonics and high resolution video. A 4K video was captured with the camera positioned where the conductor would normally stand. The camera was equiped with a very wide angle lens so we could capture the conductor's whole field of view. This gave the conductor an unusual challenge because he had to conduct around our rig.
Conducting around our recording equipment
The interface uses a Microsoft Kinect to track the positions of the user's left and right arms with skeletal tracking. Microsoft have released an SDK which allows anyone to work with their skeletal tracking algorithms which is available here. The positions of the left and right arms are then converted into a beats per minute tempo (based on their lateral movement) and a left hand and right hand volume (based in the height of each hand).
Depth image and control data from the Kinect
An application was then built to control the audio and video playback. The application recieved the control data (tempo and volumes) from the Kinect and played back the recording of the orchestra's performance, altering the tempo and volume while keeping the audio and video in sync and maintaining the correct pitch.
The BBC Philharmonic Maestro was created with the help of MagneticNorth and it has been presented at two international conferences (here and here) and has been featured at two international arts festivals (here and here), not to mention the Webby and Design Week award nominations.
Getting the chance to conduct the BBC Philharmonic
We're up against some pretty touch competition for the Webby, but if you like the idea of the Maestro you are able to vote for it here.
Comments Post your comment