Archives for June 2010

We have a new team member on Monday. Mark, who is one of R&D’s industrial trainees, is going to be with us for a while working across a variety of our projects. He’s off on the BBC induction process for the next two days. George is in an EU plenary meeting. 29 Powerpoint presentations in one afternoon apparently.

We’ve been having some reliability issues with the ingest chain for the Zeitgeist project. We need to implement some tighter monitoring across the message queues and ensure that the system is more robust. By the end of the week Chris N and Sean have done quite a bit of work on detecting and recovering from crashes but we’ll have to let this soak in for a while so launch is going to be put off until next week. Some of you have been asking what this project is - it’s a site that tracks mentions of BBC websites on Twitter, more next week.

Read the rest of this entry

Monday

Read the rest of this entry

Why was BBD R&D hosting the British Library and the British Film Institute for a meeting on 10 June?  Aside from the fact that we share a first name, what is our common ground?

Not everyone may know:
a) that the BBC charter defines an obligation to maintain an archive and provide public access;
b) that public access to BBC TV output is provided by the BFI, and access to BBC radio content is provided by the BL Sound Archive.

BFI and BL Sound Archive do far more: the BFI collects a lot of other television besides the BBC, and is the only archive collection (the only historical record) for many commercial TV companies that have come and gone.  The BL Sound Archive has enormous depth in music collections of all categories, but also has speech recordings from an equally wide range of sources -- from field ethnography (around the world) to recordings of live poetry events (around the UK).

Archives and libraries have a long history, and a well-developed and proven technology for dealing with items on shelves.  IT people think they invented search-and-retrieval systems, but libraries have been working for 200 years on classification technology.  Starting in the 1970's, libraries converted to digital technology:  the BBC card catalogue (for TV holdings) went electronic in 1978.  But content remained on shelves throughout the 1970's, 80's and even 90's -- and only now are audiovisual archives getting to the point where "files on mass-storage" can be said to be even beginning to take over from content on shelves.

The process of conversion to files began in the 1980's with electronic documents, and audiovisual archives have been collecting digital content (CD, DVD, DAT, digital videotape format) since about the same time.  But audiovisual archives have been 'going digital' (to CD, DVD etc) as a separate stage from 'going to files'.  Now digital content on physical carriers like DVD, DAT and digital videotape is becoming obsolete, and the conversion to files is moving from a trickle to a flood.

How do archives convert their practices so they can handle files with the same levels (or even higher levels) of security and integrity -- and service -- that they historically provided for shelf-based content?  This is an issue that has had a large amount of development since the mid-1990's, under the name of digital library technology.  Recent developments include formal trusted digital repositories -- and the considerable effort around digital preservation.

The British Library is a world-class institution in the digital library and digital preservation field.  The British Film Institute has a range of innovations in access (from online collections to the several UK mediatheques), all based on digital technology.  And the BBC has been one of the world leaders in audiovisual digitisation and preservation, and access (a major R&D contribution is Redux)  -- and with the PrestoPRIME project the BBC is now trying to link audiovisual requirements to digital library and digital preservation technology.

About 40 delegates from the BL , BFI and BBC -- and Rufus Pollock of the Open Knowledge Foundation -- came together to work out, jointly, the future for preserving and accessing file-based BBC content.

The meeting ended on an expansive note, with Tony Ageh, BBC's Controller for Archive Development -- outlining the possibility for UK public service and public sector institutions to work in collaboration to create an entirely new Public Space, within which content and assets produced or aquired using public funding may be discovered, without compromising existing legal and commercial concerns. We have the size, the value and the mission to change the fundamental nature of public access. More to come!

The Zeitgeist work is already proving its worth highlighting some interesting trends involving the BBC and glazed carrots. Sean is going to spend the next few days implementing monitoring on the processes and queues as well as tidying up some CSS. George and Sam are out at a P2P-Next EU meeting all week demonstrating some of our recent work and liaising with the other partners. Monday afternoon is spent evaluating what next for Digital Friendship. We're getting into proper security territory where we're not experts, and we're questioning what we're getting out of this and why we're doing it.

Read the rest of this entry

BBC R&D's website has a selection of project pages describing current projects, many of which come under the 'Production' banner.  I thought I'd give a bit more background on how some of these projects fit together, and how the work is organised.  I lead the 'Production Magic' section of R&D - a team of 17 people that looks after new developments in the area of programme production technology, focusing on audio and video signal processing.  All the projects currently listed under 'Production' come from my section.

I (naturally!) think that the work of Production Magic is amongst the most exciting in BBC R&D because our work usually leads directly to new things that are directly visible or audible to our audience.  Some other work at R&D tends to be less visible - indeed, in projects such as video compression, the more invisible it is the better!  Production Magic section currently covers four broad areas of work: audio, the fundamentals of HDTV production technology, and two areas looking at particular applications of image processing and computer vision to TV production.  I've summarised each area below:

Read the rest of this entry

Data is all around us. Public and private institutions are waking up to the fact that releasing data is not only politically useful in an age which values transparency. It can also foster innovation, improve services and reduce costs.

But while the 'Free our data' Guardian campaign, Tim Berners-Lee's linked data project, MP's expenses, data.gov.uk and many more initiatives have been very effective in getting the message across that releasing data is a good thing there hasn't been a corresponding conversation around what can be done to make sense of the data and how to present it in a way to engage people. Lots of people are talking about RDF , SparQL and ontologies but not so many people are explaining how to do something with them. There are some excellent data visualisation blogs but few sites dedicated to teaching the skills needed to create beautiful and meaningful graphics.


The Data Art project on Backstage has the aim of teaching those skills to a wide audience. We will be using BBC data sources to do this and in doing so hope to also provide a new insight into the BBC's output. BBC data sources for developers are migrating to the forthcoming developer site, and we hope Backstage will be increasingly used to demonstrate what can be done with BBC data.

The project is a collaboration between BBC Learning and the University of Westminster and is part funded by an AHRC knowledge transfer grant. We've currently got 4 projects on the site using 4 different BBC data sources and we are in the process of creating some learning resources which will be launched in July. We're also planning an event at the University of Westminster in November and an exhibition in August 2011.

One thing I've come up against in the first few months of the project is that while departments across the BBC are eager to participate there are often obstacles to releasing the data. A good example of this is a visualisation I built in flash which used an API written by Andrew McParland from R&D to allow users to search for the instances of words mentioned on BBC TV channels. It threw up some really interesting results so I plugged the same API into a Microsoft Pivot to use the same data to create a new way to navigate through to iPlayer content. Due to concerns about the use of copyrighted material and issues about compliance these demonstrations were not able to be made accessible to the BBC's audiences on our website, but in this screencast I'll walk you through the key elements of the prototypes. We hope that during the 22 months of the project we can make a case for a relaxing of the rules around releasing BBC data sources.




We hope soon to release a sitemap of the whole of bbc.co.uk, some insights into the output across TV & Radio using Infax and to collaborate with Tony Ageh to create some tools which demonstrate how data visualisation can provide unique ways to navigate around the BBC Archive. BBC R&D have been incredibly generous with the stuff they are willing to share.  We're hoping that we'll get involvement from more areas of the BBC in the months to come.  We are a small team but we have some pretty valuable skills (Flash, HTML5, Forge, etc) and we would love to get the chance to collaborate with other departments.

Please visit us at https://backstage.bbc.co.uk/data_art/

Over on the BBC Internet Blog the ongoing conversation regarding the roll out of HD terrestrial broadcasting continues.  In the latest 3 part installment you can hear directly from the BBC's critics who visited the various facilities around West London where we support HD broadcasts, including R&D.

In Part 2 of the post the various reference systems we maintain, and development systems, are reviewed.  The posts are well worth reading through if you're interested in this technology and would like to know more about the technical issues that impact upon the quality of the BBC's HD services.  It's perhaps more important now than ever, since the BBC Trust approved the launch of BBC 1 in HD.

On Friday the Internet Blog carried a post from our own Phil Layton, head of our Digital Service Development section, on the technologies used by the BBC to provide surround sound services.   Phil gives a good in depth description of the technical challenges which bar us from using the well known Dolby system for broadcasts.

We start the week catching up and planning and setting up for demos for the afternoon. The Digital Programme Guide team meet with Fraser and Brian from UX&D to discuss developing a version of the Digital Programme Guide in HTML5 for other devices. Akua and Tony head off to the shops to look for some toys to use for the Digital Friendship prototype. They come back with a monkey, an elephant and a sewing kit. The afternoon is spent with our sister team in Audience Experience exchanging ideas and demonstrating projects.

Read the rest of this entry