The Hitchhiker's Guide to Encoding: Life, Encoders and Everything (Or a brief history of HD encoding)

If anyone wants a primer in how SD Digital TV works for comparison, I wrote this - How digital television works - ukfree.tv

Andy, I hope we see some more on the subject of encoders and also picture comparison of the old and new encoders...

On first read of your article it appears that you maybe attempting to discredit comparisons people have provided on other blogs? Apologies if I have missed the subject of the article.

This may or may not be the correct place to comment but last night I switched over to BBC HD in anticipation of another cracking episode of Top Gear (many thanks for making this available on BBC HD).

But alas there was another programme on and the tv guide told me Top Gear wasn't to be shown until 22:30 !!!

But let me get back on track the reason I mentioned this is because "Small Island" was a simulcast with BBC 1. So just for a quick and dirty comparison I switched between BBC 1 and BBC HD several times. And guess what I couldn't tell the difference, the picture on "Small Island" appears to be quite soft, could you let us know if the original recording was indeed HD? or was it simply upscaled from SD?

Perhaps when making schedule changes the programmes should be previewed internally to see if the programme actually deserves the slot on the HD channel.

One other point which maybe we can discuss is how ITV HD compares? e.g. what technology are they using compared to yours? I admit I don't watch much ITV HD but have started watching "River Monsters" recently. The picture quality is much better looking IMHO than BBC HD and interestingly when I checked my recordings the bit rate was 12Mb/s. Fast motion of which there is alot in "River Monsters" is very smooth and clean. I'd be interested to know what encoders, equipment they are using. Do you consult together at all (BBC and ITV) on technology used on the FreeSat and new Freeview HD platforms?

I'm sorry to be so negative and I'd personally like to thank you for bringing this technology to us and dealing with our comments.

Best Regards

Dave K.

@Andy. Like the BBC HD PQ, this is not looking good. If I might use a courtroom analogy here:

Defence counsel:
So the new encoder/software combination provides a better received BBC HD picture, as we can now see 'noise' in the picture that wasn't visible before. Is that correct?

Witness.

Yes, that is correct

Defence counsel:

So noise in the picture is a good thing?

Witness:

Um.......

Defence counsel:
Exhibits 1 to 999 undisputedly demonstrate that artifacts and macro-blocking are now present in the received received BBC HD picture. What do you say to that?

Witness:
We don't like that evidence so we have no option but to discredit it.

No further questions your Honour.

Andy,
Thanks for the post. ....digesting ....

I thought I'd make a point to disagree with White3knight (comment #2) re Small Island. I thought it looked very good - perhaps a bit soft, but not as bad as some stuff lately. I certainly disagree that there was little difference between the HD and the SD. Last night's episode was very good with regards to retention of shadow detail and general stability. I for one didn't notice very many unpleasant artefacts at all. I suspect this is due to having a very 'clean' source material - ie no director-intended grain. This and the recent Doctor Who have been the best things on lately in terms of HD picture quality, which goes to reinforce my suspicion that the encoder has problems with 'challenging' sources with smoke/mist and grain. Clean sources certainly give a very watch-able image.

The green lines came and went, but I choose to ignore them as they are not something that I expect will be around for long.

Hi Easybourne (comment #4), I agree that the picture quality for Small Island was good but in my opinion it didn't offer anything over the picture being broadcast on BBC 1.

The picture quality of SD on BBC 1 has been extremely good as of late, I'm also upscaling SD content to HD. What equipment are you using to view with and does it upscale SD or not?

Regards

Dave K.

Firstly when the drop was made from 19 to 16 mb/s there were complaints about lots of programs not just one. It is well known that lower bitrates makes picture quality very susceptable to noise. It is not that you can see the noise it is that it actually degrades the picture quality by waisting bits try to reproduce the noise.

It should be stated that taking still shots is a bit of a hit and miss process. You may be lucky to hit on an I frame and so you would get different artifacts if you hit the middle of the GOP. Its better to put the video in an editor and look at pictures field by field to get a complete picture.

One of my main contentions about all this is that the EBU and the BBC do all their testing with a constant bitrate. In most of TV land pictures are transmitted with a variable bitrate which aims to produce a constant picture quality and utilise bandwidth in a more efficient way. It has been reported that Freeview HD has a maximum bitrate 17mb/s and Sky can go up to 19mb/s. It is about time that BBC HD on satellite started to use variable bitrates.

I am glad to see your next post will cover EBU recommendations. I totally disagee with the way the EBU come to there recomendations. Firstly is the way they do thier testing and secondly is the standard of picture quality they consider acceptable. I will say more about that tomorrow.

You mention that some old encoders were not much better than MPEG2. This is a misleading statement in that as far as I remember these were not AVC encoders just MPEG4 which is far less efficient.

My view is that improvement in encoder technology should be used to improve picture quality not to cut bitrates. It is madness to spend very high sums of money in producing high quality pictures only to throw it all away at delivery.

We have now had to put up with a sub standard service for over 4 months. Thounsands of words have been written but still the picture quality deteriates. We now have to put up with horizontal green lines flashing across the screen. Yet another "filmic" effect?

whit3knight, everything on BBC HD is HD, not upscaled SD.

Andy,

Thanks for part 2!

Regarding screen grabs and off screen photos, yes totally agree you have to careful with this type of comparison. However, the photos I have posted were only taken after noticing problems at normal viewing distance and as an indication of problems observed.

If an P or B frame has been captured, that still doesn't negate the original observation of the problem, i.e. watching on a 42" plasma at 11ft.

This really is not a case of surfing through a sequence frame by frame looking for a problem to take a photo of.

These problems have increased IMHO on demanding material.

Andy, have there been any changes to the encoder set-up over the past few days? Without being specific I have noticed a change, but I do have another variable in my set up to consider.

CompactDistance (comment #7) Actually you are incorrect as archive footage is sometimes shown as part of a programme which isn't HD.

Mentioning Wimbledon, I think the picture quality for HD Wimbledon is great but there is a really annoying sound lag. Is the BBC addressing that?

#9

It is not only archive footage that the BBC upscale. At this years proms all the shots outside the Hall were SD. Sometimes captions are SD aswell.

Interesting, It seems that my picture issues started with the drop to 16Mbps not 9.7. I certainly noticed the extra noise in the waterfall scene and I think the "zip" had gone too. I still say the encoders are killing the dynamic range.

I agree with you about the first gen encoders, I remember all content on the Sky platform taking up too much room on the disk when Sky HD started. With 5 times the detail and an encoder that was twice as good as mpeg2, a recording should have been 2.5 times the size of the SD version but actually it was 5 times the size.

Could you confirm that the first gen encoders were mpeg4 as opposed to mpeg4 AVC. I've always assumed that but I've never known for sure.

BTW, is there no advanced noise reduction you could apply to recordings that would eliminate the type of noise you point out in your example?

words fail me...

If the reference to using screen grabs as a reference is the first part of a whitewash to discredit the evidence of gross macroblocking and poor motion prediction in the programmes reported in previous blogs then shame on you and your masters.

Can't we have a BBC HD day with the new encoders at the old higher bitrate?

If it makes no difference then we'll all observe that, and the arguaments about bitrates will go away.

Interesting blog as always Andy, and look forward to the rest of the week. I too have been slightly baffled by the number people posting highly compressed jpegs of pictures of their television sets as if that's going to prove something definitive about BBC HD PQ. That's not really proving anything one way or the other, apart from the relative quality of your camera.

However, unless you are able to visually demonstrate in these blogs that the new encoders/ lower bitrate combination has not affected BBC HD quality significantly then you will have a hard time allaying the concerns of the large numbers of people posting on here.

Thanks.

The problem is that visually we can see with our own eyes a poor image on BBCHD (granted it is an improvment over the SD channels)

some things do look good, but some things look poor.


It is interesting reading these blogs, and I too look forward to the next few, as its nice to see the inside workings.

However, we know that the BBC's (and freesats) stance is to either call us all geeks, tell us we are imagining things or that the filmmakers decided it should be poor quality.

So there is not going to be any kind fo resolution to the PQ issues, its just going to end with a grown up version of "Na naaa na naaaa na we are right you are wrong", even though people can SEE a poorer picture when compaired with 16 weeks (give or take) ago.

Have I missed something here? This blog post doesn't seem to have any real structure or conclusion.

Andy> Is the gist of it: the screen shots/grabs that show the problems with the new bitrate/encoders aren't valid, unless they just happen to "suggest the new encoder is better"?

Just read your previous posts from the links above. I didn't realise it was all so complicated.

Just wondered though, do you monitor picture issue on one type of TV/monitor or several?

For example, a 1920x1080 recording will be scaled to 1440x1080 for transmission (with the quality reduction that brings) and then to many different resolutions on people's TVs. I reckon 1024x768, 1366x768 and 1920x1080 will be most common. This will surely give differing picture quality and may account for the different complaints people have about PQ.

LCDs and plasmas will also give different results (particularly relating to brightness/contrast/dynamic range.

I think that this blog post is actually a fantastic way of helping people to understand why screenshots aren't always the best way to assess picture quality. Not least because TV moves, and the compression is targeted at moving images, not still ones.

That said, watching Top Gear last night, the airport sequence with Richard Hammond racing looked dreadful. Then everything else, including the studio shots and all other packages, looked great. Was the airport thing SD upscaled? It really looked like it to me, far lower quality than everything else in the show.

Credit where credit is due 'The Hottest Place on Earth' which was aired on Saturday was excellent, the detail was spot-on the dark sceens where a bit grainy but no different from when this was shown either last year or this year, absolutly no softness or blurring on movement, gavin and stacey was crap! and was transmitted with the same encoders, could it be post-prodution or inferior equipment at making? Set your recorders for 12/12/09 at 16:00 for part 2/2.

Andy, no you personally have not, I didnt mean to imply that you have, so apologies if you have taken it that way. It was not the way I intended it.

From what I have read from you it has always seemed a decent fair blog and points made, but I really can only see this being what I said, because people can see it, but the BBC claims its perfect and fine.

Ian, you make a good point re the airport race but you may have hit the nail on the head re pcture quality.

As best as I could see, that footage was shot that way deliberately. It was grainy and had a crushed black level. I can't remember if they used that annoying vignetting filter to darken the corners too.

The thing is, that's not a problem with the HD channel, it's the director trying to be artistic and failing. Also, the extra resolution quite often shows up the director's "ideas" and makes them look worse than on the SD version.

Just to rephrase my original question.

Were the first encoders part 2 or part 10?

Andy, have followed your and Rowan's posts over the last year - and as always delighted to hear detail from so close to the epicentre of UK HD decision making. Keep up this excellent work.

Couple of observations:-

o Scary isn't it that ingest server settings [surround becoming stereo :-( ] are so fragile, and that procedures don't yet fit reality. Is this a case of responsibilities falling down a hole in the first place - with BBC, Red-Bee, Siemens, Arqiva and uncle Tom Cobbley (sp ?) all with an opportunity to contribute to quality.

o What was the rationale behind 16Mbps to 9.7Mbps? I know video compression is an area of rapid development, but 40% from 2007 to 2009 seems a bit optmistic, esp given the negative audience reaction.

o Is it intended that Freesat HD will have the same compression parameters (for BBC HD anyway) as Freeview HD? Isn't the capacity of DVB-S2 (up to 72Mbps per transponder) so much greater than that of DVB-T2 (40Mbps per channel) that Freesat could have a better quality?

Andy could you not up the bitrate during the evening when the programmes start and lower it during the daytime so as to make it more sustainable, no extra cost.

@vism The airport race was a hold-over from the last series. It was filmed last Spring. This is why it was in SD.

This seems a good series of blogs. Glad you add comments in reply Andy, it makes it worthwile reading the other comments knowing that if someone posts to ask or state something important they are likely to get an answer. And breaking it all down into a series helps keep the discussion on topic I think, which is again good.

Hi Andy,
Regarding the validity (or otherwise) of looking at individual frames - although absolute quality c/w the original is not valid except on I frames, screen grabs showing frames with dreadful macroblocking are surely completely valid as a means of showing where the compression has totally failed to cope.

The Angel Falls image you refer to was one which I looked forward to seeing and enjoyed time after time - it had massive 'wow factor', noise or no noise. How about showing it now. Would I still go 'wow'?

Myself and others have mentioned comparisons with ITV HD. When BBC HD was at 16Mb/s, it was visibly better than ITV HD, no question. I remember watching an episode of'Sharp' for example, and commenting on how it wasn't! Now, post bitrate reduction on BBC HD, I actually look forward to watching ITV HD - the Star Wars films have been excellent, for example.

Motion smearing is another major bugbear of the new encoders. This is only appreciable when watching moving images of course, so grabbing stills is pointless. For me this is every bit as big an issue as the macroblocking we're getting. Even Gavin and Stacey has been dreadful from this point of view, never mind the awful live football.

So much of BBC HD's output seems to be 25p rather than 50i now. Even programmes such as Live at the Apollo, where 25p is totally inappropriate, are using it. It makes me wince to see that big bold backdrop strobing its way across the screen! Is this a matter of Producer's discretion gone mad? Or is it rather that the new encoders cope marginally better with progressive material, and it is becoming the de-facto default choice?

Very interesting reading. And probably quite useful for most of the readers. As sometimes I find that even if I disagree with a decision, understanding the decision-making processes can make it easier to swallow.
That, and it can be quite eye-opening to see behind-the-scenes stuff. As it can show when things aren't as simple as you'd expect - or, conversely, when the opposite is true.

And as to your section onstill-frame comparisons, can you please send it to most current TV Showrooms? I stopped paying attention to still-frame comparisons when shops started using them to compare Blu-ray & DVD or SD & DH channels.
Yes, there is an obvious difference between SD and HD content. But those in-store frame shots make SD content look even worse as they show up the artefacting that's somewhat less noticeable in moving footage.

Also, I'd imagine it's possible for either side of the argument to pick the one frame that's far better/worse than the others to prove a point.

Thanks for the post Andy - I think I'll wait to read the 'full story' before I make many comments directly - so let me comment on some of the followup comments.

#post 2 Whit3Knight wrote:-

'But let me get back on track the reason I mentioned this is because "Small Island" was a simulcast with BBC 1. So just for a quick and dirty comparison I switched between BBC 1 and BBC HD several times. And guess what I couldn't tell the difference, the picture on "Small Island" appears to be quite soft...'

I agree - I did the same and couldn't see a 'stunning' difference.

With 'Life' (on Sunday) there was more of a difference (between SD and HD) in the last section showning the coral reef - but in other sections there was little difference. Now in the low-light parts I understand that - but where they showned the boat they were going to sink to 'help' the reef - there was only a marginal difference in picture quality.

#post10 Easybourne wrote:-

'The green lines came and went, but I choose to ignore them as they are not something that I expect will be around for long'

Most expect Andy to get on this and indeed he has already commented so - but along with sound glitches blog posters have have raised it seems strange.
a) We report sound problems - reply:- Indeed there are.
b) We report flashing lines - reply:- Indeed there are.
c) We report and report and report and report there are quality issues with the HD pictures we as now seeing on what? 70% of the channeels output? - reply:- Indeed there are NOT!

#post 8 tagmclaren wrote:-

'Regarding screen grabs and off screen photos, yes totally agree you have to careful with this type of comparison. However, the photos I have posted were only taken after noticing problems at normal viewing distance and as an indication of problems observed.'

I agree - I don't start watching a programme with my camera at the ready - but if during the programme I think this looks barely DVD quality I might try to grab a few stills. (ie photograph the TV in still picture mode) Now I know ths can only give an idea of a static image but it;s all we've really got.

And Andy - I assume the points about the different types of frame that are captured doesn't apply to my 'captures' because the TV still-frame function will only 'stop' on an I-frame.

And to other posters I know the image will be affected by how I take the image but even though it's a Jpeg (at 100%) I wouldn't post a picture that wasn't at least 'representational' of what I saw in the first place. ie the softness of the picture quality on 'Paradox'

Cheers, daveac

Daveac (comment #38) Some of us watching with media centre PC's, such as MythTV are able to grab stills direct from the stream. I can look into this but I thought some of the other bloggers had already done direct stream grabs (rather than just photos).

Andy, it's clear that it's a complex business - I never doubted it, and I look forward to learning more about it all in the coming week. Please keep your in Blog comments coming thick and fast too; they're very much appreciated.

Personally, I will reserve final judgement until I've read the whole series, but at this stage I feel my passions raising and I can't help but say well said to daveac at #38 above, with his problem a, b, c point. To my mind, problem c) exists - the picture is undoubtedly worse, and rather than constantly denying it (and as you do for the other problem issues) please just admit it and then explain to us all why it has to be so. If it's a very good reason, then I'm sure most of us will just accept it and shut up (and perhaps go out and get ourselves a Sky box and Blu-Ray for Xmas too). No promises from me though.

And Andy, re: #28 Ms Scott (MD of Freesat) did say that, and has admitted doing so in a personal email to me. I'm not clear on her current employment relationship with the BBC but I know she joined the BBC in 97 and has evidently worked with Danielle quite recently so, next time you meet her, please feel free to have a quiet word with her on my behalf (or ask Danielle to).

@Simon thanks for confirming the airport part was SD. I think it proves that my TV and eyes are up to the job! I noticed straight away that it looked a lot worse than the rest of the material.

I should point out, I have no objection to upscaled material at all, in fact, I wish BBC 1 and BBC 2 had HD channels to themselves, so we always got high-bitrate content.

I had a problem with the green flashes too, small, thin lines that shot across about half the screen a few times during Top Gear.

@PixelJunkie "So much of BBC HD's output seems to be 25p rather than 50i now."

You're right, the BBC reckons its about 70-80%

#39, just seen that comment, and yes, HD_fan428 did already do a fantastic job putting up evidence of screen grabs direct from the screen, both here and as evidence for my BBC Trust Appeal submission. Perhaps, Andy you are already starting here to build a case to discredit this evidence, I don't know, but as others have pointed out we don't need the evidence ourselves because we can see it sitting down in front of our TVs just watching the telly.

It's the BBC Staff and those doubters who very occasionally pop out of the woodwork on the Blog, but also for the BBC Trust who, if they don't actually watch BBC HD in their own homes, will need to see a picture, or two, just to show what we're talking about in the 600+ pages of A4 generated already in these BBC HD PQ Blogs. For HD-fan's stupendous efforts, I am very grateful! As I said before I'm now looking forward to Thursday's Blog, most particularly to see your own (I'm assuming counter) evidence on the encoder/bit-rate capabilities.

#43 I meant direct from the STREAM (not screen) obviously.

Re 2005 encoders

I may have got this wrong as it was from memory. It might not have been that they were not AVC it may be that they were not "high profile". I do remember that it was difficult to get hold of software encoders that supported high profile. I have looked at the EBU web site and they have no reports on the encoder testing during 2005 and no reference is given in the EBU report Andy quoted.

OK I have discovered that the "high" profile was not added to the specification until March 2005 and so it makes it more likely that profile was the cause of poor performance.

I can understand the decision to split this blog into parts so comments go with the right bits, but unfortunately that has led to lots of comments speculating on what the conclusion will be, or what will come later etc.

Might have been better to publish all the parts at the same time to prevent this!

So sometimes there is some noise in the original content?

So sometimes a still-frame comparison could be unfair if one shows an I-frame while another shows a B-frame?


Fine.


Now what about the ridiculous amount of blocking that's visible in difficult-to-encode content when watched on a TV by normal human eyes?!

What about the lack of fine detail when you broadcast moving images that are supposed to be full of fine details?


I know too-low bitrate MPEG-4 AVC / H.264 when I see it.


Cheers,
David.

I've been watching all the on going discussions with interest for some time now but first time I have posted

I'll be honest I haven't been watching the channel a huge amount but my feeling is that it looks a touch soft at the moment. Now I understand how some programs look various ways due to how the program makers intended them to be, I'm fine with that but I certainly haven't tuned in for a while and had the picture any any of the programs I have watched make me feel I was clearly watching HD over a good SD channel.

I was also changing between channels last night while waiting for Top gear to start, and while the content wasn't the type I would expect to make to go wow at the difference, I was surprised at how little difference there was.

I look forward to the upcoming posts over the next few days, and while my last comment on bitrate should probably be saved for the upcoming blog post on bitrate, i can't help think about how ITV westcountry was put onto a much higher bitrate when it was put into service for the analogue transmitter feed (has touched 10Mb/s at times and that's an SD channel, admittedly Mpeg2 rather than Mpeg4 for HD channels)

Andy the question i have for you i personally do not believe it for myself but it has been banded around now for a good few weeks if not months. Andy is the intention of the new encoders for a more efficent and up to date compression system or is this all about being on par with the freeview HD picture quality.

soul4real - good question, unfortunately it's not on Andy's list of ones that can be answered (incidently, you and others please check emails tonight - vmt).

see

I know we all seem to bang on about the current picture quality on BBCHD, but Ray Mears on tonight (6.30pm, 7th December)is a prime example of the way things look on the new lower bitrated BBCHD - no better than SD dvd quality. All the sparkle of HD absolutely missing. Judder on panning shots. For an HD transmission, it was quite simply very poor indeed. Some close-ups looked okay, but overall, this was not 'stunning'HD.
Andy, can you not see this?
With your own eyes?

I hope this will be considered to be on-topic.

Thank you Andy Quested for your reply to my post #38 (in post your #49)

Along with paul_geaton I'm trying to bite my tongue until the 'blog' is finished.

And to paul_geaton - Thanks again for all your hard work. I hope more people here add their names to this 'issue' by contacting you.

Well Done!

Cheers, daveac

I’m glad to see some detailed technical posts after what has felt like an age of, to put it charitably, PR. However, I’m a bit disturbed by the tone of this blog which could, as others have said, be the first step in trying to discredit the evidence that has been presented of picture degradation after the encoder change. Still, it’s also of course important to eliminate as many variables as possible, so here goes.

First, I originally posted some A-B screen captures. Here is one such again for convenience:

Nature's Great Events, episode 3. As usual, you have to double click the image to get the full size version. The performance of new encoder is considerably poorer than the old one.

Old encoder
https://img690.imageshack.us/img690/4834/zebras16.jpg
New encoder
https://img258.imageshack.us/img258/5474/zebras9.jpg

I took these grabs without knowing such dramatic differences could potentially be accounted for by unfairly comparing an I frame with another type. Fortunately for me, this isn’t the case here. The old encoder grab is a P frame and the new one a B frame. A look at the subsequent frames shows there is no appreciable difference between B and P frames in this case.

So this brings us onto the recommendation that the best test would be between I frames. The results of this are interesting. Obviously they won’t necessarily coincide, but here are two from consecutive frames:

New encoder
https://img81.imageshack.us/img81/9571/zebras9i.jpg
Old encoder
https://img42.imageshack.us/img42/1896/zebras16i.jpg

It seems obvious to me that the new encoder is better! However, before we all pack up and make appointments with the opticians, I have to question why it should be that I-I is the most reliable comparison. A quick check shows that BBC HD seems to constantly broadcast one I frame followed by 11 of the other types, so these are what we see for the vast majority of the time. I may be missing something here, but in what sense then does comparing I frames give us the best measurement of image quality? Surely B and/or P comparisons bear closer relationship to what the viewer actually sees? This would match my own experience. As I said at the time, the differences are obvious to me at normal viewing distances when you watch one version of a sequence followed by the other. I await an answer with keen interest!

A final point about using jpegs: I *promise you* that on my PC monitor the images I’ve posted are indistinguishable from the frame grabs pasted into the image editor. Certainly any differences would be dwarfed by the differences on display here. As they are about six times the size of the jpeg I’ve stuck with the latter, but if necessary, and I can find a suitable upload site, I’ll post the uncompressed files there. I’m also happy to supply the relevant clips if required so the BBC can verify my results.

A further thought: given their nature, might not I frames be more robust in the face of bitrate starvation? (Sorry to mention the b-word. :))

I've been following these blogs with increasing fascination and well done Andy for sticking your head above the parapet.

For me one question has never been satisfactorily answered. Almost all of the problems being discussed here have been caused by an obsession with reducing bitrates. The awful quality of the SD pictures from the rugby last Saturday on BBC1 shows this needs to be addressed across all of the channels. Can you, Andy, give us a rough idea of how expensive bandwidth actually is to the BBC. I can't think that the cost of renting transponders and paying other companies in the chain can make up more than a very small proportion of total production and broadcasting costs. How significant are these savings in terms of the overall budget?

Also, is it not possible for you to increase bitrates according to the nature of the material being broadcast? Sky seem pretty good at this. Why not the BBC?

I thought my eyes were going funny watching the opening segment at the airport on last weeks Top Gear in HD. It looked bad at times.

https://img9.imageshack.us/img9/1238/bbchd2230202.png
Aspect ratio uncorrected (1440x1088 pixels) so not to introduce re-compression artifacts into screen capture.

Hi HD_fan428,
I think you're exactly right. MPEG compression is a balancing act between allocating data to the I frames, which are in effect JPEG frames, and the B and P frames. If there's insufficient data rate available for the B and P frames to describe the differences from the I frames, blocking and/or motion judder is likely to result. But the I frames themselves will still look OK.

#58

B and P frames use less bits that I frames. It is possible to encode just with I frames which make the video very robust but needs a higher bitrate. The DV video format used in SD camcorders is effectivly all I frames ( reference frames). Some codecs only produce I frames to make the video easier to edit. B and P frames are used to reduce the overal bitrate but the video is less robust. If an error occures in the middle of a long GOP the rest of the frames in the GOP are messed up. So to get the lowest bitrate it is better to have a long GOP.

B and P frames are not displayed on the tv screen. The decoder calculates the from using the I frame as a reference and displays the frame on the screen.

To be honest I don't know why Andy has gone into this depth as it rather irrelavent. The photographic evidence that people have posted is perfectly valid in fact it understates the problem in that it does not show the very disturbing motion effects. All frames are compressed using the Discrete Cosine Transformation including I Frames. This means that all frames including I frames have a quality that is bitrate dependent. This all means that the picture quality will deteriate along the GOP as the errors gradually build up along the GOP. The skill is to optimize the GOP length to give the best overall picture quality for a given bitrate. So photographing the I frames will give the best quality but does not reflect the quality of the GOP as a whole.

As I have said all this discusion about codecs is a bit of a smoke screen. In the end the issue is what picture quality is acceptable to viewers and this is where the BBC has got things very badly wrong. As I am sure we are soon to discover the BBC failed to ask the great British public what quality they would expect from an HD service. The "join freesat" poll shows that 61% of viewers have seen a degrading of picture quality. So it is about time the BBC stopped this rediculous soap opera of endless discusion and actually ask the licence payer what he wants.

Does the BBC realy want BBC HD to continue to have the worst picture quality in the UK.

Andy, no flashing for me on the Chatterley affair, well not of the sort your talking about. Not too bad a picture either. Shame it's not in 5.1 though!

Thanks for your comments Andy and its good you answers some questions.

Sadly your blogs are not really discussing the major issue of lack of bandwith and the major drop in qualiity.

We (the BBC HD viewers) have all seen a masive drop in quality and its about time you faced up to the fact.

Its great having all these fob offs about encoders and what they do blah blah blah.

What I care about is the fact the quality of picture is second rate at the moment and no one at BBC HD is taking responsibility.

Why cant you be honest with us Andy about the bit rate increase??? Why cant you discuss the quality reduction without fob offs???

Us viewers are not making the quality decrease up Andy so please stop trying to act like theres no issue and talk to us about it. How long can this farce of a channel carry on for.

I meant bandwith "decrease" not increase.

Ben, email me please. (https://www.zen97962.zen.co.uk/ final disclosure)

Andy, I just caught a flash, and it wasn't the earlier kind, it was one of the green ones. 2323.

Green Flash seen 23.26

Also an audio click in and out of the Paradox trailer

To all bar Andy, here's a newsflash re: my Appeal about Freesat HD PQ sent to the BBC Trust a fortnight ago.

I quote their response: "I consider that some points which you raise have not been fully addressed in the response you received from Danielle Nagler, and I am therefore going to pass your letter to her to provide you with a further response at this stage."

An unnecessary delay, I think, but in the event that I am unhappy with Danielle's further reply the Appeal will be submitted again to the Trust. And next time, if necessary, it will be an even weightier document that will deal with the issues being raised here by Andy.

If anyone wants to contribute to that, then just email me please (my address is here: https://www.zen97962.zen.co.uk/ , in the final disclosure document). The sooner I start to utilise your help and prepare for that eventuality, the better our changes of a successful appeal.

This comment was removed because the moderators found it broke the house rules. Explain.

@74 Don't forget that Andy has a job and wishes to keep it - I feel that he is replying very well given the constraints of a corporate environment.

This does not mean that I feel satisfied with all the replies so far, but we are getting far more than I would ever have expected.

A few things to bear in mind...
The noise problem is a real one - I remember trying desperately to find decoder filter settings for Torchwood (series 2 - 2008) where many dark patches were full of dots and sparkles. If any of you still have this series on disk, take a close look at Jack's greatcoat in the title sequence where the whole team are marching together.

As for SD/HD content - some of the "classic" stuff was shot at 720, not 1080. I am thinking typically of "Lost land of the jaguar" where the "720" on the cameras was covered with black tape when they were in-shot (but not all of the time).

Those who use DVB-S cards in their computers should be very wary of posting screenshots unless they are 100% sure of their decoding path. I have seen rubbish results at 16 Mb/s, let alone 9 Mb/s. Another word of warning re DBV-S cards is that they can seriously degrade the ts if they overheat.

All that said, I join the others in saying that currently BBC HD has totally lost that "wow" factor since the drop in "b-word". I only need to look at a Bluray or older ts streams to confirm that the "wow" factor is still there for me on proper HD content.

Andy, keep up the good work - I get the impression that you at least are pleading the cause of the viewers with BBC management.

Finally an idea to build on the suggestion of the "HD day": how about a test sequence transmitted sequentially at 1080@19Mb/s (old encoder), then with the new encoder at 1080@16 Mb/s, 1080@9Mb/s then finally 720@9Mb/s. I suspect that many "good" plasma owners might find the latter looks better upscaled than the 1080 at the same bitrate.

Come on BBC! Day3 please :D

some of us have children that seem to take great pleasure in making sure the whole house is awake at 5am.

*rubs sleepy eyes*

I must say that this is all very interesting and will soon lead us to a conclusion of sorts in the coming days.

However, the worry that I have that never used to be there is due to the massive variety in quality that we see. Dr Who, Top Gear (HD segments) are generally seen as high quality examples of HD on the BBC. Life is varied, Ray Mears is (though not seen it myself) generally viewed as appalling whilst Jules Holland has gone from being a shining example of studio based HD to something substantially less with macro blocking and artifacts all over the background in the interaction between the lights and the darkness.

Why the varying quality when, even at the static bitrate of 9.7Mbps, some "busier" shows have better PQ than some not so busy and vice-versa?

#59: Dear Andy Q, I know you hadn't had much time to respond to my post, but this was of course not the answer I was awaiting with keen interest! I hope that in the fullness of time you will be able to say something less mysterious. :)

I am so glad to have found this blog, although I will confess I don't really understand the heavier technical discussions here. You see, I'm a non-techie middle aged housewife who for some time has wondered what has happended to the glorious HD quality that I had on the BBC HD channel when my other half invested heavily in expensive equipment some 2 years ago. And we were thrilled at the time.

The 'jaw dropping' quality of the HD channel is no longer so. Can it be our set up? No, it's still brilliant on other non-BBC HD transmissions. Is it my aging eyesight? No, I'm happy to report that my prescription hasn't needed a change in years. So when at last I found this and previous blogs the mist was lifted from my eyes - but unfortunately not from the HD channel.

I will not pretend to understand bitrates & blocking, I am not a geekish punter, I am a licence payer who once enjoyed BBC HD in all it's glory and that is sadly no longer my experience. So, I post in order to join the ranks of those who have noticed the difference and are saddened by it, and even more saddened that the people in charge seem so contemptuously dismissive of the viewing public who attempt to challenge their stance that there is no noticable deterioration.

Many moons ago I worked for another great British institution, Messrs Marks & Spencer. Their ethos then was 'the customer is always right' and when they veered away from that they suffered greatly for it. Unlike with M&S the viewing public cannot withdraw financial support from the BBC but the principle remains the same. Come on guys, the BBC HD service IS suffering and will continue to suffer unless someone there is brave enough to put their head above the parapet and admit that in this case the customer IS right.

And some of us find this background information genuinely enlightening. I'm more interested in the process than the end-result, personally. It's how my geek-nature manifests itself - I love behind-teh-scenes looks. Especially about broadcast technology. So I'm actually enjoying this part of the series.

@81 Tiggs
I agree with you. Understanding what goes on in the broadcast chain is fascinating and if this series of blogs was meant to be purely educational, then they would be right on the money.
But it isn't. Their primary raison d'etre, as far as the majority of the readers of them are concerned, is to respond to the avalanche of posts about the deterioration in BBC HD picture quality as now seen in viewers homes, compared to the initial BBC HD transmissions with the built in wow factor.
If they don't do that, then in my view, they have failed to meet their objectives.

the bottom line is after all this talking and big words bbc hd looks
bad. freeview hd will have the same bad picture quality as BBC HD if
BBC HD gets its way,i watch bbc hd on my 52 inch sony 3500 which cost
me £3000 , its very easy to see now how bad the picture quality as
become, looks soft, looks graing,lac of sharpness. its afact i can
see it looks bad, 9.7mbs is no good. when it was 16mbs it looked
very good... thats the bottom line i can see it.
the only thing we can all do is complain to the BBC, email or phone.

Last night, the upscaled Gordon Ramsey's F Word on Channel 4HD looked better than Sunshine on BBC HD!!!

I remember Sunshine looked excellent the last time it was on.

I'm sure a bunch of chimps could do a better job of running BBC HD!

Do not get sucked in by all the techno speak it is a distraction designed to keep us quiet. I find it patronising and a shear waste of time. The BBC HD PQ is still crap and continues to be crap apart from the odd occassion where it is "reasonable" on some programs. I am not being negative just realistic!

I do not watch it any longer it depresses and annoys me to much

Mike

@mike

I totally agree, I've just stopped watching BBC HD completely now, it just makes me so mad that I've invested a huge sum of money into the equipment needed after seeing the "sales pitch" of the original HD quality only to find it fractionally better than SD most nights after "the bit drop".

You can talk all you like Andy about techno babble but be honest with yourself, are you REALLY certain that the PQ is now just as good as it used to be? If so... wow....

No matter how much techno-talk that appears here it will not change the fact that the BBC HD channel is easily the worst quality HD of all the channels. Like many thousands of other people I feel ripped off and disgusted with the BBC.

If I were to drop the "pound rate" at which I paid my license fee to say £25.00 but told you that it was only your imagination that it had dropped and you must be a geek for even suggesting that there is a difference as it is simply the bank using a better encoder to send you your money would I get away with it... I wonder?

Back to Sky 1 HD, you pay for it, but at least its HD.

BBC HD poor picture quality. this might be ever bodys last chance to
get some think done about it , the bbc hd channel do not care, we all
no how good BBC HD looked at 16mbs , after the bit rate was cut to
9.7mbs the picture looks, soft, grainy , and as very poor sharpness.
the only way it is going to look good again if it put back to 16mbs.
The BBC HD BLOG is trying to control us all, we all need to complain
NOW or it will be to late. freeview hd is going to look just as bad.

so i have contacted the INDEPENDENT NEWS PAPER the story will be in
it on thursday 10th DECEMBER. we all have to complain, and take it
to the next level out side of BBC HD or thair will be no change.
the BBC is trying to control all these complaints.
p shakeshaft.

The Independent have published their story. Predictably, the DM have it too:

https://www.independent.co.uk/news/media/tv-radio/bbc-criticised-over-hd-picture-quality-1837376.html

https://www.dailymail.co.uk/news/article-1234797/BBC-viewers-quality-HD-channels.html

This comment was removed because the moderators found it broke the house rules. Explain.

This comment was removed because the moderators found it broke the house rules. Explain.