The Hitchhiker's Guide to Encoding: So Many Tests, and Thanks for All the Recommendations (Or the BBC and the EBU)

Thanks Andy, interesting stuff - really looking forward to the next installment though!

I think the question to be posed from my perspective on this is why are these new encoders being used at a fixed bitrate to which it blatently looks as if they are not able to operate correctly at yet? You mention you're on v1 of the firmware - isn't that the equivalent of a company having rolled out Vista as soon as it came off the shelf and then wondered why they were getting loads of user complaints that nothing works?

Surely you only release new hardware/software etc when it has been fully tested, retested, viewed, tested, and ensured that there is 100% no difference to what was previously being received by viewers instead of jumping in headfirst into a new encoder that might well end up being as efficient as is hoped with the requisite PQ instead of leaving us with currently 4 months and counting's worth of increasingly poor and varied PQ?

I note your comment on picture formats but why are you broadcasting BBC HD for overseas use at 1920 x 1080?

Seeing as it's panto season...

Andy: "Depending on the manufacturer and their current stage of development good HD at 8-10Mbs is achievable now."

Audience: "Oh no it isn't!"

BBC HD Management: "Oh yes it is!"

Audience: "Boo, hiss, geroff!"

BBC HD Management: "Where's the problem? We don't see one?"

Audience: "It's behind you!"/"on the screen" etc. etc. ad infinitum.

You get the idea...

Is the 8-10Mbs for non-live encoding? The salesmen may be saying it's OK to run an HD channel at that bitrate, but the consumers are saying something else. I fear the smoke machine is nicely warmed up and the mirrors are in place...bring on the test results.

Thanks Andy, this is great stuff.

This may be a stupid question, but when you look at averages wouldnt it make more sence to broadcast the preview loop at a lower bitrate (or keep it as it is now)

and then when broadcasting the actually channel, use a varible bitrate and allow it to use more b/w if needed, but also it will use less when not needed, so it would average out.

I look at linowsat.com (and dont know how accurate it is) and see that a lot of the Sky channels average a lower amount than BBCHD with its fixed bitrate.

Now films etc.. do have black bars from tiem to time so I suppose that needs to be taken into account.

I couldnt find Sky1's average bitrate on that site (was sure it was there last time I looked) but as BBCHD only broadcasts on limited hours couldnt the offset from broadcasting the preview loop in a lower bitrate be used to increse the average during its active hours, and wouldnt the avergae be give or take around the same anyway?

Thanks Andy,

I may be wrong but I don't recall anybody saying "1440 Horizontal pixels is not HD"

I do however recall people asking why the BBC has policy of using 1440 when other broadcasters don't seem to have an issue with using 1920?

Yes not all cameras are full 1920. Yes not all TVs are 1920 capable. However lets face it, it won't be very long before most are.

Some people listen to small portable radios, however you don't limit the audio bandwidth to 5KHz because they are not capable of reproducing beyond that.

So, could you clarify the BBCs position on this?

OK Andy,

Lets take your word for it, these new encoders have developed within about 6 months to enable a further 20% reduction in bitrate, even though one document says their development is fairly mature.

In that case can you explain why I am still seeing obvious picture defects at a normal viewing distance? Do I have a problem or defficency with my equipment?

Also, can you say when the permanant mix / fade fix was applied?

In addition, I don't think many posts have been about the frame size being 1440x1080 - the vast majority of posts have been complaints about the picture quality not the resolution. I hate to think how it would look if you tried to run it at full resolution with the current bitrate.

However, it does beg the question why BBC HD being transmitted in Scandinavia via Thor@1degW gets full 1920x1080 resolution at 15Mbs? Because they're paying an optional subscription for it and wouldn't tolerate the version the UK license fee payers are receiving perhaps? Why isn't that running at 8-10Mbs, Andy?

Andy,

You say

" As I mentioned earlier, the old encoders had reached the end of their life but the new encoders are at the beginning. They exploit more of the AVC toolset than the old encoders even though we are on version 1 of the firmware. We will continue to add improvements as and when they are available"


Is that not admitting that the encoders are currently deficient in some ways?

Can you state in what ways you believe the current encoders to be deficient in that case?

Why would you need to improve something that is already, according to everything we have already been told, perfectly acceptable?

Also, from existing policy, if the PQ is already perfectly acceptable, won't you just be using encoder improvements to reduce the bit rate further?

@ post 8. tagmclaren wrote:
I may be wrong but I don't recall anybody saying "1440 Horizontal pixels is not HD"
------------------------------------------------------------------

I was wondering where that quote came from as well.

Danielle Nagler previously asserted people had said the bitrate reduction had a "catastrophic" effect on the picture quality", yet when I asked if someone could help me find where the word 'catastrophic' was used, I got no responce.

I am beginning to sense an Orwellian rewriting of history in order to provide an argument where the stakeholders concerns can be more easily managed.

Another very interesting technical blog explaining how the BBC go about the business of delivering HD.

However, none of this goes anyway to explaining to viewers why HD looks better on Sky then on the BBC.

Well with regard to this blog post and bit-rate it seems strange that yesterdays Life 'Plants' showned the real limits of having a fixed bit-rate at this level.

Some of the close ups of Venus Fly Traps etc. looked very good - but wide shots of plants and forests looked poor and 'smudged'

I one shot was about the worse picture I'd ever seem on BBC HD (near the end) a camera shot rising out of 'what was supposed to be' a wheat field - it was like an abstract picture. Other shots of grasses and fils of flowers were also sud-standard.

This meant that this one programme had some of the best of recent pictures on BBC HD alongside the absolute worse (at least in my opinion of course)

Cheers, daveac

Andy, it seems that the BBC have a policy of reducing bandwidth on it's TV channels. When DTT launched the channels had a far higher bitrate which has been slowly but surely reduced. We've had channel 302 (Red Button 2) and the News multiscreen removed, and 301 and BBC Parliament have been reduced to a resolution of 544 x 576 which is below what I'd define as standard definition, and looks awful on today's HD televisions.
Now we've got a HD channel broadcasting at possibly the lowest resolution you can get away with at 1080i, with its bandwidth reduced as low as you can get away with, and you're attempting to justify it through a change of encoder, which EVERYBODY can see it not up to the job yet at such low bitrates.
I know you've been forced to give up some space on Freeview, but why was this allowed to happen? I think DVB-T2 should be used as an opportunity to move towards broadcasting everything in this mode, SD and HD channels, and in MPEG-4/H.264. This would essentially, due to increases in efficiency, give you a lot more space to play with. Is this likely to ever happen?

@Andy

We all appreciate the time you have invested in bringing to us this series of blogs. I have 3 observations so far:

1) I have noticed that you are very quick to answer non contentious questions but seem to duck the contentious ones as a matter of course. However, I think Balderdash's question in post 10 merits an answer as it will give us an insight into the business side of the BBC. OK, so it isn't a technical question, and may be outside of your remit, but I am sure we would all appreciate it you could find the answer from the appropriate person and post it here.

2) On the subject of contentious questions, these are some of the key ones asked many times but remain unanswered:

- Why was the higher bit-rate not sustainable.
- Why can't we have a short period back on the higher bit-rate as a simple test.
- Why can't the satellite transmissions be statistically multiplexed to make better use of the available bandwidth on the transponder

3) This may be unfair, but this blog gives me the distinct impression that you were under instruction to trawl through all available documents until you found a reference or two that sit well with what the BBC is currently doing. It's possible to demonstrate anything is true if the will or the need is there (politicians do it all the time), but that will never change the fact that the BBC HD picture quality that we see in the comfort of our own homes now isn't what it used to be.

I wish one could edit after posting!

Corrected:-

Well with regard to this blog post and bit-rate it seems strange that yesterday's Life 'Plants' showned the real limits of having a fixed bit-rate at this level.

Some of the close ups of Venus Fly Traps etc. looked very good but wide shots of plants and forests looked poor and 'smudged'

One shot was about the worse picture I'd ever seem on BBC HD (near the end) a camera shot rising out of 'what was supposed to be' a wheat field - it was like an abstract picture. Other shots of grasses and fields of flowers were also sub-standard.

This meant that this one programme had some of the best of recent pictures on BBC HD alongside the absolute worse (at least in my opinion of course)

Cheers, daveac

Have to agree with some posters Andy that you are avoiding plenty of questions and being selective about the ones you answer. What is the reason you keep failing to answer the fundamental questions that keep being asked?

As far as I know, the BBC specify delivery of HD programmes for transmission on HDCAM SR tape. The format of the signal on tape is 1920x1080. The BBC also specify that post-production should ideally be done using a high-bit rate codec - eg Apple Pro-Res 422 HQ or Avid DNXHD 185. These codecs process the video at 1920x1080. Decent studio HD cameras shoot at 1920x1080. Even the Sony EX1 camera used for lower budget programmes shoots at 1920x1080.

Faced with this, it's nonsense to suggest that transmission only needs to be at 1440x1080, because any vaguely quality production will have been shot, post-produced and delivered as 1920x1080. Even if some footage has been acquired at 1440, it will have been up-scaled during post-production. Surely it's not a good idea to up-scale it, down-scale it again for transmission, and then up-scale once more in the viewer's TV?

Listen folks they have gone through all our blog querys with a fine tooth comb, why you think it took so long for us too get these answers now, they have dug there heals in and thats final! when i ask the question in relation to the satellite bitrate and freeview, if they have to be of the same quality, which is a simple YES or NO, and get an answer to go and read the previous blogs and if you don't find the answer come back to me, i personally expect better than that. Andy said in Post:64 of Brief History of HD Encoding how he has been watching channel 4 all evening and not seen Flashing, but never mentioned once that he has seen blurring or a programme that looks soft. In the next few days he will tell us how the are the only channel who show a wide variety of programmes, so would you noy think you would have sufficent head-room to cope with the varing demand.

Are all programmes encoded to h.264 directly at transmission time? Or does pre-recorded material have the luxury of being able to be encoded - with the resultant opportunities for better "quality" picture encoding - when it is loaded into the playback systems?

I've followed this debate with interest, albeit that I don't have access to the HD Channel at the moment.

From afar, it seems that the encoders angle is perhaps muddying the waters and the issue is thus: The entire raison d'etre of the HD channel is the picture quality (5.1 sound). People feel, subjectively, that the picture quality is inadequate, particularly compared to HD offerings from other broadcasters.

The BBC can quote as many bitrate readings or encoder specifications as they like; these details are interesting but are ultimately irellevant. If the punters feel that the picture quality isn't good enough, then the channel is, by definition, not fit for purpose.

(There should, of course, be an "and" before the reference to 5.1 sound in my previous post.)

Andy,

Can you explain what the visible effect of "GOP pumping" is?

Thanks for this, Andy.
It's interesting that all your quoted bitrates above talk of 12mb as a minimum for HD, apart from the following:
"...with the expected future developments in video coding, it is assumed that HD fixed bit rate requirements will be reduced to 8-10 Mbit/s per programme. There will also be advances in the transmission system such as DVB-T2..."
which is a speculative comment about the future.
I, too, am concerned with your evident reluctance to answer the more controversial questions.
I have been watching BBC HD since the day Sky HD went live, and remember talking to their technical guys, and asking whether they were embarrassed that the BBC provided the best content on their new technical platform. (The response was "No comment", by the way.)
I am sad and disheartened that this is no longer the case.
The BBC HD picture isn't bad, but it could be much better with a small increase in bitrate or, as has been suggested, the use of variable bandwidth.

Fixed bitrate in this day and age is something that could be termed a 'schoolboy error.' Fpr mainly static images you don't need anything like 9Mb/s and for complex scenes you need more, so why waste bandwidth where it isn't needed and not provide enough when it is?

Here's an idea.

If the recordings (on hdcam) of a couple of years ago were 1440x1080, they would look good when transmitted at the same resolution.

Now that hdcam sr is being used more, would the scaling down from 1920 to 1440 cause a lot of the picture probs were talking about?

Perhaps we just need a small increase in bandwidth (10.5 maybe) to allow 1920x1080 to be transmitted.

A question on stat mux.

Can mpeg2 and mpeg4 be stat muxed together?

If not, the whole idea of stat muxing is a moot point. What can the beeb multiplex it with? Can't see ITV wanting to get involved.

Correct me if I'm wrong but aren't these EBU standards just based on a minimum that the various contributing bodies and commercial interests think they can get away with rather than a fixed standard of excellence, based on quantity of channels rather than quality.

I get the feeling that any encoder improvements will not result in an improved viewer experience, just a further reduction in bandwidth.

I'm sure the Emporer's taylors had plenty figures to prove how wonderful his new clothes were too, as did the DAB apologists.

I'm no geek, despite being called that by parties involved with this attempted smoke and mirrors, all I want to know is why BBCHD can now transmit the worst examples of HD programming in this country, why some high profile programming has suffered in comparison with previous broadcasts and what is going to be done about it.

"what is going to be done about it."


Nowt it seems.

As the world is currently debating on how we can best save the planet from overheating can we make our contribution here?
Andy, in order to conserve energy could you please answer the following questions. We will then have no further need to continue with this series of blogs.

1. Will the BBC be increasing the bitrate in the near future? Yes?No

2. Will the BBC be changing the picture format to 1920x1020? Yes/No

3. Will the BBC be introducing a variable bitrate? Yes/No

Although EBU Tech 3333 states that 1440x1080 should be a “supported” resolution the final paragraph of Tech 3328 sect. 1.1 states:

“Strictly speaking, services that are not based on a 74.25 MHz luminance sample rate should not technically be labelled as 'HDTV'.”

The preceding table on page 7 states 1440x1080 has a luminance sampling rate of 54MHz (i.e. not HDTV)!.

Grafix: don't expect an answer for at least another 3 months they have to do a spell check for YES/NO answers first

I hope all the null packets they can now transmit alongside BBC HD on satellite are worth it!



btw, BBC R&D's own term for the quality they aim at is "substantially artefact free".

Sadly, very few BBC broadcasts actually use the bitrates that BBC R&D say they need to.


I bet in a year or two, something will slip out about what BBC R&D _really_ said about these new encoders. As with DAB and Freeview, I bet this will contradict the "official" story.


But who cares eh? At least they can send more null packets!

Cheers,
David.

Have a read what was said by someone from the bbc tech team, is this not the same ideology the hd team should have blog #42



https://www.digitalspy.co.uk/forums/showthread.php?p=37162417

The first thing I want to ask is on what authority does the EBU determine what picture quality is acceptable. We must remember that the EBU is a private club for broadcasters. It is run to support the interests of Broadcasters not the interest of the viewers. Many of their documents are kept secret just go here

https://tech.ebu.ch/publications

and try to read anything from there Forecast 09. You will get a "You are not entitled to access this information". The BBC is a member of the EBU and licence payers pay the membership fees and we are not allowed to view the contents. This is particularly anoying in that some of the presentations are given by the BBC at our expence. So anything published by the EBU must be read in the context that it is a secretive club for Broadcasters. This means that any recomenations they make are made in the interest of broadcasters not viewers.

As has been pointed out the BBC subsamples to 1440 pixels from 1920 and the reason is very simple. For any given bitrate there is a resolution which will give an optimum picture quality. The higher the bitrate the higher the optimum resolution. The low bitrate used by the BBC means that the optimum resolution is lower than 1920 and nearer 1440. That is why the BBC subsamples to 1920. It is pointless to argue if this is HD or not because there is no definition of what HD is. The point is that 1440x1080 is a lower definition than 1920x1080. Some TV manufacturers have coined the phrase "Full HD" to distinguish 1920x1080. The all means that the BBC is transmitting a lower definition than other broadcasters.

The EBU argument about it not worth transmitting 1920 does not apply as most HD material is shot at 1920.

Andy

Thanks for the blog (I should have said this in my original response above) and the previous and up and coming ones and the response to my initial note. However, I will be interested to see if there is any reference to what people are mostly asking - as to why there are such variances in PQ and what can be done about it.

Surely, as another poster on here has correctly identified, if so many people are seeing a huge degredation in PQ over time and, most importantly, from the same time, then something is surely amiss somewhere. You have BBCHD at home and (I would bet) Sky HD to compare and contrast. I hope that your later blogs identify what differences are seen at the receipt end of the broadcast chain.

I hope you didn't infer that I assumed no testing had been done - it just seems odd that if sufficient testing had been done, then why all of a sudden are so many issues being experienced?

I look forward to the end of the week - like a new type of Advent Calendar...

Andy,

Would I be right in saying that 1440x1080 was a temporary botch to the HD standard to allow for cams that couldn't handle full hd and that a channel shouldn't be called a hd channel unless it's 1920x1080i25 or 1280x720p50

#Andy

The reason I said they were secretive is that they do not alow non members to read most of their publications.

I am not realy sure why you have posted all this. The discusion is not about how effient the new encoders are or what the EBU says. The argument is over picture quality. For any h264 encoder old or new if you increase the bitrate you increase the picture quality. All we are saying is that the majority of viewers want a higher picture quality than is being transmitted by the BBC. Simples.

@vism

1440 subsampling was adopted by the HDV format so they could fit 1 hour of long GOP MPEG2 onto a DV tape. I think some of the early AVCHD recorders also subsampled so they could use a lower bitrate.

Andy,

Could you confirm what Trevor has said, I thought it had something to do with ccd sensors or upscaling SD from 720 to 1440.

While I'm sure it uses less bandwidth, I don't think that was the main reason.

Gosh, a lot to digest. I'm only up to #12, so far. Refering to that comment, and #8 as well, and the mysterious quote "1440 Horizontal pixels is not HD" of Andy's. I have a record of everything said on all the Blogs and I can confirm that, like the word "catastrophic", it just isn't there.

In fact, in the couple of dozens mentions of "1440" the usual sort of Blog comment is exemplified by this one from Steve Bagley, which makes quite a different point:

"For instance, Sky Sports HD is encoded at 1920x1080, while BBC HD is encoded at 1440x1080 (a sensible decision as up until recently 90% of HD sources, on any channel, are likely to originate as 1440x1080) this means that for same level of compression, Sky Sports is going to need more bits than BBC HD. And its not as if the the previous BBC HD encoders were perfect. They really struggled with credit crawls against black at 16.5Mbps (quite often you'd get tiny splats of magenta or green squares for a single frame where it forget to encode part of the image)"

and, likewise, this one from trevorjharris:

"They are still claiming up to 5 times picture detail. Well the BBC transmit 1440x1080 and that is only 3.75 times more detail, and this does not include the loss of detail due to the low bit rate"

Perhaps the closest thing to Andy's are these from SkyCaddie:

"How can you say you use best quality when full HD is 1920 x 1080 and you tranmit at 1440 x 1080"

and tagmaclaren when he, as he has already pointed out himself, said:

"The remit for BBCHD is to offer a "very high quality service matching or exceeding industry standards for resolution." So, why are you transmitting @ 1080i 50 * 1440?"

Andy, these are not quite the same as what you've said we're saying, are they? Personally, I wouldn't want to suggest that you or Danielle are purposely trying to re-write history. However, the words being used by you both to summarise what viewers are writing in these Blogs can be misleading and may actually distort the truth e.g. when Danielle modified my complaint re: the bitrate reduction from an "obvious, considerable and detrimental effect on the channel's PQ" to a "catastrophic" effect on the picture quality".

I think what needs to be pointed out is that when you want to refer to something we've said just use an actual comment, in quotation marks, referring us to where you've got it from. Apologies if, post #12, someone has already made or clarified this point, but if I wait until I get to the end before wading in I'll forget everything I want to comment on.

"This year for example, several manufactures were showing early versions of HD coding at 4-6Mbs (average not minimum)."

On a slightly different note this pretty much makes Blu Ray (and the now defunct HD DVD) optical media redundant as a film would easily fit on to a regular 8.5Gb DVD.

As we aree into quoting from EBU documents, from 3328:-

Referring to a table of luminance sample rate, 54 V 74MHz (1440 V 1920)

"The lower the level in the above table that is used, the lower the bitrate needed to produce
'artefact free' images, for a given scanning algorithm, but also the lower the potential detail in the
picture - which is important for the HDTV experience."

also

"It is reasonable for broadcasters to inform their viewers about the quality they have provided in
their services. This is a sensitive issue, because many broadcasts today use ‘sub-sampling’ prior to
broadcasting, to allow a lower delivery bitrate at the expense of some loss of detail in the picture.
Strictly speaking, services that are not based on a 74.25 MHz luminance sample rate should not
technically be labelled as 'HDTV'.

Too right it's a sensitive issue! :-)

Balderdash (comment #10) - I'd also love to see an answer Andy on why the subscription broadcast of BBC HD on Thor (for Scandinavian viewers) is at higher resolution and bitrate?

I'm guessing these questions are going to get ignored so I hope the BBC Trust will ask the same.

Regards

Dave K.

From EBU tech 3334 Feb 09

"MPEG-4 transmissions (as with other advanced coding systems) will particularly benefit from statistical multiplexing. In a large statistical multiplex, with mature encoders, future HD services may be able to operate with an average bit rate of about 8-10 Mbit/s.

In a standalone service, up to 16 Mbit/s will be needed, depending on the development of encoders in the future. In a small statistical multiplex, the bit rate needed will lie between the two."


So, mature encoders in the future stat muxed may get down to 8-10 Mbit/s.

BBCHD currently operates standalone.

Thats an average too, I wonder what the peaks are, as it reads as in a varible rate not fixed.

Andy #50

Your remit is to match or exceed industry standards for resolution.

So, can you clarify the reasons for using 1440 when other broadcasters seem to have no particular issue with using 1920?

thanks

If the 'advances' we have had so far are any indication I shudder at thought of more. Any benefit these advances bring do not seem to be shared with the viewer.

Andy,

Are you using a GVT vibe em3000?

Hello Andy, hope the carol concert went well. Now you are full of Christmas spirit, I know you dont want to comment yet on PQ on Freesat, but can you confirm that there is a difference and why on Thor's BBC HD service?
Its a good question that others have raised and could easily be answered today by you without, Im sure, pre-empting anything to come over the next few days
Thank you.

ps anyone have access to both services?

Dear Andy,

Thanks for your references on standards.

It seems to me that there is an unspoken problem. The elephant in the room: External political pressures to solve political problems. Some mandarin at OFCOM wants to maximise revenue in return for his/her knighthood.

Therefore there is pressure to shoehorn 4-5 HD channels into a multiplex. In order for those channels to squeeze in, something has to give. The excellent work on DVB-T2 has enabled just over 40Mb/s space to be available (so far). MPEG4 as you indicated should improve bitrate by 2:1. However, that is a theoretical figure on a like for like basis. MPEG2 is probably within its theoretical maximum in terms of technology, whereas MPEG4 is in it's infancy, as you indicated; V1 of the firmware. Given that 19.5Mb/s in a mature MPEG2 environment, it seems on the surface that an immediate 2:1 improvement to an immature MPEG4 environment does not seem likely somehow. Please convince me.

Furthermore, the aforementioned pressure on shoehorning suggests that external pressure has been brought to bear to find 'ingenous' solutions to reduce the bitrate; e.g. a reduction from 1920 to 1440 pels/line, disguised in spin as "efficiency".

It may well be one of a range of 'suitable' formats to broadcast, but does it fulfill the public service remit of 'world class' technical quality expected of a premier broadcaster? In this regard, the BBC is running a 'Bentley' whilst Sky is running the 'Rolls Royce'.

Lastly, there seems to be a race for the minimum possible bitrate for a given source, but there seems to be no room for QUALITY. Tanberg promote their latest MPEG2 coder as able to produce 10% better pictures or to reduce the bitrate by a similar amount to increase channel capacity on a multiplex. How many broadcasters are going to take the 10% extra quality route? I suggest none.

Efficiency is a tension between Quantity and QUALITY and I suggest that in the current circumstances, it is tipped too far towards quantity concerns rather than quality requirements.

...so long and thanks for the BITS.

#59, Squegg. I'm glad someone finally brought up the elephant in the room so eloquently. I'd been waiting 'til Friday to see if it came out, but I think you're right to bring it out in the open now.

I concur with all you've said, bar one detail - I'm not sure it's a Bentley, more like a ZIL.

Who actually set the HD standard for Europe?

#59, Squegg. I would have thought 40Mbps should be plenty for 5 channels when using stat mux, particularly if the encoders keep improving.

I would be interested to know if there is an unwritten rule within the beeb to make the picture/bitrate the same for satellite and freeview but I can see why there should be. There will always be variations between the two even if the beeb aim for equality because of different channels in the mux's on the 2 platforms.

the bottom line is after all this talk and big words hd is 1920 by 1080,this is what it sould only be..
freeview hd will have the same bad picture quality as bbc hd if BBC HD
gets its way, i watch BBC HD on my 52 inch sony 3500, which cost me
£3000.its very easy to see now how bad the picture quality as become,
looks soft, grainy , lack of sharpness. its a fact i can see it looks
bad 9.7mbs is no good. when it was 16mbs it looked very good.
that is the bottom line, what you can see.....
the only thing we can all do is make a complain to the bbc or this will
carry on.

vism at #52
I've been trying to find that out. Obviously the EBU is involved but, as these things are pretty much global, I think the standards were agreed by the International Telecommunications Union. Trouble is, all the documents with the right titles seem to be subscription only.

Squegg at #59
I agree with you about bit-rate improvement. It's prediction based on previous trends which may not happen. It may not even be mathematically possible to reduce bit-rate and maintain a reasonably accurate potrayal of the original data. Let's face it, the picture we see is only an approximation to the original because most of the data has been discarded in the encoding process.

Hi,
Re Andy in post 56
"An update on the flashing vision mixer. It is to be taken out of service for a full check and re test. We are running on an older mixer for a couple of days"

Pity. I so wanted it to be the new coder that was going faulty!

Regarding the improvements in compression possible with MPEG-4 over MPEG-2, a highly respected authority in the subject I was discussing this with recently (not as far as I know a contributor to this blog) remarked to me that "to date, the expected improvements of MPEG-4 over MPEG-2 have failed to materialise in practice". His opinion was that, in practice, MPEG-4 was no better (or worse) than MPEG-2 currently. This seems to correlate with what we've all seen on BBC HD, where the bit-rate reduction has degraded the picture quality despite the supposed advantage of the MPEG-4 compression.

Regarding the discussion of 1440 vs 1980 pixels in BBC HD transmissions, a couple of observations: Firstly, although we might feel instinctively that this matters, in practice, it probably doesn't. I say this because some programmes I've seen that have been broadcast at 1440 pixel res have looked absolutely stunning. For example, the (pre 9.7Mb/s) transmission of "Lost Land of the Jaguar". The naturalists-cum-cameramen were brandishing Panasonic DVCPRO HD cameras all over the place, but the quality was terrific. Also, if you look at the test card that goes out during the Preview Loop, even the finest (ie highest fequency )graticule is crystal clear, at least on my telly. My point during my previous blog entry was more to ask why the BBC persists with 1440 when 1920 pixel production is now probably the norm and is easily affordable. HDCAM/DVCPRO HD cameras are old hat now in what is a rapidly developing field, and to be working with - apparently - cutting edge encoders but working with a backward-looking policy seems contradictory to me.

Andy, I hope the Carol concert went well, welcome back.

I'm still awaiting your final few installments before I really get stuck-in but, I have to say now, for all these technical words @128fish sums it all up perfectly for me.

When people are finally able to get Freeview kit, and perhaps buy it after having seen demos of Blu-Ray and $ky, you're going to start getting an awful lot of complaints.

They'll be flooding in as soon as they first tune-in to BBC HD, if you've given them nothing more in terms of PQ than Freesat gives us "as a general rule" at present.

#61 Paul, I havered over Mondeo for a while...

#63 Vism, That presupposes an mean rate of 8Mb/s per channel which is less than the previous half-MPEG2 rate. I agree that stat-muxing will help, but only as far as it goes within mux-ing limits. The clue is from recent references from Andy and Danielle over the introduction of 'more challenging' material. i.e. material with more movement in it.

Also, let's not forget that there is already a massive amount of compression going on. The raw data rate is around 39Gb/s (1440x1080x25) {so much for the vaunted return to square pixels!}.
It would be interesting to know the average number of I frames per second. Anyone?

Encoder advancements will, I'm sure be inevitable, but attention should, as I've said before, focus on improving QUALITY rather than the diminishing return of shaving the odd 0.5Mb/s off the bit rate.

It seems that just "it'll do" is the new gold standard. Typical British thinking.

#68, Squegg, unfortunately it's not the odd 0.5 Mb/s they're aiming for. By 2019 they want it down by another 7.7 Mb/s. Our own Andy Q at this year's HD Masters:

“Headroom is vitally important. It’s the thing that allows programme
makers to degrade their pictures before they send them to the public and still look very good,” said Andy Quested, Head of Technology, BBC HD.
Unfortunately, “quality drops at each stage of post production.” BBC HD transmits using MPEG-4 at 16Mbps and some things are easy to encode, such as drama, while others, such as sport are a lot more difficult. In tests, at 5Mbps, the progressive stuff still stands up to scrutiny, although it’s getting a bit noisy, he explained. “As speed changes, the effect of the codec changes.” Codecs work differently, so that Dirac may produce watchable pictures at rates that AVC doesn’t, and the various codecs are improving all the time. He believes that within ten years,
viewable HD pictures at 2Mbps will be possible.

Just come in and wife was watching paradox on BBC1, oh I said its on the HD channel and switched over. woo she said "thats a big difference isnt it, NOT". "Dont know why you get so excited about HD"
Sad thing is she is so right. (holds head in hands in despair)

#68: On the basis of a very small sample which could be wrong, it seems BBC HD runs mostly with a GOP length of 12 (with occasional exceptions that I've measured as up to 20 frames). Thus at best about every 12th frame is an I frame. I've no doubt it probably looks pretty good during those frames! My little experiment on this is here:

https://bbc.kongjiang.org/www.bbc.co.uk/blogs/bbcinternet/2009/12/the_hitchhikers_guide_to_encod_1.html

I'm still wondering why I frames are the preferred subject for A/B comparisons, when on this evidence the other types can be so much worse at 9 mbps than at 16, and they make up so many more of the frames we see. In both ways, then they seem more representative of viewer reaction on here.

Perhaps the next blogs will reveal all?

#70, I've got Paradox on too, although it's definitely not my cup of tea - there's nothing else on tonight. But satboy50, your wife's quite right - there's little to mark it as good HD. It's not got 5.1 sound to distinguish it for a start, and as for the picture it really is nothing special. It can look OK when the camera is still, but the camermen seem to have forgotten to bring a tripod along to shooting, and they all seem to have the shakes too, so the camera has hardly ever stopped moving. Or perhaps that's one of the new HD "looks" that Danielle goes on about.

Paul its part of new BBC standards, tripod legs have been reduced by 40% to improve the quality

There's been some vertical banding in certain brownish shadow areas in Paradox

bbc hd sounds like we are all not happy with the very poor picture
quality. i phone up the bbc today they say if people are not happy,
you have to make a complainat to the bbc , email or phone or nothing
will happen.. you all have conplainats, and then wait for a response
, if you are not happy then you can complain to the BBC TRUST.

@ Andy Quested


If this blog post forms the basis of the final conclusion then any assumptions here need to go to the EBU first.

Can the BBC actually go below current standards without EBU ratification? If it is part of a body that ratifies standards for EBU members then surely it needs to be reviewed rather than pushed onto the public without prior testing.



The BBC is following what is laid out for DVB-T2 services by using multiplexing which is giving the channel an average of around 12.5Mb/s and allowing the channel to use up to 18Mb/s if needed.

Why are satellite broadcasts being used for tests?
Is there an explanation as to why the BBC is one of a few channels out of the dozens of HD channels in Euorpe that uses a lower resolution?





When writing a technical document the EBU will always take into account changing technological standards.
It's DVB-T2 standards are up to date.

The encoders mentioned in BMN085 and BMN87 used for instance could have been advanced prototypes which you mention later on. The documents are for members only so a reasonable assumption would be the EBU used some of the encoders yet to be released in order to create a future standard set bit-rate to keep ahead of changing technological standards.




If the BBC is unsure of what encoders were used in BMN085 and BMN87 or the relative status of them compared to what the BBC uses then it needs to find out and have them benchmarked against what the BBC uses to see if they are equal or if there is a difference as you are making an assumption there is a difference in these encoders when it is unknown if these are old encoders that were tested or prototypes based on the technology you use now.




As the BBC has gone below EBU regulations for bit-rates specified surely it must raise the bit-rates back up to the current EBU standard until the EBU ratifies any evidence put forward by the BBC that the currently accepted rates should be lowered.

As a relative newcomer to this whole debate, I have little to offer on the technical side - yet am concerned at the level of obfuscation being thrown at the basic issue of PQ by this 6-part blog...

Whilst I can clearly see the degradation in overall quality over the past few months, one thing that perplexes is that the PQ did not 'fall off a cliff' at start-August & has plateaued since - NO, it seems that it continues to degrade on an almost-weekly basis. Is it just me, are other posters noticing this too - it seems that now the norm is soft & one must really hunt for that elusive gem that offers some glimpse of what HD should be?

OTOH I have just watched yesterday's (repeat of) 'Travels with Vasari' & thought the PQ excellent with only the odd soft 'looks like SD' shot in there. I hope you caught this show yesterday Andy & can proffer an honest answer to this conumdrum - how can this show look so good (& a few other rarities e.g. the recent Waters of Mars), yet most of the other BBC HD offerings (I offer up Flog It, Jimmys etc) look so hopelessly soft & definitely not worthy of the moniker HD?

Andy,

Excellent comments above. All summed-up with 6 basic questions for you from the channel's viewers, worth restating here. If you don't answer them tonight then I hope you will by Friday. 3 x Why? and 3 x Will?

1. Why was the higher bit-rate not sustainable?

2. Why can't we have a short period back on the higher bit-rate as a simple test?

3. Why can't the satellite transmissions be statistically multiplexed to make better use of the available bandwidth on the transponder?
&

1. Will the BBC be increasing the bitrate in the near future? Yes?No

2. Will the BBC be changing the picture format to 1920x1020? Yes/No

3. Will the BBC be introducing a variable bitrate? Yes/No

Some people seem to think the EBU is some broadcasting law enforcement agency or some ivory tower authority. The EBU "is" people like Andy, his team and their counterparts in national broadcasters around Europe. Those technical docs are being added to right now by the testing Andy is doing right now on us the BBC HD viewers, those that are left.

Next year those mpeg4 compression recommendations will state you can get away with 9.7Mb/s for 1440x1080@50i if the source material is very clean. Don't run a blog though because the more discerning viewers will complain like crazy the first time you have to play out any material that's less than perfect and it all falls apart.

But seriously, I've seen the general drop in overall quality (with the exception of certain programmes), I've also seen an improvement in certain difficult scenes especially water but plenty more scenarios where the encoder fails without much grace. Stat mux would be great, I see that CH4 HD uses about the same disk space for a 1 hour show but they are 1920x1080 and their better upscales look better than BBC's worst native HD but they have other channels to stat mux with.

So come on ITV & CH4 give Andy something to stat mux with, I'm sure if BBC HD stat muxed with no-one until the others turn up there would be big screams when ~40Mb/s dropped to 10Mb/s ish. I suspect what we watch on Freesat IS the very same stream that no-one is watching yet on Freeview HD.

Andre, great comment, thank you. And BBC HD, for the moment, all PQ issues are forgiven, thanks to you for bringing me the Thick of It tonight. Classic comedy, loved it! Can't wait until episode 8, I'm on tenterhooks (like for Andy's Fri blog).

Andy, How about answering the Thor question myself and others have raised?

Andy of course we can hang on for the answers we have waited months for answers to the basic questions that you keep avoiding - however a lot of the negative blog comments would stop if these questions were answered - your constant 'avoiding' of the basic questions is not only frustrating for us but smells of a cover up.

So here they are again......

1. Why was the higher bit-rate not sustainable?

2. Why can't we have a short period back on the higher bit-rate as a simple test?

3. Why can't the satellite transmissions be statistically multiplexed to make better use of the available bandwidth on the transponder? &

1. Will the BBC be increasing the bitrate in the near future? Yes?No

2. Will the BBC be changing the picture format to 1920x1020? Yes/No

3. Will the BBC be introducing a variable bitrate? Yes/No

Thanks for the reply Andy, but it didn't really answer the question didi it? :-)

Obviously with mpeg encoding "resolution" isn't a simple measure of pixels, lines, or samples per line, but your answer doesn't really explain why 1440 instead of 1920.

You might well be able to convince me it is of no benefit, however that still doesn't explain why other broadcasters think it is desirable or necessary. After all they want to save on bandwidth costs too.

Under ideal circumstances and with good source material running with 1920 and an appropriate bit rate for said resolution the PQ could exceed the satandards of the rest of the industry. Just as is suggested in the BBC trusts remit for BBCHD.

We don't just want it to "stand up well" as you put it, we want it to be the best. We collectively pay the BBC rather a lot for these standards to be maintained and expect it to be so. (Individually I think the BBC is a bargain BTW ;-) )

Did anyone catch the repeat of Sunshine last night? There was a scene when Steve Coogan's character was singing to his wife in a club which had instances of the worst HD I have ever seen. In fact I would say that certain shots were of a sub-VHS quality! I switched over to Sky1HD, the picture was smooth, a little soft maybe, but streets ahead of BBCHD. How is this acceptable for a showcase channel?

@Andy Quested

The EBU and other bodies are usually ahead of the curve in regard to regulations, they have the latest DVB-T2 recommendations which the BBC are following giving BBC HD roughly 30% more capacity on Freeview HD than on satellite.

Is it not possible to upgrade the bit-rate to stay within the current recommendations of 12Mb/s while submitting the BBC findings to see if it gains EBU support or if the EBU finds the current bit-rate is to low based on its own findings?



Citation 5 on page 6 of EBU Tech 3334 only references EBU BPN085, BPN086 and BPN087 (EBU Members only).
Tech 3328 doesn't reference EBU BPN085, BPN086 and BPN087, an assumption would be the two aren't linked, the document has more citations in the bibliography than through the document itself.



An assumption could be EBU BPN085, BPN086 and BPN087 used advanced industry technology being created which you later reference
''Two or three times a year they do show off concepts or the next generation prototypes at the main trade shows but products to the broadcasters arrive as and when they are stable and ready.''

If EBU BPN085, BPN086 and BPN087 have used technology only recently or yet to be released it would alter your later findings.



On the resolution issue EBU Tech 3299 informs EBU members.

The specification comprises 4 HDTV production systems:
• System 1 (S1) with 1280 horizontal samples and 720 active lines in progressive scan with a frame rate
of 50Hz, 16 x 9 aspect ratio.
• System 2 (S2) with 1920 horizontal samples and 1080 active lines in interlaced scan with a frame rate
of 25Hz, 16 x 9 aspect ratio.
• System 3 (S3) with 1920 horizontal samples and 1080 active lines in progressive scan and a frame rate
of 25Hz, 16 x 9 aspect ratio.
• System 4 (S4) with 1920 horizontal samples and 1080 active lines in progressive scan at a frame rate of
50Hz, 16 x 9 aspect ratio.

The application of this specification is intended for, but not limited to, the television production environment.

https://tech.ebu.ch/docs/tech/tech3299.pdf

This was written in December 2004 so why is there the need for the BBC to provide a lower resolution now?


TS 101 154 V1.9.1 itself is described itself as ''Specification for the use of Video and Audio Coding in Broadcasting Applications based on the MPEG-2 Transport Stream''


The lower resolutions you reference are for legacy resolutions.


The outside broadcast industry itself is conforming to a 1920x1080 50p 3Gb/s standard which the rest of the HD industry will follow, once the majority of content in available in 1920x1080 is there any reason for the BBC not to increase its resolution and follow this standard once the majority of content is in this resolution?

Andy, I should say that I was seriously impressed when I discovered that these blogs exist, cheeky comments aside, you (and everyone blogging here) deserve a great deal of respect for interacting with the viewers like this.

When I compare good CH4HD upscales to BBCHD's worst HD, I mean that the overall experience of watching is better, not that the upscale looks like HD and your HD looks like SD as many have said. I have a calibrated 1080p DLP projector with 90" image, if there's an artefact there I'll spot it! Personally I find excessive compression artefacts more objectionable than overall softness of a good upscale. Squirming "lumpy" backgrounds in the subterranean sections of Gryff Rhys Jones's Rivers or most of Criminal Justice for example. Sadly I never watched beyond half of the first episode of CJ it was too painful to watch.

Have finally seen Montezuma now (with 5.0 audio !?) and yes Andy, PQ was excellent, more like that please.

I see the mixes and fades are much cleaner now, great stuff, there seem to be subtle improvements generally too but keep going, it's not there yet...

I never understood the SD looks better comments coming from some until my mother bought an LCD flat panel, prior to a firmware upgrade BBC HD was making it go nuts, it seemed to be a very efficient compression artefact amplifier! It's still not immediately obvious what's HD and what isn't even now though.

It's supposed to be a competent set, Samsung 40B6000. All the "enhancements" firmly switched off and lined up by the most excellent and thoroughly appreciated BBC HD test card.

BBC HD something is wrong with the sound quality, on strictly come
dancing, i can hear white nosie in the back ground.

BBC HD POOR PICTURE QUALITY . this might be everbodys last chance to get
some think done about it, the BBC HD channel do not care, we all no
how good BBC HD looked at 16mbs, after the bit rate was cut to 9.7mbs
the picture quality looks, soft, grainy, and as very poor sharpness.
and poor detail. the only way it is going to look good again if it is
put back to 16mbs. the BBC HD BLOG is trying to control us all .
we need to complain NOW or it will be to late. freeview hd is going
to look just as bad.

so i have contacted the INDEPENDENT NEWS PAPER , the story will be in it
on thursday 10th DECEMBER . we all have to complain, and take it to the
next level out side of the BBC HD or thair will be no change . the BBC
is trying to control all these complaints.
p.shakeshaft

andy
no andy the sound quality is very good. but i can hear white noise
it is not fan noise.. andy do you know what white noise is.
the picture quality allso looks soft.. no all this show is in HD
parts of this look in SD.

andy
thank you i will check next weak. but what about the picture
quality which looks soft, and as poor detail, allso parts of
strictly come dancing look like SD quality.

90 @128fish, like the article - good effort!

now after my complain to BBC HD about very poor picture quality ,
after the bit rate was droped from 16mbs to 9.7mbs, i have recivied
the standand reply, this is a link to danielle nagler blog about
picture quality. we all know danielle can not see what is wrong
with the picture quality. and we can...
i can remeber when every program look very good, at 16mbs
so now i have to write to the BBC and try to complain again.

p shakeshaft

#14, daveac: "Well with regard to this blog post and bit-rate it seems strange that yesterdays Life 'Plants' showned the real limits of having a fixed bit-rate at this level.

Some of the close ups of Venus Fly Traps etc. looked very good - but wide shots of plants and forests looked poor and 'smudged'

I one shot was about the worse picture I'd ever seem on BBC HD (near the end) a camera shot rising out of 'what was supposed to be' a wheat field - it was like an abstract picture. Other shots of grasses and fils of flowers were also sud-standard.

This meant that this one programme had some of the best of recent pictures on BBC HD alongside the absolute worse (at least in my opinion of course)"

I agree with paul_geaton that it's probably not worth posting any more images on these blogs, but I couldn't resist that one!

Wheatfields:

https://www.zshare.net/image/69848228c2546062/

What a horrible mess. I couldn't believe it when I saw it happening. (That's an I frame, before anyone asks.)

But I must also agree with the first part of daveac's post, echoed strongly elsewhere by Alsone, that large parts of this programme consisted of stunning footage. (From a quick glance at other episodes, they didn't look like this.) I was able to get drawn into the compelling experience that HD can offer and that I'd almost forgotten about. Not all such sequences were time lapse, though they were all reasonably tight close ups.

One example (OK, this one is very close):

https://www.zshare.net/image/698483230c45139d/

Some sequences, such as the mosquitoes hatching, even seemed like they might have been originated with a different temporal resolution, so lifelike were the movements. And then again yes, some of the wider shots were a bit muddy, but I felt able to forgive them because of the rest of the programme.

So I agree: this showed the excellent potential of the system, but also its drawbacks as soon as there is too much change.

#97 The last paragraph was written in a hurry. I should have said that the excellent potential of the system was evident here, but it relies on the rest of the production chain being right and the encoder not being overwhelmed by too many demands given its fixed bitrate.

This comment was removed because the moderators found it broke the house rules. Explain.

This comment was removed because the moderators found it broke the house rules. Explain.