How the News: Killed 17 Million Numbers
Simon Pitt |
Friday 19th March
The largest number is about 45 billion. Although mathematicians suspect that there may be even larger numbers
If you're a regular reader of this site, you may have detected a slightly anti-news bias. I hate the news. Partly because it pretends to be so much more important than it is, and partly because it's either blatant scaremongering or shameless advertising. News markets itself as the big liberator of the people. The journalist is the intrepid freedom fighter. But, in countries like England, this is all rubbish. Generally, the news is just a platform for PR and adverts. Nevertheless, broadcasting news is a "public service", and consequently this gives executives free reign to spend as much money as they want on it. The BBC News Channel alone costs over Â£60 million a year.
One of the big problems with the news is that it simplifies issues into extreme, competing sides. Take numbers, for example. Years ago, there were loads of numbers: four, 35â…ž, 6,283 etc. Nowadays, in the news, there are only two numbers:
And they're even trying to get rid of one of them. Generally, journalists want to make their audiences sit up and go "shit!" rather than lie back saying "m'eh". Here are three recent headlines:
- M'eh. That's hardly anything.
- Shit! That's loads!
BBC to defy critics by claiming it 'generates Â£7.6 billion for economy'
Desperate Gordon Brown plans Â£500billion bank gamble
Piracy in Europe costs $13.7 billion
All of those numbers are meaningless. Who can imagine Â£500 billion, or understands how much $13.7 billion is? I don't even know if they mean a British Billion or an American Billion and it doesn't matter. I wouldn't react differently if they told me they meant Â£500,000,000,000,000 or whether they meant Â£500,000,000,000. A variance of Â£450,000,000,000,000 doesn't make any difference to my reaction! In all the examples, the numbers just make us go: "That's loads!"
Most numbers fall into both the "m'eh" category and the "shit!" category depending on how you phrase them. Here's an example. Earlier on in this article, I fell into this trap when I explained that the BBC News Channel costs Â£61 million a year. Â£61 MILLION! Shit! That's a lot.
However, I could have said, "the BBC News channel costs less than 1.3% of the BBC's total budget". Suddenly it's a "m'eh, that's not much".
I went to a talk recently by Michael Blastland who suggested that we should replace large numbers with faces:
BBC to defy critics by claiming it 'generates Â£ for economy'
Desperate Gordon Brown plans Â£ bank gamble
The meaning isn't really changed.
...often the difference in working which would eventually result in the answer being the right one or a random letter of the Cyrillic alphabet can be minimal.
Numbers give claims an air of authenticity. Most people in the country are either numerically illiterate or lazy. They either can't or don't interrogate any of the figures given in the news. And so they don't see that the figures are misleading or dubious. Here's another example.
Robert Ainsley, Bluff Your Way In Maths
Poor old Danny Cohen has been getting a lot of stick recently. When the BBC announced they were going to cut a few things that no one really listened to, everyone jumped to their defence screaming "why don't you cut BBC Three, that really is crap!". It's sort of like that scene in The Office where Ricky Gervais complains about people calling him fat when there is an even fatter person than him nearby: "if we're handing out insults for being fat, look at him! And he wears glasses... At least start on him and then move on."
Striking back like a good controller, Danny wrote a letter to The Guardian:
In the hours we broadcast, BBC3 now reaches more young people than any other non-terrestrial channel. We have grown by 30% with 16-34s in just two years. This means that more than 17 million people are watching BBC3 every week.
Shit! That's loads!
Actually, that really is loads. 17 million! In fact, as Ian Pollock pointed out in Ariel, that's nearly a third of the country. Nearly a third of the country is watching BBC 3 every week. Since 75% of their viewers are over 16 and under 55, pretty much everyone between 16 and 55 must be watching BBC 3!
Let's just have a think about this. 17 million! That sounds very precise. In exactly the same way as it did when the BBC said they had 17 million iPlayer views. Or when they said that spam produces 17 million tonnes of CO2. Or when 17 million tuned into Eastenders. Or when 17 million people stopped buying CDs. Or iPhone sales hit 17 million. Or when 17 million people fell into poverty due to the global crisis. Or when Microsoft sold 17 million copies of Windows XP in two months.
Now, I'm not suggesting that Danny Cohen made up the figure of 17 million just because it sounded big and specific. I'm not suggesting that any of the other examples made up the figure either. What I am saying is that in all of the cases, the specificity and size of the figure gives it an air of authority that leaves people saying "shit! That's a load" and not questioning it.
Let's return to Danny Cohen's figure. Ian Pollock attempted to work out how Danny calculated this by checking the BARB figures for BBC 3. (Remember: BARB just takes a small sample and extrapolates up.)
The average hourly audience for the week was 336,000. If I multiply that figure by the number of days in the week (seven) and the number of transmission hours per days (also seven) I get a figure of 16.5 million. [...] Surely the apparent size of the weekly audience should be reduced by taking into account people who watch for more than one hour, and on more than one day?
The figures on the BARB website also contain the rather unusual statistic that, on average, each of the 17 million BBC 3 viewers watched it for 20 minutes a week. If this were true, I'd imagine Danny Cohen would have to ask himself some very serious questions. Like why none of his viewers ever watches a programme all the way through.
Ian Pollock, Ariel, 16/03/10
However, it's the word average that's misleading here. This reminds me of the statistic that the average person has fewer than two legs. While it may be mathematically sound, in reality it's nonsense.
Unusually, Danny Cohen replied (slightly irritated) to the letter within a few hours:
To explain concisely, our weekly reach figure of 17 million comes from BARB, the audience measurement tool used as standard across the industry. A quick look at the BARB website clearly shows that a Channel's Reach is calculated as the amount of people in the UK who watch a Channel across a week. They are not double-counted, and in BBC Three's case this figure has now reached 17 million different people a week
Like the politician that he basically is, his response was quick, emotional and completely failed to add any additional information to the debate. It's the equivalent of pointing at BARB and saying "he told me to do it".
Danny Cohen, Ariel Online, 16/03/10
Also, for people that missed all of the passive aggression in the above message, I've highlighted it here:
To explain concisely, our weekly reach figure of 17 million comes from BARB, the audience measurement tool used as standard across the industry. A quick look at the BARB website clearly shows that a Channel's Reach is calculated as the amount of people in the UK who watch a Channel across a week. They are not double-counted, and in BBC Three's case this figure has now reached 17 million different people a week.
That gets a passive aggression score of 6 million on my made up passive aggressive scale. 6 million! Shit! That's a lot!
Cohen genuinely seems to believe his own statistic here. (I say this because he puts "not" and "different" in italics, which makes it look like he really means it). Surely, though, he can't be thinking about what the numbers actually say. Does he really believe 17 million people each watched BBC 3 for 20 minutes last week?
Let's have a look at how BARB actually calculates the figures:
Viewing estimates are obtained from a panel of television owning private homes representing the viewing behaviour of the 26 million TV households within the UK. The panel is selected to be representative of each ITV and BBC region, with pre-determined sample sizes. Each home represents, on average, about 5,000 of the UK population.
336,000 watched a typical programme on BBC 3. If we divide that by 5,000 (the number of people each box represents) we get 67.2 So, in real terms, 67 people watched BBC 3.
BARB also says there are 26 million TV households in the UK. However, the British population is 60 million. So, if they record 67 people viewing, on average that could mean as few as 20 TVs in their sample are actually watching BBC 3. "M'eh," you might say, "that's hardly any".
Here's another way of looking at it: 17 million people watched BBC 3 for an average of 20 minutes over the week. This means that 5.6 million hours of BBC 3 were watched over the week. So, 800,000 hours of it were watched each day. BBC 3 broadcasts 7 hours a day, so 110,000 people watched each hour. In terms of BARB, 20 people were watching at any given hour. If there were three people in each household watching the same TV (the national average), that could mean as few as 6 TVs were tuned into BBC 3 at any given time. That really is hardly any.
As Charlie Brooker notes, if one person nips out for a quick wee, 5,000 people stop watching. However, with a sample this low, if one family decides it's time to call it a night, one in 20 of your audience have turned off.
Since there is a three to one ratio of people to BARB boxes, I'd be very cautious before saying, "they are not double counted". And I almost certainly wouldn't italicise "not".
While the size of BARB's survey may be fine for large channels, it isn't for smaller channels. A sample size of 5000 means that they have a margin of error of 1.4%. (The reason for this involves some quite complex maths, which you can read here if you're interested. Suffice it to say, it's to do with statistics and stuff ). This isn't too bad for BBC 1 with its 82.8% share of the audience. However, it's a different story for BBC 3 with its 1.2% share. While this doesn't mean that -0.2% of the population are watching BBC 3, it does mean that you have to be a bit cautious with your figures.
The bottom line is, Danny Cohen's claim of 17 million viewers could come from an average of twenty televisions watching BBC 3,
Of course, you miss all this when you write "17 Million watch BBC 3". All that says is "Shit! That's a lot of people watching BBC 3!" He's used the numbers to prove a point.
This is what most journalists do. Generally, they start writing their article knowing what they want to say. They've spoken to some people, or got a feeling of what the current atmosphere is like and then hunt for evidence to support that view. When I started writing this article, I intended to show that lots of money in the BBC is spent on news. The impression around the BBC is that news departments are much richer and the BBC talks a lot about hoe good its new is. Consequently, I found figures that supported this.
The sequel is never as good as the original. But why? Is it because the director, the actors and the cameraman suddenly get bad at their jobs? No, it's because most films are mediocre, and the great Oscar winning film is the exception. So, the sequel is more likely to be mediocre than it is to be another great movie. This is called regression to the mean.
It's amazing how powerful these beliefs can be. Michael Blastland tells the story of a journalist looking at the figures for alcohol consumption during the recession. He saw that alcohol consumption had gone up.
"Oh," he thought, "that follows, since people have lost their jobs and have gone out to drown their sorrows."
Then he realised, this was before the recession had started. After this, alcohol consumption had actually gone down.
"Oh," he thought, "well, that makes sense, since during the recession people have less money and so can't afford to go out."
Then he realised the figures were for 2007.
In each case, he was able to justify the figures by fitting them into his existing worldview. The point is, he could have justified it either way, and then used it as a headline to support his argument. In many ways, this is the fault of arts degrees teaching students how to structure an argument.
Most journalists are arts students, and use the skills they learnt at university in their articles. Rather than showing what really happened, they build arguments to support whatever it is they want to believe.
What journalists should be doing is investigating the events, and then writing an article based on what actually happened, rather than starting with a point of view and then finding figures to support it.
Statistics is a highly complicated issue, and increases or decreases are usually because of the regression to the mean (that is, the figures returning to normal after an anomaly). This, however, won't stop journalists from publishing stories saying that teenager pregnancy has gone up or down.
This number blindness it why the Conservatives confused an increase of 5.4% with an increase of 54%.
"It makes no difference at all to the conclusions of a wide-ranging report"
Exactly, what sort of report is it, if a variance of 48.6% doesn't make any difference.
If I'd said "A decimal point was left out in a calculation. It makes no difference at all to the conclusions of a wide-ranging report!" in an e-mail to a colleague which was hacked and stuck on some Russian server, it would have been branded "order-of-magnitude-gate" and I'd be on the front page of every bloody national newspaper.
If it makes no difference, why did they include it? They included it for the "shit!" factor. They saw the figure they wanted and thought, "Shit! That's a lot!" When in fact, m'eh, it was hardly anything.
Professor Francis T. Bargle, Quoted in KTAB News