Peddling certainty

Some of you may remember the ‘Dodgy Dossier’ produced by the British government prior to the invasion of Iraq. This laid out the government’s evidence that Iraq had weapons of mass destruction. I gave it a read at the time. It was most unconvincing, consisting of numerous statements along the lines that ‘Iraq could have this’, ‘It might have that’, and so on. The Executive Summary, by contrast, was very different. All the caveats had disappeared, to be replaced by an almost 100% certainty that Iraq was knee deep in deadly weapons. The next day when the media reported the dossier, they just reported the Executive Summary. The ‘coulds’, ‘mights’ and ‘possiblys’ in the main text were nowhere to be seen. I knew then that something fishy was going on.

My career in military intelligence was relatively short, but one thing I learnt from it is that intelligence analysts tend towards caution in their assessments. They don’t want to be proven wrong, and so lace their reports with words like ‘possibly’ and ‘probably’. This is especially true when providing analysis rather than reporting pure data. Whenever you read something which claims 100% certainty, you should be immediately suspicious.

As regular readers will know, I am of the opinion that there are good reasons to suspect Russian involvement in the poisoning of Sergei and Yuliia Skripal in Salisbury. The behaviour of the alleged poisoners, Petrov and Boshirov, was, to say the least, odd, and their interview on RT utterly unconvincing. Which is where we come on to the organization Bellingcat, which claims to have identified Boshirov’s real identity.

The first thing to say about Bellingcat’s research is that it is ingenious. Unfortunately, their claims about Petrov’s and Boshirov’s passport applications are unverifiable, as we don’t have access to the originals, and so can’t check want Bellingcat is saying. Still, it’s undoubtedly interesting. Likewise, the organization’s latest investigation, which claims to identify Boshirov as a colonel in the GRU, named Chepiga, deserves to be added to the file as something worth further investigation. I absolutely don’t dismiss this stuff out of hand.

But there’s something which annoys me about Bellingcat nonetheless. It’s the certainty with which it makes its claims, and then the certainty with which those claims are reported by the press. The intitial Bellingcat report on Petrov’s and Boshirov’s alleged passport application stated that the organization’s investigation,

Has confirmed through uncovered passport data that the two Russian nationals identified by UK authorities as prime suspects in the Novichok poisonings on British soil are linked to Russian security services.

Note the word ‘confirmed’. This is incorrect. What the investigation does is provide information to suggest a link between the suspects and Russian intelligence. It doesn’t prove it. If you find the evidence convincing, I’d allow you to say ‘probable’, or to put in some sentence like ‘we assess with a high degree of confidence that,’ or whatever. If you did that, then you’d be writing like a proper intelligence analyst, pointing out to the reader that this is an assessment not a fact, and that there is some uncertainty. But Bellingcat doesn’t do that. It’s ‘confirmed.’

The second Bellingcat report on the subject makes the same error, beginning with the headline ‘Skripal Suspects Confirmed as GRU Operatives’. Note again the word ‘confirmed’ – no doubt there at all. The same happens again with Bellingcat’s latest. This starts with the headline ‘Skripal Suspect Boshirov Identified as GRU Colonel Anatoly Chepiga.’ In this case, the word ‘identified’ is categorical. Bellingcat is claiming that this is definitely true, not raising the possibility that it might be so. As the article which follows says,

Bellingcat and its investigative partner The Insider – Russia have established conclusively the identity of one of the suspects in the poisoning of Sergey and Yulia Skripal. … Bellingcat was able to conclude with certainty that the person identified by UK authorities as ‘Ruslan Boshirov’ is in fact Colonel Anatoliy Vladimirovich Chepiga.

But have they established this ‘conclusively’? What they’ve actually done is tracked down a Russian military officer, Colonel Chepiga, and provided some evidence to suggest that he might be in the GRU. They’ve then provided some photos to show that Boshirov looks a bit like Chepiga. But that is absolutely not ‘conclusive’ proof that Boshirov is Chepiga. Again, Bellingcat  is making categorical claims that its evidence doesn’t support. Boshirov may indeed be Chepiga – I don’t rule that out – but it is wrong to say, as Bellingcat does, that on the basis of its evidence one can ‘conclude with certainty’ that the two are the same.

This is especially true as Bellingcat  made no forensic effort to compare the photos of Boshirov and Chepiga. I’m no fan of Craig Murray, who it seems to me has popped far too many red pills but, as you can read here, he at least bothered to run some facial recognition software, and got results which suggest that the two aren’t the same guys at all. I’m not at all qualified to comment on forensic matters. Perhaps further investigation will reveal that the two faces are in fact identical. Or maybe they won’t. I merely raise the issue to say that if you want to make the sort of identification Bellingcat makes you have to do a bit more work before coming out with statements about concluding ‘with certainty’. It’s dishonest reporting.

And it matters. The reason it matters is that the press doesn’t tend to go for nuance. If you make claims of certainty, the press will run with them and repeat them as if they are true. And this is what we saw in the British press following the Bellingcat story:

Observe how all these headlines treat Bellingcat’s claim as proven fact. Now, perhaps further investigation will prove Bellingcat to have been right. I consider it perfectly possible. But as a former intelligence officer, the claims to absolute certainty bug me. Proper reporting requires analysts to make the reader aware of all the underlying assumptions and uncertainties, as well as the additional information which is required to confirm the hypothesis being advanced. Bellingcat doesn’t do any of that. It peddles certainties. And that sort of thing has gotten us into all sorts of trouble in the past. Reader beware.