Assessing Impact of Honest Mistakes

I would like to introduce a framework for assessing the impact of “Honest Mistakes”. These are game changing decisions that officials get wrong because they are, errr, human (!). For example, wrong decisions on goals scored / disallowed, players sent off / not sent off. The need for such a framework is, hopefully, obvious. If you have ever tried to debate a contentious decision online or indeed in person (remember that?) you will know that the following is self-evident - the vast majority of supporters (including me):

• are at best partially blinded with unbreakable bias;

• will not change their opinion irrespective of the evidence presented; and

• don’t know the Laws of the Game.

So, my goal is to come up with a method of assessing Honest Mistakes and their impact.

Assessing Honest Mistakes

I will admit to being a Celtic supporter (obviously) and therefore have the same confirmation biases nearly all football supporters do in a partisan sport. Furthermore, I have never refereed nor studied the Laws of the Game. It doesn’t stop me voicing an opinion on decisions, of course. However, with tens of millions of pounds at stake through Champions League qualification, is that really good enough? Of course not, and in such circumstances my instinct is to ask an expert. Therefore, I’d like to introduce one – the Yorkshire Whistler. The Yorkshire Whistler is someone I have never personally met, but whom a mutual contact in the South Yorkshire football network (I live in Sheffield) put me in touch with. I am delighted and grateful he has agreed to adjudicate on contentious SPFL decisions. Firstly, the need for anonymity is, sadly, self-evident, given the level of vitriol some resort to when debating the Scottish game. The Yorkshire Whistler has 10 years refereeing experience after playing at semi-professional level. He is a Sheffield Wednesday supporter (an Owl AND a referee – poor guy!) with no more than a passing interest in Scottish football. In his own words: “My knowledge and following of the game north of the border is basic and somewhat limited. Other than watching the classified results on a Saturday afternoon to see if the ‘big 2’ have been upset, my only real interest is when the Celtic/Rangers derby matches come around. I always take the opportunity to watch this fixture as a complete neutral and it’s a game that rarely disappoints!” The Yorkshire Whistler will review incidents of note in the SPFL and provide me unbiased and expert assessment in the style of Ref Watch on Sky. I will relay the verdicts, unvarnished, to you.

Impact of Sendings Off

The second pillar in my framework is again provided by experts. The Royal Statistical Society, no less. The Red Card Cliche is an article that sets out to debunk the “it’s harder to play against 10 men” football myth. Which is up there with “you can only defend corners man to man” and “you have to play two up front to be successful” in the Football Myths League. I digress. This analysis assessed 1520 English Premier League matches to understand the impact of sendings off using regression techniques. I encourage you to read it but I will summarise. Sendings off reduce the number of points a team is expected to win. The average impact is summarised thus:

Team Suffering the Sending Off Average Drop in Expected Points

Home

0.86 Away

0.35

I will cover the concept of Expected Points more fully below but for now accept that the outcome of every game is 3 points for win and 1 point for a draw and 0 for a loss. The values expressed above indicate the average number of points further away from 3 each team is following the sending off. The reason a home team has more than twice the negative outcome from a sending off compared to the away team is due to the fact the home team has more starting expected points to lose. I will use these averages to assess the impact of a sending off wrongly awarded or not awarded on expected points outcomes. They are averages meaning that the impact of a sending off in the 10^th minute will differ from that in the 70^th minute. But I don’t have the data to say what that impact is by game time or state. So, imperfect as it is, we’ll take the raw average. Bear in mind the article notes “It seems safe to assume that if a weaker team (as measured by league points/ranking) has a player sent off, this is likely to have a bigger impact than when the stronger team loses a man.”. In the context of the SPFL where Celtic and The Rangers are usually significantly stronger than the opposition, this is relevant. That is, if a player is sent off (or not sent off) the impact is far greater when those clubs are involved. However, lacking specific data on this, I will use the (understated) EPL averages above.

Expected Points

You may have come across the concept of Expected Points (xPts) to describe the league points teams are expected to have achieved based on Expected Goals For and Against. Unfortunately, this is different but has the same name! Expected Points in this context is predicated on, again, you accepting the premise every match is played for 3,1,0 points as the outcome even if a cup tie. Once you accept that, what the xPts model does is place a value on every goal scored (or disallowed) in terms of how much it adds to the points you can expected to win from the game up to 3. It is best to use some specific examples:

1. Souttar’s header for Hearts against Celtic was in the 89^th minute of the game with the score 1-1. With only 1 minute plus added time to play, the xPts added to Hearts for that goal is 1.7. That is, with the scores level their xPts was already slightly more than 1 (as the home team) and that goal took it to over 2.7 xPts out of the maximum of 3 points for a win. Intuitively we all get an 89^th minute goal allowing a team to lead dramatically changes the likely outcome with so few minutes left to equalise.

2. Abada’s 16^th minute opener against Dundee United recently added 0.7 to Celtics xPts. The goal was early in the game but home teams don’t often lose games from being in front. With 74 minutes left to play, it took Celtic from slightly above 1 xPts to just over 1.7 on the quest for 3. Dundee United’s equaliser 2 minutes later added 0.6 xPts to them.

3. On 21^st August Turnbull complete his hattrick in the home match versus St Mirren. Whilst this was undoubtedly an important personal landmark for Turnbull, it did virtually nothing to change the game. The goal was scored in 84^th minute with Celtic already leading 4-0 and playing against 10 players (see above). Thus, the goal added 0 to Celtic’s xPts.

What this concept allows us to do is assess the impact of a goal wrongly disallowed or wrongly allowed in terms of the number of xPts it added / should have added had the correct decision been given. For example, if Dundee had a goal disallowed against Celtic at home in the 10^th minute and this decision was judged incorrect, assuming the score at the time was 0-0, Celtic “benefited” by 0.7 xPts from that decision. We can then cumulatively tally the xPts outcomes that have occurred due to Honest Mistakes to see potentially what impact the correct decisions made on teams’ points totals.

Summary

We now have a framework to assess Honest Mistakes and their impact:

1. using unbiased expert opinion – the Yorkshire Whistler

2. impact of sendings off – Royal Statistics Society

3. impact of goals on xPts – American Soccer Analysis

Importantly, this will not be MY opinion. What I will do is present the decisions and impacts of the major contentious moments in SPFL. Tomorrow I will present the decisions from July and August in the SPFL and then another piece covering the rest of the season to date. I will then look to update this as and when, maybe weekly the way things are going.