The problem with trends that are sort-of predictable but not completely

Photo by Gurdain Bharj on Unsplash

Behavioural science has long emphasised the difficulty humans have with probability, with a range of biases or heuristics leading people to under- or overestimate the chances of something happening. But what I've been noticing lately is the particular difficulty people have in making sense of events that have a deterministic and a stochastic (random) element.

As an Arsenal fan, the Champions League semi-final against the French team PSG was a very good example of this. Arsenal lost the two-leg tie, and the pundit reaction was that this demonstrated that the PSG team was more talented and its tactics better. In other words, the pundits treated the outcome as the result of a purely deterministic process.

To define what I mean by deterministic trend with a stochastic element, a deterministic trend is one where the outcome is the result of a predictable process. For example, if an outcome, y, was always the result of multiplying x by 3 and adding 4, this would deterministic. If in contrast the outcome, y, depended on what number came up if you rolled a dice, this would be stochastic.

Combining these two creates a deterministic process with a stochastic element, ie if y is the result of multiplying x by 3, adding 4 and then adding the number that came up when you rolled a dice. In this situation the outcome is mostly but not completely predictable.

Sport is a prime example of events that are the result of deterministic trends with stochastic elements. Stronger players and teams normally win, but not always because there is a substantial dollop of luck involved, though the balance between deterministic and random varies by sport. Track athletics has a relatively small element of luck (Bolt was the fastest and mostly, though not always, won). Golf seems to have a very large element of randomness; even at his best, Tiger Woods didn't win most or even the majority of the tournaments he entered. Team sports, and particularly football, which involves a lot of group play, also seems to have a large element of luck, though probably less than golf (these are all informed guesses; I'm not sure if anyone has actually studied or ranked different sports in this way).

Going back to the Arsenal example, the statistics for the matches with PSG indicate that Arsenal had more shots than PSG, more shots on target and a higher expected number of goals (a measure of how likely each shot is to score). The data suggests therefore that Arsenal was the stronger team, ie the deterministic element indicated that on average they should win, but the random element meant that they didn't.

My hypothesis is that people don't like the ambiguity that is inherent in a deterministic trend with a stochastic element, and so either bracket it with purely deterministic events or purely random ones. Football is an example where it is easier, and where people want it, to be purely deterministic. In contrast, people want to think storms are purely chance (acts of God).

The significance of this is that how you think about the causes of an event affects how you respond to it. In Arsenal's case, the data indicates that they shouldn't necessarily change their tactics or team because on average they would win, in direct contrast to the conclusion of many football pundits. In terms of the environment, the fact that it's simpler to think of adverse weather events as random makes it easier to believe we have no influence on them and therefore don't need to change our behaviour. Unfortunately, yet another barrier to trying to tackle the climate crisis.