In his first major speech of 2023, British Prime Minister Rishi Sunak talked about how he wanted all British schoolchildren to learn maths up until the age of 18. As many commentators said, it was a slightly odd topic to tackle when large parts of the country were on strike and there is widespread concern over the cost of living crisis and the effects of rising inflation. I do have sympathy for Sunak’s point, namely the importance of financial literacy when it comes to everyday life. However, it is not just everyday life where numeracy is important. It is increasingly important when it comes to advertising, especially given the greater focus on a more ‘scientific’ approach to media spend.
It is a topic I am particularly interested in given I have worked with JC Decaux on how to speak the language of the CFO and the Board, and it is now a subject where I run a training course for those in advertising (obvious plug there). Yet, I was reminded of it again when I read Les Binet’s piece on Meta and Google moving away from digital attribution as a way of measuring advertising effectiveness and more towards econometric modelling/case studies. Research from Google has shown that digital attribution models are poor – to put it mildly – at estimating the incremental increase from an advert.
As Binet put it, if one million people see your advert, and it leads to one thousand extra purchases, how many extra people have bought your product due to the advert? It may be anywhere between zero and one thousand (it could even be less than zero if people thought the advert was truly that bad). Attribution models can’t get close to the answer.
It is a point I have been making for several years. The example I use is the pint of Guinness. Guinness has done some wonderful work with its adverts, as well as its sponsorship. Yet, the way attribution models often work is to say that if the last thing you saw before you bought the pint was a coaster advertising Guinness, the coaster triggered the sale and the previous brand advertising work gets ignored. That doesn’t make sense and it is a point Facebook highlighted in their research over the underestimation of the effect of brand building media.
However, there is a bigger point here. Having worked as an Equities analyst for c 20+ years, Binet’s comments are further confirmation of the classic phrase any analyst knows when it comes to models which is “Garbage in, garbage out.” There is a tendency to treat models’ answers as gospel, providing automatic validation to an argument.
That is not the case. The biggest fundamental error when it comes to models is assuming that the answer it generates is the most important part of the process. It is not. The most important part is the assumptions behind it. That is far more important than the output it generates.
These assumptions reflect the biases of those who produce the models, even if they are subconscious. This is why for stocks, you will get as many estimates as analysts who cover the stocks because each one will take their own view as to the future. For me, perhaps the most revealing angle of the (in)famous story of how several years ago Adidas in Latin America realised that their reliance on paid search was significantly wrong was that the company was using attribution models provided by the major tech advertising players, all of whom had a vested interest in the outcome.
This of course has real-world implications. I estimated last year that, for U.K. advertising alone, the lifetime shareholder value lost due to ineffective advertising was probably around £120 billion (and yes, I will be making my own assumptions there). Multiply that globally, that figure rapidly goes above $2 trillion. These are not small amounts.
What can be done? Let’s go back to the start. If the focus moving forwards is going to be on more econometric modelling and accurate case studies, then the points raised still very much apply – namely the answers do not count, it is the inputs and assumptions behind them. And for those in advertising, it is going to be critical to become more comfortable when it comes to numerical (and financial) literacy and in being able to at least look at the data and numbers, and to spot the obvious issues. If you do not, then you run the risk of missing some very big problems. Back to the numbers.
As usual, this is not investment advice.
