The Advertising Research Foundation, an influential ad research body founded by the ANA and 4As, has announced it is launching a study into the reliability and validity of attention measurement tools. At a time when agencies are increasingly incorporating attention metrics into their media buying, the ARF’s study aims to lay out the scientific grounding behind these metrics.
“Recent years have seen increasing interest in direct measures of cognitive and emotional response to advertising,” said Scott McDonald, CEO and president of the ARF. “As a result, a number of new services have entered the marketplace with different approaches to the measurement of attention and/or emotional responses to ads […] but, we still don’t know enough about the reliability and validity of these measures and their rightful application to advertising and media evaluation.”
The study will focus on three distinct areas:
- Differing definitions of attention and emotion, their relationship to direct measures of neurometric response, and their validity as tools for creative evaluation and as predictors of market performance of ads or content
- Validity, reliability and replicability of synthetic measures of attention, based on AI and machine learning approaches
- Validity of attention measures (conventional or AI-based) for comparative evaluation of media as vehicles for ad placements
Due diligence
The current interest in attention metrics on the one hand looks like a long overdue refocusing on what really matters in media measurement.
It’s no secret that whenever an advertiser buys an impression, there’s every chance it might end up completely ignored. Digital display ads may sit completely unnoticed at the side of the screen, a radio ad might play while the listener is out of the room, and a TV ad might roll while the viewer is busy on their phone.
Media buyers acknowledge this is part of the cost of advertising – you can’t force people to concentrate on your ads. But attention arguably matters more than anything else in advertising, and attention measurement aims to help buyers allocate spend to channels which receive the most attention, cutting out a lot of waste.
There are however a lot of challenges. Firstly, ‘attention’ isn’t really defined in advertising – and pinning down a definition is difficult. If a user is looking directly at a TV ad, and is able to hear the audio, you might assume it has their attention. But perhaps their mind is completely elsewhere, processing whichever show they’ve just been watching. On the flipside, if someone has gone through to the kitchen to boil the kettle during their ad break, it might be assumed those ads don’t have their attention. But they might hear a snippet of a brand’s jingle, putting that brand in the consumer’s mind.
Hence, the ARF’s first challenge of laying out a definition for attention. Whichever definition the ARF chooses, the key will be to ensure it’s useful as a predictor of ad performance or creative effectiveness – otherwise there’s not much point in measuring it.
The second challenge is evaluating the various different methods of attention. Whatever the exact definition, attention is undoubtedly a neurological phenomenon: some spark of awareness in a person’s brain. That makes it impossible to measure directly (at least with our current batch of ad tech – perhaps Elon Musk’s Neuralink should consider spinning up an ad measurement business?).
Attention vendors have come up with a variety of proxies, from eyeball tracking, to lab-based recall testing, to essentially just viewability measurement with higher standards. So the ARF, once it has its definition of attention, will test how well these various techniques really do measure it.
To do this, the ARF will use lab-based neurological techniques to measure how good a set of ads and programmes are at capturing attention. It will then run that same media through attention measurement vendors’ tools, and see which ones are best at replicating the lab-based results.
The overall aim is to provide a solid scientific foundation for attention measurement’s use in media – which is particularly important given the large role they’re being touted for.
“[The excitement around attention measurement] has caused some to push for incorporating these measures into next-generation currencies for media buying,” said the ARF’s CEO Scott MacDonald. “It is the ARF’s view that these discussions of attention-based currencies are premature in the absence of better information on the validity, reliability and predictive power of these measures. That’s what this study seeks to address.”
