What are some of the common pitfalls and biases in ad testing and how can you avoid them?

Ad testing should not be conducted simply to check a box on the list of things to do before running a new campaign. It needs to be thoughtful and well-designed in order to gain the highest quality insights possible. This begins with finding the right audience sample. Test participants should represent your target audience well and be able to provide a diverse range of opinions and views. This is not the time for convenience sampling. The more prep you put into setting up your ad test, the more useful and accurate your result will be.

As creatives, it can be difficult to remove our own preferences from ad testing. When you’ve invested time and energy into developing a campaign, it’s natural to develop a vision for how it should be executed. However, the point of ad testing is to ensure the campaign includes the most effective creative assets with the most impactful delivery method. Thus, ad tests must be structured to exclude any of our own preferences or biases. Because even the most innovative creative concepts will fail if they are not right for your target audience.

Testing each element of your campaign is crucial to mitigating the halo effect. For example, an ad for a pet food company that features a photo of puppies is likely to receive a positive overall response from viewers. However, if the headline, CTA, and body copy fail to support the ad’s conversion goals, the ad is a failure despite its well-liked visuals.

Furthermore, it's crucial to account for the impact of external factors on ad testing results. Factors such as current events, cultural context, and market trends can all influence the way an ad is perceived and remembered. By staying informed about these external influences and considering them during the testing process, we can make better-informed decisions about the performance of our ads.

Another aspect that is often neglected is the importance of evaluating the emotional impact of advertisements on the audience. Emotions can significantly influence memory recall and skew testing results. By incorporating measures that assess emotional responses, such as facial expression analysis or sentiment analysis, we can gain a more comprehensive understanding of the true effectiveness of our ads.

For instance, advertisers should consider incorporating a range of qualitative and quantitative data collection methods, such as in-depth interviews, focus groups, and online analytics. By gathering information through multiple channels, we can better contextualize our findings and minimize the risk of biases, including the recency effect.

One of the primary issues is that professionals tend to focus on randomizing the order and presentation of ads as the main solution to counter the recency effect. While this is indeed an essential step, it may not be sufficient on its own. In order to truly mitigate the impact of this bias, we must adopt a more holistic approach that encompasses a variety of testing methodologies.

Your test participants should feel comfortable sharing their honest opinions of an ad, even if the topic is controversial. You can encourage honest, thoughtful responses by allowing participants to answer individually, rather than in a group. You can also set ground rules prior to a focus group that ensures each participant respects other group members’ right to form and share their own opinions on an ad, without fear of ridicule or censure.

While it is true that new and innovative ad formats can attract attention, it is also crucial to maintain a certain level of familiarity to avoid alienating consumers. Some professionals might focus too much on novelty, potentially compromising brand recognition and consumer trust. By blending innovation with familiar elements, advertisers can capture consumer attention without sacrificing the core aspects of their brand identity.

Testing ads over time and across different channels and platforms is necessary to avoid the novelty effect. While this is true, some professionals might underestimate the importance of crafting a cohesive and consistent brand message throughout these different channels. Ensuring that your ads convey a consistent brand identity, regardless of the platform or format, can help build trust and familiarity among consumers, mitigating the potential negative impact of the novelty effect.

It is crucial to recognize that the novelty effect can be short-lived, as consumers can quickly become desensitized to new ad formats or techniques. Therefore, it is essential not only to test ads in various contexts but also to continuously innovate and adapt advertising strategies to maintain consumer interest.

Testing finished spots is one thing. But the exercise of testing animatics or boardamatics is ridiculous pseudo science. “Here is a thing that’s not the ad, but can you imagine if this was a finished ad, if you would like it?” It’s a waste of time and money. Focus group participants can’t make the creative leap to judge the work at that stage. The list of great campaigns that did test (or would have tested) terribly is long. Save your 💰.

By combining a variety of data collection methods, assessing emotional impact, and considering external factors, advertisers can gain a more accurate understanding of their ads' effectiveness and make data-driven decisions that lead to better results.