Lenskold Article Series
by Jim Lenskold
A Well-Oiled Machine: How Measurements Improve the Marketing Process
Many of us are looking for the perfect way to measure advertising’s effectiveness and specific results. Unfortunately, no Holy Grail exists. The marketing environment is complex, and a mix of different research initiatives is required to truly tell us how well we are performing and where improvements are necessary. Measurements are a critical part of the marketing process, but in most companies, marketing falls far short of its potential by not tapping into the proven or emerging methodologies available.
What is it that makes marketing measurements so challenging? Well, in order to demonstrate the impact of your marketing programs, you need to measure the customer and sales activity that follows those initiatives and compare it with the activity that would have happened without them. The difference is the incremental impact of your efforts.
Sounds simple, perhaps, but how do you know what would have happened without your marketing, given that your company and competitors have other sales and marketing initiatives running at the same time?
I met a CMO who explained to me how he tracks sales activity prior to the marketing campaign and compares it with sales activity during and after the campaign. When there is a positive increase in sales, he attributes the lift to marketing. When sales decrease, he attributes it to external factors in the market. In reality, the sales levels are constantly fluctuating, so the measurements are far from valid regardless.
As amusing as his approach sounds, this was actually how the company made its decisions—and I’ve heard another dozen people repeat the same story.
One of the best measurement tools I’ve found for isolating the impact of marketing initiatives is market testing, or experimental design: using test and control groups. The control group (from which a portion of the marketing is withheld) represents what would have happened in the absence of that marketing component. The approach can be used to measure the effectiveness of an entire campaign, the incremental value of one marketing channel within a campaign mix, or the impact of increasing ad spending over the existing level.
Marketing mix modeling can also be used to establish the correlations between marketing and sales activity. The analysis shows which portion of previous sales activity can be attributed to each of the various marketing initiatives. The portion of activity that does not correlate to any marketing is considered the base activity that would be present without any marketing. This methodology does not require a control group and can be run on historical data. Its key limitations, however, include the inability to measure the impact of smaller initiatives and that it provides limited diagnostic analysis to understand exactly what is driving the sales activity.
I’ve been a major skeptic when it comes to using quantitative survey research as a primary source for assessing the impact of marketing on actual sales outcomes. Survey research is an excellent source of understanding customer perceptions and attitudes, but it is hard to trust self-reported purchase behaviors and the correlation to marketing activity. Recently, however, I’ve been converted after seeing an innovative measurement methodology that integrates modeling with quantitative research.
Establishing Links
The use of survey research by a major consumer products company probably wouldn’t surprise you, but what if I told you that a leading defense and aerospace supplier uses survey research to measure ad effectiveness? Raytheon sells primarily to the Department of Defense and other government agencies, but it launched a new corporate ad campaign in 2003 and used just such a methodology—one created by the Brand and Analytics Group of Phoenix Marketing International (formerly Cambridge Brand Analytics)—to measure its ad performance.
Phoenix’s unique methodology consists of conducting surveys with decision-makers in Raytheon’s industry, where impact on purchase consideration is linked back to specific ads by Raytheon and its competitors.
In other industries, the Phoenix research will establish a link directly to sales or customer response activities. In Raytheon’s case, the link to purchase consideration is considered the best match to the corporate communications objectives that are centered on helping their customers succeed in their “warfighters” mission.
Lucy Flynn, director of brand and marketing communications at Raytheon, relies on the Phoenix analysis for quantitative results when reporting to the executive team. “Our analysis provides the rationale for shifting corporate positioning to align with the priorities of our clients,” she says, “and the hard data allows us to make solid strategic decisions with confidence.”
By gaining insight into the performance of each specific ad extending across all forms of media, the communications team at Raytheon has been able to improve its effectiveness by determining the positioning and messaging that best differentiates the organization’s ability to support the mission of its customers—and ultimately the mission of the men and women in uniform. That has helped the company refine campaign strategies, reallocate dollars into more effective ad executions and create new executions.
Raytheon also gets a read on its public relations impact. By incorporating PR articles into the Phoenix research, the company can confirm a respondent’s prior exposure to the articles and derive its impact on image and reputation, which helps to guide the PR strategy.
“If we aren’t getting the right message to our audience, we know it’s better to adjust our plan than to continue with the current ad spending,” says Flynn. And, as Flynn reminds us, “the worst way to measure is to not measure at all.”
The Innovative Methodology
Using a combination of Internet, mail and telephone research, Phoenix’s research participants are presented with actual ads and marketing communications for all major competitors in the category. That eliminates questions as to the validity of ad recall based strictly on verbal or written descriptions, and large sample sizes provide the necessary statistical validity.
Add sophisticated modeling to all that and you get a predictive and diagnostic tool for marketing measurement. Whether the research is modeled against a response step in the marketing process, purchase intentions, self-reported sales activity or even actual sales activity, the analysis establishes correlations to the survey data that have proven to be quite accurate.
The strategic value of this methodology is really in the diagnostics. The analysis identifies the performance of each specific ad, as well as competitors’ ads, in terms of breakthrough and recall, message relevance and persuasiveness—all critical components of effectively influencing customer behaviors. Marketers get a clear picture of where barriers and opportunities exist in terms of media plans, message and creative execution.
In the end, however, effective marketing performance management requires a plan that integrates multiple methodologies. Running controlled market tests in conjunction with the Phoenix research will deliver precise incremental sales results backed with clear analysis about where campaigns are effective at influencing awareness, perceptions, competitive differentiation, intentions and actions, and where campaign modifications can make the greatest impact.
Company executives are increasingly putting pressure on CMOs to demonstrate their contribution to the bottom line. Marketers need to understand the measurement tools that are at their disposal and put them to use.
Marketing measurements require a commitment of budget and staff resources, adequate time to complete measurements prior to major strategic decisions that are dependent on the insight, and a culture of using continuous analytics and insight as a competitive advantage. The path to better measurements seems clouded by many challenges, but once you move forward, the clarity from increased insight and knowledge will prove to be quite rewarding.
Copyright © 2005, CXO Media Inc., reprinted by permission from CMO Magazine.