Taste of Marketing Measurement

September 21, 2012 By Rick Shea Leave a Comment

Last month I attended ‘Taste of the Danforth‘, my favourite summer event in Toronto. Now in its 19th year, this weekend long event along a 3km stretch of Danforth Avenue typically attracts around 1.3 million people over the weekend. I live nearby, attend almost every year and generally eat too much while I’m there.

I planned to meet a friend on the Danforth Friday evening but when the forecast called for rain, we called off our plans. The forecast turned out to be wrong and as I sat at home on a fairly dry Friday night, I wondered just how many other people didn’t go to Taste of the Danforth due to the threat of rain and how much that might have reduced sales for each restaurant participating in the event.

Marketing Effectiveness

I also thought about how event organizers seemed to have been very effective at creating awareness for the event. I saw a lot of local media coverage and I noticed how Taste of the Danforth was listed in every what’s-going-on-in-Toronto/Ontario listing I saw in print and on-line.

Marketing effectiveness measurement tends to focus on whether specific program objectives were achieved, such as attracting and keeping profitable customers and creating value for the business. Yet, as I was reminded by sitting at home when I should have been out eating too much, marketers can do everything right and still end up with bad results due to factors outside of their control, such as bad weather (or a bad forecast).

Event Objectives

I’m guessing the event’s main marketing objectives are to generate awareness, attendance and trial of both the area and of individual restaurants, and to also increase post-event traffic and revenue for local merchants.

While I sat home that night, I watched a segment of a local newscast about Taste of the Danforth in which a restaurant owner told the interviewer that this event generally makes his year. I imagine that some restaurants might aim to sell enough souvlaki in three days to possibly pay for a renovation or to simply make enough that weekend to stay in business for another year.

Objectives like this point the way towards some of the metrics I would include on a scorecard to measure the effectiveness of the marketing, but there are other factors to consider.

Those Pesky External Factors

For this event, the number one external factor outside of marketing’s control that can impact its success is the weather. I imagine that the event organizers must make the appropriate sacrificial offering (lamb would seem appropriate) to the Greek god (Zeus?) most responsible for weather. A hard rain could severely reduce attendance for an evening or whole weekend.

The Measurement Dilemma

There are three general approaches to choose from regarding external factors.

1. Ignore Them

This option will always be as tempting as all the delicious foods one finds at Taste of the Danforth. I can’t imagine how I’d ever determine how many people didn’t go to Taste the Danforth that Friday night because they thought it might rain, or how many sticks of souvlaki didn’t get sold as a result.

2. Model Them

This option is worth considering when you have a lot of data. If the organizers have 19 years worth of data that would correlate daily attendance with weather forecasts and actual rainfall, then that would be a start. Still, for most marketers, the costs of sophisticated models and analytics can quickly become too high relative to the size of the marketing expense they’re meant to measure.

3. Track Them

I think it is well worth tracking any external factor that could impact results, such as weather, competitive activity and labour disruptions. I arbitrarily score each external factor on a five-point scale, where the low end of the numerical scale corresponds to “very negative” and moves up through “somewhat negative”, “no impact”, “somewhat positive” and “very positive”.

I keep this very unscientific scoring of external factors separate from the rest of the scoring I do on the factors that seem to be within marketing’s control and on the results that can reasonably be attributed to the marketing. I don’t muddy the waters by including the external factors in the calculation of the overall score, but I do note them and score the severity of those factors.

The value of tracking external factors comes when you analyze a group of marketing programs, such as past years of Taste of the Danforth. Imagine looking back and seeing great year to year variations in attendance and not knowing in which years it rained all weekend, or there was a transit strike.

Without tracking external factors, it would be easy to come to the wrong conclusions about the effectiveness of specific marketing programs. It would be hard to decide whether specific programs should be repeated or changed and also to learn which tactics were the most and least effective.

Always track those factors outside of your control and the degree to which they may have helped or hindered your results. That will at least give you a taste of what else might have been going on at the time that may have impacted your results.

Filed Under: Approach, Metrics Tagged With: Effectiveness, External Factors, Management, marketing, Measure, measurement, Metrics

Down In The Alley

July 19, 2012 By Rick Shea Leave a Comment

A few years ago, the couple who live across the alley behind my house decided to host a small gathering on their driveway for those of us living nearby on either side of the alley. A few homes had sold recently and they decided that this would be a good way to welcome the new neighbours and help everyone get to know each other a little better. It was fun, and this annual tradition lives on 4 or 5 years later.

This year’s edition of the alley event happened last Saturday afternoon. One of the things I enjoy about these neighbourhood gatherings is having the time to learn about each other in a relaxed setting.

On Saturday, I was having a good chat with my exceedingly well-named neighbour, Rick. He asked about the nature of my work and so I gave him a bit of an overview about how I measure marketing effectiveness. That quickly led to him asking me this question:

“What did you think of General Motors’ announcement that they were pulling their Facebook ads?”

My first reaction was that I supposed GM felt their ad spending on Facebook wasn’t working.  As we continued, our conversation shifted to the timing of GM’s announcement, mere days before Facebook’s highly anticipated initial public offering. We concluded that something must have gone wrong in their relationship with Facebook for GM to announce their decision at such a sensitive time.

After Rick, who is an actor, entertained the local kids by juggling while walking on stilts, I went home, considered our conversation, did some research and organized my thoughts.

What I Found & My Thoughts – #1

In case you missed it, GM announced they would eliminate $10 million of advertising spending on Facebook. This still leaves another $30 million which they spend on their Facebook marketing initiatives, although I don’t believe any of that spending becomes Facebook revenue.

Clearly, GM thinks there’s an audience on Facebook worth engaging through marketing, but not so much for advertising, at least not yet.

What I Found & My Thoughts – #2

The $10 million is a drop in the bucket compared to GM’s 2011 total US ad spending of $1.8 billion ($3 billion globally), and Facebook’s 2011 revenue total of $3.7 billion, most of which was for advertising.

Smart marketers who spend $3 billion annually on advertising almost certainly also measure the effectiveness of that spending pretty rigorously. It is a natural part of the process to question, evaluate and optimize all parts of that spend on an ongoing basis, and the Facebook ad spend would be subject to that scrutiny.

What I Found & My Thoughts – #3

It has recently been reported that Facebook and GM are back in talks to renew GM’s advertising and that GM is asking Facebook for more data to bolster their measurement efforts.

Perhaps the problem was not so much that GM’s Facebook advertising didn’t work, but rather that GM couldn’t prove whether or to what degree it did, or didn’t. I also wonder whether GM’s pre-IPO announcement was a negotiating tactic to get the data they want from Facebook.

What I Found & My Thoughts – #4

I noticed that following GM’s announcement, their rival Ford tweeted something to the effect that Facebook ads are effective when used properly. Let’s assume the people at Ford are also pretty smart and measure rigorously, too. By implying they know their ads are effective, their tweet also implies they are better than GM at measuring Facebook ad success, and thereby raises some related questions:

  • Does Ford use Facebook ads differently and in a way that makes measurement easier?
  • Is Ford better than GM at setting measurable objectives for each ad?
  • Does Ford already get better Facebook data than GM?
  • Was Ford’s tweet was just an attempt to position themselves as smarter than GM?

We can’t know the answers to these questions, but we can remind ourselves of a few marketing measurement fundamentals:

Set clear and measurable marketing objectives: To know whether a marketing program worked, you have to first define exactly what it would mean for your program to “work”. In other words, what outcomes would make you happy?

Your objectives must be reasonable and attainable: A clearly defined objective isn’t necessarily attainable. A good outcome can still fall well short of an unreasonable objective, and be classified as a failure, when in fact the failure was in the setting of the objective.

You need to be able to get the data you need, consistently, reliably and cost-effectively: This may be at the crux of GM’s discussions with Facebook. GM may know exactly where they want to go with their Facebook ads, but they just can’t tell if they’re getting there, which when you’re behind the wheel of a $10 million dollar ad spend, is sort of important.

It will be interesting to see whether GM and Facebook can reach an agreement. My guess is that GM won’t want to walk away from advertising to Facebook’s massive and targetable audience, particularly if it seems their competitor(s) may be having success in this regard. Maybe GM just needs to know if they’re meeting their objectives and whether their Facebook ad spend has them driving on a six-lane superhighway, or somewhere down in the alley.

Filed Under: Approach, Effectiveness, Metrics Tagged With: Effectiveness, Management, marketing, Measure, measurement, objectives

Happy or Not Happy?

June 21, 2012 By Rick Shea Leave a Comment

Last week I spoke on Marketing Measurement at an event called ‘Effective Marketing in a Digital World’. During the pre-event networking, I ran into a friend I hadn’t seen since he and his wife had their first child.

Naturally, our conversation revolved around his daughter and he showed me a photo of her that he keeps on his phone. As I remarked on her beautiful blue eyes (all credit goes to his wife) and how cute she is, he also pointed out what a good baby she is and how she doesn’t cry too much or too loudly.

Then, perhaps influenced by the topic on which I was about to speak, we started joking about how his daughter rates quite highly on two important metrics for measuring baby quality; cuteness and crying volume. We decided one could use these two metrics to categorize all babies into one of four quadrants of a matrix, as follows:

  • Quadrant #1 – Very Cute babies that cry quietly
  • Quadrant #2 – Very Cute babies that cry loudly
  • Quadrant #3 – Not Very Cute babies that cry quietly
  • Quadrant #4 – Not Very Cute babies that cry loudly

Of course, few parents would put their babies into the 3rd or 4th quadrants, but assuming some did, here’s how parents in each quadrant might feel:

#1 – Pleased to have a quiet and very cute baby
#2 – Hoping baby will grow out of this ‘Loud Crying’ phase, but thankful for the cuteness.
#3 – Hoping baby will grow out of this ‘Not Very Cute’ phase, but thankful for the quietness.
#4 – Hoping for improvement on both characteristics.

We laughed about the inappropriateness of categorizing babies this way, agreed to get together soon and continued networking with others.

The next day, as I reflected on the great people I met and the conversations we had, I recalled that silly matrix conversation. Then I remembered how I had once devised a similar matrix to categorize all marketers by their measurement efforts and whether they were happy with those efforts. In this case, the four quadrants were:

  • Quadrant #1 – Companies who measure marketing and are happy with their measurement
  • Quadrant #2 – Companies who don’t measure marketing and are happy they don’t
  • Quadrant #3 – Companies who measure marketing and are not happy with their measurement
  • Quadrant #4 – Companies who don’t measure marketing and are not happy they don’t

Unlike the baby matrix where most parents would say they are in the 1st or 2nd quadrants, I think a lot of companies would say they are in the 3rd or 4th quadrants. This is not surprising as marketing measurement cannot be done perfectly and so there is always a way to improve.

It can be helpful to decide in which quadrant your company sits, and why, as this can lead to improving your marketing measurement.  Let’s look a few key characteristics of companies in each quadrant:

Quadrant #1

  • Spending enough on marketing that they need to evaluate and manage that spending.
  • Learning what they need to know to improve marketing decisions and business results.
  • Spending appropriately on measurement relative to the size of their marketing budget.

Quadrant #2

  • Those with small marketing budgets have little or nothing to measure.
  • Those with larger marketing budgets who don’t measure are either making great instinctive marketing decisions based on limited information, or they may just be unaware of their ineffectiveness and any missed chances for improvement.

Quadrant #3

  • Measuring but not learning enough to improve marketing decisions.
  • Current measurement efforts may be inconsistent or sporadic.
  • May not have a standardized approach, making it difficult to compare individual program results to benchmarks and other programs.
  • Might be overspending on measurement, making it too big a percentage of their marketing budget.

Quadrant #4

  • May not be achieving their business objectives and are feeling pressure to better manage their marketing spending to that end.
  • May not measure due to a shortage of resources, such as time, money, people and expertise.
  • May lack clear, measurable marketing objectives that facilitate effective measurement.
  • Measurement may seem too daunting to undertake, given the increasing complexity of marketing and customer decision-making processes and, possibly, a resulting perception that the only suitable approach to measurement must also be complex and therefore too costly.

Where Are You Now and Where Do You Want To Be?

I know, that’s a little vague. I’m not looking for answers like “I’m at the office and I want to be at the cottage”, although that is a very good answer. I’m wondering which quadrant your company is in currently, and whether that’s good enough for you.

If you’re already happily in Quadrant #1, congratulations, you can leave now for the cottage! However, if you’re in one of the other quadrants and you’re spending a significant amount of money on marketing, you may still have some work to do before you pack your SUV, particularly if you’re not achieving your business objectives.

One of the points I made in my presentation is that to be effective at marketing, you have to do four things well:

  1. Research: Insights about markets, competitors, customers, etc.
  2. Strategy: For the business, your brands and how you will go to market
  3. Execution: The marketing programs that help you find, develop and keep profitable customers
  4. Measurement: To know if your strategies and executions are delivering

One of the main benefits of measurement is the ability it gives you to make improvements to your strategies and executions. If you are not measuring, or if you are not happy with your current measurement efforts, there is a solution, and that is to build an effective measurement process. If you need my help, I’ll be at the cottage.

 

 

 

 

 

 

 

 

 

Filed Under: Approach Tagged With: Effectiveness, marketing, Measure, measurement, Metrics, objectives

Inputs & Outputs

May 17, 2012 By Rick Shea Leave a Comment

One of the challenges in writing a monthly newsletter is writer’s block. It generally hits me in one of two ways. Either I have no idea what to write about, or I have an idea, but no story or setting for the idea.

I have two approaches to deal with writer’s block. I find that going for a walk in nearby Monarch Park is a great way to clear my head and then somehow the ideas come to me. Finding a story or a setting for my idea can be harder. Something has to happen so I can connect the idea to a story. Usually, I need to read something or get out and do something. Through interacting with a new person or situation, a story sometimes emerges.

Monday evening, faced with neither an idea nor a story for this newsletter, I ventured out to a McGill Alumni event at the Carlu where I could mingle and meet people. Among those I met were two relatively recent graduates (relative to me, that is) with whom I had a very enjoyable, wide ranging conversation. Unfortunately, nothing in our conversation triggered an idea or a story for this month’s newsletter, although I was happy to learn about “The Undercover Economist” Tim Harford, whose writing I’m already enjoying.

On my way home, I thought about other people I’d met lately and then the idea came to me. I realized how a discussion a couple of weeks ago with a highly skilled and experienced market researcher related to how marketing scorecards are an effective way to organize diverse types of data.

We discussed how the various things that can be measured about marketing are either inputs, the things that influence the desired customer behaviour, or outputs, the results of that customer behaviour. This concept can be very helpful in determining how to organize the marketing metrics on your scorecard, and in deciding how to weight them within your overall scoring system. Let’s look at some examples.

Marketing Input Metrics

First of all, there are two broad categories of inputs; those you control and those you don’t. Inputs under your control are generally related to how well you execute the program you are measuring. Examples could include:

  • The percentage of the in-store displays or signs you printed and distributed that were actually and properly put up in store
  • The percentage of all the promotional labels or neck tags your merchandizing partner actually affixed to your products
  • The number of and cost per impression of all your on and off-line marketing communications related to this program

Inputs outside of your control that might impact the success of your program could include:

  • Competitive activity – they dropped or increased their price, promoted heavily while your program was in market, had a PR disaster on Twitter, etc.
  • Weather – no one showed up at your well promoted event because of a massive snow storm

Marketing Output Metrics

There are also two types of outputs, but they are defined a little differently. The first are those outputs or results that are directly attributable to your marketing program. Examples might include:

  • Number of unique visitors to a landing page on your website built for this program
  • Click through rate from your landing page to the buying page
  • Number of new customers who bought using your promotion codes

The other type of outputs are those that are potentially but not definitely or entirely attributable to your program.  These are typically key business performance metrics that can be influenced by a variety of inputs. Examples might include:

  • Revenue for the brand being promoted
  • Market share of that brand
  • Average price per unit sold during the program

Grouping your metrics in this logical fashion on your scorecard can make it easier for you to select your metrics and make decisions about how to weight them by group. Inputs directly under your control and outputs directly attributable to your program should be more heavily weighted than outputs potentially attributable to your program. This is especially true if you tend to have a lot of programs in the market simultaneously. Whatever weightings you use, be consistent over time to ensure you can meaningfully compare programs to each other.

Exclude those inputs outside of your control from your overall calculations. It would be very hard to set objectives and to score against those objectives, or to know how much of an impact they really had. We know that a blizzard of the century will keep more people home than a light dusting of snow, but the amount of snow that makes people decided to stay home is different for everyone. Still, note whether you think external factors significantly impacted your results.

As I wrote this, I realized my opening story does connect to the idea for this newsletter, after all. My story was about an input, an activity under my control, in this case networking and meeting people. That created an output that was at least partially attributable to my networking efforts. I may have still come up with the idea without going to the Carlu, but I might not have found my story!


Filed Under: Approach, Metrics Tagged With: Effectiveness, marketing, Measure, measurement, Metrics, ROI, scorecard

What Problem do you Want to Solve?

April 19, 2012 By Rick Shea Leave a Comment

Earlier this week, I did a little light reading on big data. I’ve been hearing a lot about big data lately so I thought I’d investigate.

Truthfully, reading about big data is hardly light reading. Big data presents a big challenge and is emerging as a hot topic in marketing and general business management circles.

What is big data? Well, it’s not about presenting numbers in larger fonts to make it easier for people over 40 years old to read, although I’d probably appreciate something like that. Big data relates to the fact that businesses (and not-for-profits, and governments, etc.) operate in a data rich environment featuring increasingly voluminous, complex and diverse data.

For many organizations, there is more data coming at them than they can handle. The data is evolving rapidly and outgrowing their ability to analyze and glean the insights they need from the data to make better business decisions.

I liked the closing section in this article from which I’ll paraphrase advice from Christer Johnson, IBM’s head of advanced analytics in North America. To get started in tackling big data, first decide what problem you want to solve. That’s great advice in many aspects in life, including big data and it certainly applies to marketing measurement.

I’m reminded of the oft-quoted John Wannamaker, a pioneer of the department store concept in Philadelphia in the 1860s, who famously said:

“I know that half of the money I spend on advertising is wasted; the trouble is I don’t know which half.”

I think of John Wannamaker as one of the founders of the discipline of marketing measurement, as he may have been the first one to define the problem. I’m not convinced he ever solved the problem, but at least he knew what he needed to know. Here’s my take on the problem he was trying to solve.

For context, John’s quote comes from a time with a much less complex marketing environment, before there were any broadcast, internet or mobile media. John’s choice of advertising tactics was probably limited to a few simple options such as:

  • newspaper ads
  • flyers handed out to passers by
  • outdoor signs
  • a guy with a sandwich board on the sidewalk in front of the store
  • a boy cycling around the store, yelling out this week’s specials (a very primitive form of Tweeting)

Yet, in that simple world, John Wanamaker didn’t know which half was wasted. If John were alive today, he’d probably admit that he didn’t even know if it was half, or one quarter or two thirds that was wasted. All he really knew was that some forms of his advertising were more effective than others, and he wanted to know which they were.

With all due respect to John Wannamaker, I’d like to restate his well-known quotation as:

“I know that some of my advertising programs are more effective than others; the trouble is I don’t know which ones. Mostly, I just want to know the best way to spend my money.”

We can modernize this problem statement by substituting the word “advertising” with “marketing” and then it can serve as a starting point for most companies’ marketing measurement efforts. Like John, all managers with a marketing budget need to determine how to optimize that budget to meet or exceed their business objectives.

In these more complex times, with many more marketing tactics to choose from, there is also a lot more data to analyze and understand. Each program may target different customers, using different tactics with different objectives and performance metrics. Gathering the data for those metrics can involve a variety of sources, analytics tools and research techniques.

All that diverse data, big or otherwise, can certainly be a big mess if you don’t have a way to organize and analyze it. The analysis needs to happen in a way that enables comparing each program’s outcomes to the others, so you can identify the best ways to spend your marketing budgets.

A well-designed marketing scorecard can solve this problem. The key is to design your scorecard in a way that makes comparisons between diverse programs meaningful, and helps you to solve the same problem John Wannamaker was trying to solve all those years ago, to find the most effective ways to spend his money.


Filed Under: Approach Tagged With: Management, marketing, Measure, measurement, Metrics, ROI, scorecard

Measure Well, Sleep Well

March 15, 2012 By Rick Shea Leave a Comment

If you know me or have been reading this newsletter for any length of time, you may know that photography is my favourite pastime. What you may not know is that organizations sometimes bring me in to take photos of their events, which is how I found myself at the AllerGen 2012 Annual Research Conference.

AllerGen is a not-for-profit organization whose role is to mobilize Canadian science to reduce the illness, mortality and socio-economic costs of allergic disease. The conference showcased the latest research in this regard and while often over my head scientifically (not hard to do), I found it quite interesting.

During an afternoon break at the conference, a distinguished looking gentleman named Douglas Barber approached me to talk photography. Our pleasant conversation eventually shifted to the conference and he told me a story that I quickly realized fit my thinking on marketing measurement.

Douglas explained he is on AllerGen’s board and that an issue of concern to him is the cost to the Canadian economy from the “asthma drag” on productivity. He explained how asthmatics can be less productive at work or even miss entire days of work following sleepless nights caused by asthma. Parents of asthmatic children can also experience the same productivity losses. Douglas also told me how he once did a quick “back of the envelope” calculation to estimate that asthma costs our economy between $10 and $20 billion per year in lost productivity.

Sometime after Douglas did his quick calculation, a full study was done to properly analyze and estimate the economic impact of asthma’s drag on productivity. The study concluded the annual costs are $15 billion. That’s right; a costly and complex measurement process produced the same answer as one expert using a pen and the back of an envelope.

Two aspects of this story relate to my views on marketing measurement:

  • Douglas’s back of the envelope calculation relative to the full study is similar to how a marketing scorecard can be a proxy for a sophisticated and costly marketing measurement process. In both cases, the less sophisticated approach doesn’t need to be perfect, just accurate enough to support analyzing options and making the right decisions. As I like to say, it’s not about precision, it’s about the decision.
  • The back-of-the-envelope estimate worked because it was done by an expert using a sound methodology. Douglas has an extensive business background and apparently knows more than just a little about productivity and related calculations. Scorecards are a proven methodology that you can enhance with expertise about your marketing and your business.

There is another lesson in Douglas’ story, and that’s the need to right size your measurement efforts to the magnitude of the decisions you need to make.

Research Investment Decision

  • Douglas’ back of the envelope calculation and the full-blown study produced essentially the same estimate and both pointed toward making the same decision. It’s a pretty compelling proposition if investing perhaps a few hundred million dollars into research would lead to recovering even just 10%, or $1.5 billion of the lost productivity, especially as that benefit would be realized every year.
  • The problem is that any decision to potentially invest a few hundred million dollars needs to be substantiated by more than a back of the envelope calculation. In this case, the cost of the research needed and the probability of recapturing that 10% are two other variables that I think would need to be estimated. It’s understandable that a full-blown study was needed to examine the overall business case.

Marketing Investment Decision

  • Similarly, for companies that invest tens of millions annually in marketing, it makes sense to support the decisions that need to be made with sophisticated marketing measurement efforts that might cost hundreds of thousands, or more.
  • For most companies with smaller marketing budgets, a practical lower cost approach such as one using a scorecard may well be the right sized measurement solution. In most cases, the overall measurement expense likely needs to be a small single digit percentage of the total marketing budget.

I like simple and elegant solutions that deliver what you need. A marketing scorecard’s simplicity keeps measurement costs down, while its elegance allows the flexibility to include a suitable level of expertise and sophistication to right size your measurement efforts to your marketing budgets.

Whichever measurement approach you choose, be sure to combine a sound methodology with the right expertise to learn what you need to know to make the right decisions. Measuring well will help you to sleep well and be a productive marketer!

Filed Under: Approach Tagged With: Effectiveness, Management, marketing, Measure, measurement, return on investment, ROI, scorecard

Wine Scoring & Marketing Measurement

February 16, 2012 By Rick Shea Leave a Comment

Tuesday evening I was browsing the latest edition of the LCBO’s ‘Vintages Release Catalogue’. This catalogue provides descriptions and sometimes wine critics’ quality scores for the new wine products about to be released through Vintages stores in Ontario. As I browsed, two thoughts came to mind.

Firstly, I noticed that most of the scores in this catalogue were between 88 and 92 on a 100-point scale. It struck me that this suggested the majority of the wines in this catalogue were of very similarly high quality, with almost all wines rated within a narrow 5-percentage point range. I found that odd, perhaps unrealistic, and decided to think about it. Secondly, I noticed that the wine descriptions were making me thirsty.

Wine Glass, Red Wine

Seeing the wisdom in choosing the beverage that best suited the task at hand, I poured myself a glass of red to compliment my thinking, sat down with the catalogue and made a few calculations and notes. Here are some highlights.

  • Vintages published scores for 57 of the 120 wines in this catalogue. The wine critics quoted used the 100-point scale for 48 of the 57 rated wines. The other nine were based on 20, 5 or 3-point scales.
  • Of the 48 using the 100-point scale, 41 (85.4%) received a score between 88 and 92, and 30 of those were either 90 or 91, which confirmed my first observation. The other 7 wines were rated higher than 92, leaving no scores below 88.

Taking a sip from my glass, I contemplated why so many wines received such similar scores, and how all this relates to marketing measurement. Here are a few thoughts:

Wine Scoring: The LCBO is in the business of selling wine and I suspect they have a policy of only publishing scores of 88 or higher. I tested this theory by looking at the two previous Release Catalogues and wasn’t able to find a wine scoring 87 or lower. Perhaps they’ve learned that lower scores reduce sales and so don’t publish scores below 88.

Marketing Measurement: Marketers are in the business of spending money effectively to drive positive business outcomes. Instead of measuring only the best marketing programs or those you might want to cast in a favourable light, measure and rank all programs so you can identify which are most and least effective, and then optimize future strategies accordingly.

Wine Scoring: By my rough count, the 48 scores using the 100-point scale were sourced from 26 different wine critics. While each used a 100-point scale, I have a hard time believing all 26 used the scale in exactly the same way. I also suspect that some critics are more generous with their scores than others, like my calculus teacher in CEGEP. The other important issue is that scoring wine is a highly subjective exercise. It isn’t at all uncommon for two or more tasters to disagree on a wine’s quality and the corresponding score. Experts have different opinions on subjective matters.

Marketing Measurement: To minimize inconsistencies, reduce or eliminate subjectivity and personal bias from your measurement processes. Having 26 experts using similar but sometimes different methods of scoring your marketing programs based on their personal opinions would not be a recipe for consistency. One person needs to lead your measurement efforts using one methodology that your organization understands and supports.

Wine Scoring: I did a little reading on wine scoring and discovered that wine critics can be inconsistent in the scores they award to the exact same wine on different occasions. For example, the influential wine critic Robert M. Parker has apparently pointed out that he sometimes assigns different scores to the same wine at different tastings, but that those scores tend to be no more than 3 points apart. It seems that differences in tasting conditions and the taster’s emotions can lead to different scores. To address this, I believe Robert Parker publishes average scores when multiple tastings produce different scores.

Marketing Measurement: Consistency is important in making comparisons meaningful. Pick one methodology that can be used consistently across all programs. Consistency should help you to avoid having all your scores cluster within a narrow range where differences may not be significant, or actionable. Programs can differ significantly in their effectiveness at meeting your objectives, and so their scores should reflect those differences. Also, if data for one metric is collected at various times, or from different sources, you might want to follow Robert Parker’s lead and use an average score for that metric.

Advice for Wine Drinkers: Don’t worry about the difference in quality between a wine that scores 88 and another that scores 92.  Both are high quality wines and the difference in scores may come down to who tasted it, under what conditions and the tasters’ preferences. Here’s the fun part. Through trial and error, you should eventually be able to determine which wine critic your tastes best align with, and then the ratings and tasting notes from that critic will help you to make better wine purchasing decisions.

Advice for Marketers: Similarly, there will be some trial and error involved, but not nearly as much fun. Select a measurement methodology that you can apply fairly, without personal bias, and consistently across all programs. Be disciplined about measurement and it will ultimately highlight which marketing programs best meet your objectives and create value for your business. That will help you to make better marketing decisions, which you may wish to celebrate by opening a bottle of your favourite wine!

Filed Under: Approach, Objectivity Tagged With: bias, comparison, Effectiveness, Management, marketing, Measure, measurement

Do Daily Deal Coupons Work?

January 19, 2012 By Rick Shea Leave a Comment

Daily deal coupons are all the rage these days, led by Groupon and their numerous competitors. Since I have a pretty extensive background in consumer promotions, particularly couponing, I tend to get pulled into discussions on daily deal coupons and I often notice the frequent related media coverage. Setting aside all the buzz around them, they’re really just a new twist on an old marketing tactic, but do daily deal coupons work?

Traditional paper coupons and price discounts have worked quite nicely, since the late 1800s in the case of coupons, and I suppose for centuries or millennia in the case of price discounts. One of the main reasons to discount is to attract new customers to try your products or services, and then hopefully sell them more stuff and/or turn them into profitable repeat customers who will pay full price on future purchases. Daily deal coupons can certainly do this.

It is beyond the scope of this newsletter to address the dos and don’ts of daily deal coupons, but you’ll get some insights if you Google “Groupon horror stories” and “Groupon success stories”. I will say that you’ll want to have a way to limit response levels or to make sure you can live with whatever level of response you get.

To address whether daily deal coupons work from a measurement perspective, my answer leads to more questions, starting with “Well, it depends, what did you want them to do for your business?”

Drilling down a bit further, answers to the following four fundamental questions should inform the planning, objective setting and measuring of any marketing program. Clear answers should point the way to what and how to measure, and whether the program “worked” to meet its objectives.

Let’s look at each of these.

1. Who are you targeting? While many answers are possible, the best answer is often “new customers” to help grow your business.  Merchants benefit most when they structure their daily deal coupon offers to attract new customers, rather than subsidize existing customers who would have bought without a coupon.

2. What do you want them to do? You’ll want new customers to buy for the first time whereas you may want existing customers to buy something more or different than usual. In addition, you might try to prop up an under-performing aspect of your business, such as your slow month or time of day, or a product that isn’t selling well.

3. How much value will that create for your business? This one is especially important and a bit tricky. You’ll need to consider the short term (this transaction) and the long term (the customer’s lifetime). Ideally, you’d like this transaction to create enough value to at least cover your costs, but if it doesn’t you’ll need to make up the difference and ideally much more over the lifetime of those customers who buy your daily deal.

Short Term – This Transaction: Consider your variable cost of providing the products or services you will sell through this coupon and compare that to the revenue from your share of the coupon selling price, which you will split (often 50/50) with the daily deal provider. Also, consider whether you’ll receive your share of the revenue when the coupon is bought, or when (and if!) it is redeemed.

Long Term – The Customer’s Lifetime: How long is the lifetime of a typical customer? On average, how many times will each customer buy over that lifetime, and how much will they buy each time? If each new customer’s lifetime is just this one transaction, it may not be worth your while to offer this coupon. But, if you can convert enough of those new customers to loyal repeat customers for many years, then discounting to get them in the door should be worthwhile.

4. How many people do you need to do that for this expense to be worthwhile? Once you know what a customer is worth to you over whatever time frame you want to use, and you know your costs, then you can set an objective for how much value (metrics include new customers, number of transactions, transaction values, etc.) you want this marketing investment to generate. Then you can measure against those objectives.

Clear Objectives Make Measurement Easier

To measure whether or not a daily deal coupon or any marketing program worked for your business, you need clearly defined objectives. In other words, to measure whether you have succeeded, you must  first define success. Clear objectives will tell you what metrics to use and where to find the data.

It’s one thing to attract new customers with discounts, and quite another to keep them. Can you convert discount shoppers to loyal customers? Success will come down to your company’s ability to deliver a superior customer experience in the short term, and to build a positive relationship with each customer over the long term and maximize both the lifetime and the value of as many customers as possible. Of course, that’s something every business has to do well, however they find their customers.

Daily deal coupons may be a relatively new marketing tactic, but there’s nothing new about the fundamentals that determine whether you should use them, whether you’ll be successful and how to measure your success. Clear objectives will help you to decide whether to use daily deal coupons and to evaluate whether using them worked, however you define success. To compare the success of a daily deal coupon program to any other type of marketing program, well that’s a topic for another newsletter.

 

 

Filed Under: Approach Tagged With: Management, marketing, Measure, measurement, objectives, Planning

Emissions Test

December 15, 2011 By Rick Shea Leave a Comment

My car is getting old. Like me, it doesn’t always feel its age, but the reality is that by most measures it is getting old. One measure of its advancing age is that in order to renew my vehicle licence recently, my car first had to pass an emissions test.

In Ontario, when the Ministry of Transportation sends you your vehicle licence renewal application, they indicate if your car needs to be emissions tested.  This time around, my middle aged car was due for a test.

The Ministry requires the test be performed at one of the accredited Drive Clean facilities listed on their website.  I found a local facility and within 15 minutes of arriving, I drove out of there with the clean bill of health for my car that I would need to renew my vehicle licence.

While waiting for my car to be tested, I entertained myself by reading the wall poster and brochure that described the emissions testing procedure. As a guy who cares about good measurement practices, I was impressed by two things:

  1. The Ministry has a consistent and transparent measurement process.
  2. Their process includes a clear and simple scoring system.

Let’s look at how each of these illustrate what a good marketing measurement system needs to do.

Measurement Process

These are the steps that all Drive Clean facilities follow when testing a car:

  1. Perform visual inspection
  2. Drive car onto Dynamometer
  3. Insert probe into tailpipe
  4. Accelerate vehicle to 40 km/hr
  5. Computer analyses emissions and compares to standards for make, model and year
  6. Print test results
  7. If Pass, owner can renew vehicle licence
  8. If Fail, owner must repair car at a Drive Clean Repair facility, then re-test

Here’s what I like about this process, especially as the same things are true of a good marketing measurement process:

  • Its clear steps can be repeated consistently, making the results meaningful and comparable to other results, standards and benchmarks.
  • The process is well supported by documented procedures, such as the 20 page “Drive Clean Guide” that I found on line. The guide is amended and reissued periodically, as needed.  Well documented and updated procedures ensure consistency in measurement and clarify who is responsible for doing what and when.
  • The standards that define passing and failing results are clearly documented.

Scoring System

The scoring system is quite simple.  There are a few metrics for which your car must obtain a passing score.  For example, on Hydrocarbons, as measured in Parts Per Million, my car needed to be at 66 or less in order to pass. Fortunately, my reading was 3 and thus my result on this metric was a “Pass”. (I lobbied for extra points for such a low score, but to no avail!)

The key elements of this simple scoring system that relate to marketing measurement are:

  • Metric: Define the metric (Example – Revenue) and the unit of measurement (Example – for Total Company, for Specific brand, per square foot, per average transaction, etc.).
  • Objective: Define the target result, or limit, or range of acceptable results. (Example – Average revenue per transaction of $50 for Brand X in 1st Quarter of 2012).
  • Result: The actual outcome for the metric and objective as defined.
  • Scoring: You need to score how good the result is as compared to the objective. Will you use a pass/fail, a 10 or 5 point scale, a score out of 100, or some other scale?  Whichever you pick, be sure you can apply it consistently and fairly.

Summary

A properly documented measurement process will provide the consistency needed to make comparisons between programs meaningful.  A clear scoring system gives you a way to convert all your results to a common metric, which also enables comparing programs to each other.  These two essential components of a reliable marketing measurement system support making well informed strategic marketing  decisions, which comes down to making choices between programs, refining your strategies and finding the most effective ways to market your products and services.

Filed Under: Approach Tagged With: Effectiveness, marketing, Measure, measurement, Metrics

Pictures at an Exhibition

November 17, 2011 By Rick Shea Leave a Comment

I’m a long time hobbyist photographer. While I occasionally get hired to take photos for commercial use (e.g. events, product shots, architecture), I mostly just do it for fun. In fact, I had never formally exhibited my photos prior to this past weekend when I participated in an art show on my street.

Now in its fourth year, this show is basically an art walk limited to the one block of the street on which I live. I was one of 10 artists exhibiting a variety of art in 5 houses, on both Saturday and Sunday afternoons.

Reflecting on the event Sunday night I decided that it was a lot of fun, I’m really glad I exhibited and I’ll probably do it again next year. Since that simple statement doesn’t make for much of a newsletter on marketing measurement, I thought I would elaborate on how I came to that conclusion.

What I Spent

To begin with, my cost to participate was very low. Since the walls in my house were already covered with plenty of framed photos and ready to exhibit, I just needed to shuffle a few around and decide on prices. I probably spent all of about $50 to get ready, including salty snacks and chocolate chip cookies for my visitors. So, my costs were minimal.

What I Sold

I sold a total of 7 photos for $525. I’m happy with this outcome because:

  • Over the 2 days, I estimate 45 people dropped by my house. One person bought 2 photos, meaning I converted 45 prospects to 6 customers, for a conversion rate of 13.3%. This seems like a good rate for an art show, but I don’t have a benchmark to compare it to.
  • My sales surpassed my expectations. I thought I would do well to sell two or three photos and maybe pocket $200.
  • My customer acquisition cost ($50 ÷ 6 = $8.33 per customer) was much lower than my average revenue per customer ($525 ÷ 6 = $87.50 per customer) making each transaction and the overall event profitable.

What I Learned

For some background, our marketing efforts consisted primarily of:

  • Post card invitations hand-delivered to homes within roughly a 2 block radius
  • Each artist emailed invitations to their own list of contacts
  • The organizers solicited and obtained support from local politicians who emailed local residents and tweeted our event
  • Free on-line publicity, most of which came through a new local website called GrownUps55plus, for which I’m saying thanks by this mention

Secretly wearing my marketing measurement hat while disguised as an artist, I randomly asked those visitors I didn’t know the usual “how did you hear about us” kind of questions.  I learned that, in addition to those friends and neighbours I already knew, most people either:

  • Lived in the neighbourhood, or
  • Were friends of one of the other artists

From a product point of view, of the seven photos I sold, 4 featured trees as the main subject, 1 featured a tree and a window, 1 featured a window, and the other featured a racoon. Selling the racoon photo gave me a chuckle because earlier that day two other people had separately A) cursed the racoon, and B) saluted it with a middle finger. Clearly, one person’s art is another’s neighbourhood menace. Also, people like trees.

Another interesting fact is that the turnout was much higher last year when some houses estimated traffic at 300+ visitors. Perhaps that’s a sign people are less confident in the economy this year? Imagine if I could have applied my 13.3% conversion rate to 300 people instead of 45!

What I Would Do Differently Next Time

Based on my informal measurement efforts and observations, and input from the other artists, here are some thoughts for us to consider for next year:

  • Proximity and familiarity seem to bring people out, so we should expand our post card coverage beyond 2 blocks and each artist could invite more people.
  • Recruit more houses and artists to participate; the additional artists can invite their contacts, and a larger event with more art for sale should appeal to more people.
  • Continue to solicit free publicity and politician support.
  • Find a volunteer with public relations expertise, perhaps a student looking for experience, to help us get more local and on-line media coverage.

Conclusion

I didn’t apply measurement to our marketing efforts in any sort of disciplined way, yet maybe that’s appropriate for such a low budget event. Still, by simply looking at what I spent, what I sold, what I learned and what I would do differently next time, I should be able to improve my sales next year, and that’s what counts.

Whether for a small event like this, or a major marketing initiative, the best way to improve next time is to make sure you have a way of learning from what you did this time. Measure, review, reflect and capture what you’ve learned, so you can optimize future strategies.

There’s other soft data to consider, too. I had a great time, met some neighbours, saw some friends and enjoyed seeing people enjoying my photos.  I look forward to next year and in the meantime, I’ll be out shooting windows and trees, or maybe even a racoon.

 

Filed Under: Approach, Effectiveness Tagged With: Effectiveness, Management, marketing, Measure, measurement
« Older Posts
Newer Posts »