February, 2012 – This article I wrote for Luxury Daily discusses the validity and reliability of research among “luxury” and “affluent” consumers. ( http://www.luxurydaily.com/how-reliable-is-your-research-among-luxury-and-affluent-consumers )

By Ron Kurtz

There are many ways that weak research, combined with poor analysis, can lead to bad conclusions.

Research among “luxury” and “affluent” consumers is vulnerable to errors in various areas. For example, the lack of an objective or universal definition of “luxury” creates uncertainty as to what respondents understand the word to mean.

In addition, the level of “affluence” associated with true luxury consumers is subject to various differences of opinion among researchers. Some use a level of income, a criteria which seems to be much less stable and reliable than net worth, according to a recent study by the Federal Reserve Board.

The specific level of income or net worth can also vary substantially among researchers.

Rich definition?
Most consumer research, including that among luxury and affluent consumers, is based on online surveys among panels of people who have “volunteered” to respond to frequent and often lengthy questionnaires.

The validity and reliability of such research should be carefully considered given a recent study that showed how panel members are often treated poorly.

The study showed that the poor treatment of panel members included “paying them pennies, giving them boring, lengthy, or irrelevant surveys, frustrating them with multiple closed studies, and bombarding them with opportunity after opportunity.This is most definitely not how you want to treat people upon whom you are depending for your success. And if you or your research vendors are not paying attention, this is exactly what may be happening in your research.”

While the report provides a review of general consumer panels, these same panels are the sources of respondents for surveys of affluent and luxury consumers.

It is hard to imagine how such people can be representative of the true luxury and affluent consumers. And, as a result, it is doubtful that the research results are projectable.

The new report is a sequel to a 2009 report by a prominent market research firm that ran an internal test on a few panels it had used or was considering. It arranged for a selection of mystery shoppers to sign up for each panel and be typical respondents for a month.

The new test used the same methodology to evaluate 12 major panels, including Toluna, e-Rewards, Clear Voice, Surveyhead, Opinion Outpost, MySurvey.and six more. All were evaluated from the perspective of the typical panel member.

Panel, not pummel
Some of the problem scenarios identified in this study of research panels were:

“Yours is the tenth questionnaire in a row that the respondent has completed that morning, and many of the others were long, boring, and irrelevant. The respondent is tired and inattentive.

“Or let us say that the respondent has attempted 12 different questionnaires this morning before trying yours. One of them asked 10 minutes’ worth of questions before telling the respondent they were not qualified – and tossed them out with no reward.

“One of them froze when the respondent was mostly done.

“Another one told the respondent they were not qualified and kicked them out before they could answer a single question.

“Two more were actually called “surveys” but were trying to get the respondent to compare car insurance rates.

“Five of them were already closed by the time the respondent tried to respond, even though the invitations were all sent yesterday or today. The respondent was disqualified for two more because they do not own a pet, even though they stated in their panel profile that they have no pets.

“The respondent is tired, frustrated, and annoyed, and now they are evaluating a new product concept that you really hope they will like. Now just how reliable is your data?”

BOTTOM LINE, if you buy syndicated research, you should ask if it is based on a panel survey and if so what company provided the panel.

If you are doing your own survey, you should be careful in how you select a panel provider. Or perhaps you should consider a different survey methodology.

Good research can be a valuable contributor to good planning and decision making. Good research requires careful planning and analysis. While this is not always easy, it is well worth the effort.

Posted in Affluence Research, Luxury Defined, Luxury Research Methodology | No Comments »

February, 2012 – This article from Inman News discusses an upscale, gated and guarded subdivision developed by Disney. ( http://www.inman.com/news/2012/02/21/your-very-own-magic-kingdom-a-price )

By Mary Umberger

ORLANDO, Fla. — You say you’re thoroughly smitten with all things Disney? You say you’ve been known to joke that you’d practically live at Walt Disney World if you could?

Got millions of dollars to spare?

That’s more or less the cost of admission for owning a home at Golden Oak, an upscale, gated and guarded subdivision that Disney is developing almost in the shadow of its iconic theme parks here — the development is just three miles away from Disney World.

But unlike the popular Disney theme parks, these homes are not approachable for the masses.

At prices ranging from $1.5 million to upwards of $8 million, the developer promises a house and neighborhood with the hallmarks it has carefully cultivated for decades: meticulous attention to detail; extensive personal service; and, if you’re so inclined, a daily dose of Mickey, Minnie and the crew.

Golden Oak’s development team is courting a very well-heeled clientele of Disney diehards that it’s certain are out there, in addition to others who may be lured less by “The Little Mermaid” than by the promise of tight security and a deep trust in the Disney brand.

To read the full article click here.

Posted in Affluence Research, Home Purchases & Remodeling, Vacation Homes | No Comments »

February, 2012 – This article from JustLuxe.com discusses the complicated nature of affluence research and what to be careful of when using the results. ( http://www.justluxe.com/luxe-insider/trends/feature-1729354.php )

By Susan Kime

My qualifications for writing this piece are these: I have done graduate level research, created my own design and collection methodology, and taught research design and program evaluation for two years, at the university level. I know the gnawing questions regarding research design, data collection methods, and validity/reliability measures. These are issues that kept me awake at night, worrying.

Is there something in the design, or in the research questions or in the sample size that will literally invalidate two years of work? Is there such a thing as true objectivity? And how valid are mail surveys? How generalizable are the results, especially if I can’t do longitudinal studies? How many similar observations and answers determine a true trend? And, was Heisenberg correct after all, with his Uncertainty Principle? If the more precisely one property is measured, the less precisely the other can be controlled, determined, or known, and where is the Tylenol-PM?

I have written two articles on JustLuxe about Luxury’s Mixed Messages, and I have not discussed one of the most challenging reasons for the mixed messages: the complicated nature of affluence research. There’s a reason: when I discuss the numbers and trends in this type of work, I often do NOT know the methods of data collection, and the designs used. I know, as most do, the emerging numbers and the generalizations posited from those numbers.

Affluence research is a type of market research — a method of surveying individuals using controlled and specific processes. It produces quantitative, measurable data, that often can be used to predict buying patterns, future demand, identification of target markets.

Quantitative market research refers specifically to the collection of factual, measurable data that includes personal data — sex, age, annual income, or number of children, of the person being surveyed. It also includes data like the number of times the interviewee visits a particular store or restaurant, the amount of money she spends on groceries every month, and the number of hours she spends watching television or listening to the radio.

A great amount of valuable information can be extracted from these data, so it is incumbent on the researcher to be as stringent as possible in his or her design and data collection procedures, as most errors can be traced to problems with how data are gathered. In particular, these errors occur due to problems associated with research validity and reliability, the two cornerstones of all quantitative and qualitative research.

Regarding validity, the question relates to whether the research is really measuring what it claims to be measuring. For instance, if a marketer is purchasing a research report from a company claiming to measure how people prefer the marketer’s products over competitors’ products, the marketer should understand how the data were gathered to help determine if the research really captures the information the way the research company says it does. And, regarding reliability, the question relates to whether research results already garnered, can be applied to a wider group than those who took part in a study. In other words, would similar results be obtained if another group containing different respondents or a different set of data points were used? If results are similar then it is likely the method of data gathering is reliable. Assuring research can be replicated and can produce similar results is an important element of the scientific research method.

With this said, Ron Kurtz, President and CEO of the American Affluence Research Center has taken on some of these issues with his Trends Caveats. His 30 year-plus research experience involves an MBA from Harvard, a book he wrote on Market Research Strategy And Techniques, and successful market research projects for Ford, Gillette, and groups in the travel field. He formed his own market research firm, American Affluence Research Center in 2002, where he has done a substantial amount of longitudinal survey research with the top 1% of the affluent population.

His Trends Caveats have been recently disseminated, and deal with his concerns with trends forecasts and the problems they evoke. I recently asked him about his motivation for writing them, and he replied, “I wrote my Trends Caveats out of a concern for those who read affluence research and accept findings that look positive and happy. Of course people want to have happy outcomes. But right now, media are inundated with trend forecasts and predictions from marketing and research agencies, consultants and pundits of all types. These predictions are often based on anecdotal research, old data or large changes in very small numbers, if they are quantified at all. This becomes problematic data, and I wanted people to be aware of it.”

Here are the Caveats:

1) Affluent consumers are not prone to substantial changes in their basic behavior and values from year to year or even over an extended period of time. This is evidenced by research begun in the 1970s by Thomas Stanley, author of “The Millionaire Next Door” and the more recent “Stop Acting Rich and Start Living like a Real Millionaire.” Our affluence research since 2002 has been consistent with that of Mr. Stanley’s.

2) A change from one year to the next is not necessarily a trend, especially if it applies to a large increase in a very small percentage of the market. That is more likely to be a fad that may or may not become a meaningful trend over a period of two or more years. For example, a market segment that triples from one percent to three percent is not a trend but may be an indicator of an emerging market.

3) Affluent consumers are not necessarily luxury consumers. Of course, the definition of “luxury” is in the eye of the beholder, which could be another important caveat when considering the forecasts of the trend pundits. Only the wealthiest one percent of U.S. households appear to be knowledgeable about the price points and brands of true luxury products. Before the recession, some luxury consumers were among the so-called mass or aspirational affluent. These consumers have been largely shaken out of the true luxury market.

4) It is important to stay focused on the key marketing priorities of attracting new customers and retaining the loyalty and increasing the purchases of existing customers. Marketers should avoid chasing emerging consumer market segments if that will cause them to be distracted and dilute efforts targeting their existing primary consumer markets.

5) Traditional marketing communications channels should not be forsaken, especially if marketers are targeting affluent and luxury consumers. The conversations among marketing professionals seem to be exclusively focused on the opinions and statistics regarding the importance of the various forms of digital channels of communications. An unintended consequence of digital media is that the consumer audience has been substantially fractionated. Equally important, many in the large numbers of digital fans and followers of luxury brands are aspirational consumers and other “luxury-curious” voyeurs who cannot afford the products.

6) The true affluent, who are typically careful spenders who live within their means, are the more knowledgeable and more sophisticated consumers. Their priorities have always been quality and value when making a purchase decision. In addition, the vast majority of the affluent have always avoided ostentatious or conspicuous consumption. These are not new priorities for the affluent.

7) Last but not least, there is no substitute for using common sense when thinking about how to be consumer-sensitive in all aspects of the relationship, interaction and communication with customers. Just put yourself in the shoes of your customers. This Golden Rule applies to product, pricing, service, post-sale relations, communications and all forms of interaction with the customer.

Many of these are arguable, yet as a whole they certainly define some of the issues faced with affluent market research paradigms — both in theory and practice right now.

Posted in Affluence Research, Luxury Research Methodology | No Comments »

February, 2012 – This article discusses the dirty little secrets of online panel research. ( https://www.gplus.com/Consumer-Services/Insight/New-study-raises-serious-questions-about-surveys )

By Ron Kurtz

Most consumer research, including luxury consumer research, is based on online surveys taken by people who have “volunteered” to frequently take lengthy questionnaires. If you buy panel research about luxury and affluent consumers, the new report “More Dirty Little Secrets of Online Panel Research” is a must read that will cause you to question the validity of the research since it shows how poorly panel members are often treated.

The author of the report, Ron Sellers of Grey Matter Research, notes of panel members that “Paying them pennies, giving them boring, lengthy, or irrelevant surveys, frustrating them with multiple closed studies, and bombarding them with opportunity after opportunity is most definitely not how you want to treat people upon whom you are depending for your success. And if you or your research vendors are not paying attention, this is exactly what may be happening in your research.”

While the report provides a review of general consumer panels, these same panels are the sources of respondents for surveys of affluent and luxury consumers. It is hard to imagine how such people can be representative of luxury and affluent consumers.

Copies of the report can be downloaded at: More Dirty Little Secrets of Online Panel Research.

The new report is a sequel to a 2009 report by Grey Matter Research, which ran an internal test on a few panels they had used or were considering as vendors. They arranged for a group of people to sign up for each panel and record their experiences as typical respondents for a month.

The test included 12 major panels, and the results were published in the report Dirty Little Secrets of Online Panel Research. A few panel mergers, plus requests about panels that weren’t included the first time, led to the new report. It evaluates Toluna, e-Rewards, Clear Voice, Surveyhead, Opinion Outpost, MySurvey.and six more from the perspective of the typical panel member.

Some of the problem scenarios identified were:

“Yours is the tenth questionnaire in a row that the respondent has completed that morning, and many of the others were long, boring, and irrelevant. The respondent is tired and inattentive.”

The respondent may have attempted 12 different questionnaires before trying yours. One of them asked ten minutes’ worth of questions before telling the respondent they weren’t qualified and tossed them out with no reward.

Another survey froze when the respondent was mostly done. Another one told the respondent that they were not qualified and kicked them out before they could answer a single question. Two more were called “surveys” but were actually trying to get the respondent to compare car insurance rates.

Five of them were already closed by the time the respondent tried to respond, even though the invitations were all sent within 24 hours. The respondent was disqualified for two more because they didn’t own a pet, even though they stated in their panel profile that they have no pets, and therefore shouldn’t have been invited to take the survey in the first place.

The respondent is tired, frustrated, and annoyed, and now they are evaluating a new product concept that you really hope they will like. Just how reliable is your data?

Bottom line: If you buy syndicated research, you should ask if it is based on a panel survey and if so what company provided the panel. If you are doing your own survey, you should be careful in how you select a panel provider. Or perhaps you should consider a different survey methodology.

Posted in Luxury Research Methodology | No Comments »