Read an Excerpt

The Art & Science of Interpreting Market Research Evidence

John Wiley & Sons

Chapter One

Summary

Market research is moving away from its roots as a discipline that was detached from the business decision-making process, and is now more actively engaged with decision-facilitation.

This shift has required new methodological thinking: an 'holistic' analysis approach that provides clients with a rounded view of what all their (qualitative and quantitative) marketing evidence is saying.

The new approach also requires analytical frameworks that combine hard market research data with prior management knowledge and intuition.

These new frameworks must be disciplined: intuitive thought can be powerful, but it can also be wrong. The new holistic approach to data analysis therefore needs to be based on the rigorous evaluation of prior management knowledge, as well as drawing on conventional data analysis methods.

We introduce a ten stage guide to analysing market research data in an 'holistic' way. The ten steps are: analysing the right problem; understanding the big information picture; compensating for imperfect data; developing the analysis strategy and organizing the data; establishing the interpretation boundary; applying the knowledge filters (what we know about the survey process); reframing the data (to give us fresh insights); integrating the research evidence andtelling the research story; decision-facilitation; and completing the feedback loop (evaluating the effectiveness of the research data in achieving a successful decision outcome).

In sum, in this opening chapter we explain how the evidence available to market researchers is changing, as a prelude to outlining the critical thinking skills - the interpretation power - needed to master the new world of information.

Reducing uncertainty

The underlying principle behind market research is powerful, yet simple. Market research is about helping individuals make informed, evidence-based judgements and decisions. It is about asking intelligent questions of users, and potential users, of products and services about their opinions and experiences, listening carefully to what they say, and then interpreting the implications of this feedback. This interpretation is then used to help organizations reduce the uncertainty surrounding various decisions that need to be made.

The idea has been around for nearly a century. Thus, today, it is commonplace for businesses, and public sector organizations, to use customer feedback as one of the inputs into their marketing strategies and public policies. There is little sign of there being any downturn in the demand for market research. Who, after all, would speak against taking all the relevant soundings on any issue, interpreting these viewpoints, and taking this into account when making a decision? The issue is not about whether or not market research is useful, but a question of how research evidence should be interpreted.

Interpretation power

Information was once power. But, today, the power lies in interpreting what information really means. In the hands of a skilled analyst, survey data may unearth invaluable insights into what makes people 'tick'. But the same data, in the hands of a journeyman analyst, may lead to a creative idea being stifled at birth. The need, therefore, is to cultivate the talent, skill, and techniques required to make sense of customer data. It is this issue - the intelligent, holistic interpretation of market research data - that is at the heart of this book.

The drivers of new holistic market research

It is possible to identify several distinct developments that have shaped the growing demand for a more holistic approach to the analysis of market research data.

Clarifying contradictory market signals

The sheer amount of market and customer information available has increased phenomenally. There has also been a change in the type of information used to inform decisions. Increasingly, use is being made of data that are more imperfect, messy, grey, and less robust than many of the sources used in the past.

The challenge for market researchers is to develop the skills and techniques needed to weave a story from a combination of different, less than perfect, often confusing and contradictory, information sources. Researchers can no longer restrict themselves to working with single, reasonably robust, sources of customer opinion.

Providing grounded business acumen

There is also the expectation that market researchers will operate with a sound contextual awareness of the wider, strategic business picture. Market researchers are required to have a mature understanding of what the client's business is trying to achieve. This has been a driving force in encouraging market researchers to put their heads above their hitherto methodologically defined parapets.

Understanding the complete customer

Market researchers are now expected to better appreciate the 'complete' customer experience. As companies strive to build a complete picture of a customer's interactions with the organization, so too are researchers required to stretch their thinking to find ways of capturing and understanding a wider range of customer data. For example, to understand how a customer relates to a bank, it becomes important to capture that individual's experience across the various personal, business, current and savings accounts they may hold at the bank, and also at other financial institutions.

The quest for understanding and 'insight'

The word 'insight' has many different connotations. Yet whatever the exact interpretation placed on the now rather overworked 'insight' word, there is a clear message here from clients: they want more originality, innovation, clarity, and depth of thinking from their market research analysts.

Bridging the data-decision gap

Users of research data want the market research industry to be more committed and involved, than in the past, with the decision-making process, and with the initial implementation of decisions. Decision-makers, assailed with often baffling signals from massive amounts of information, need researchers to cut through this complexity. They want researchers to say what the data really means, rather than to sit back and adopt a more detached, data-centric position. The information professionals who can add this value will be at a massive premium in the future.

From detachment to engagement

In response to the above demands, we have seen the arrival of a new approach to data analysis that represents a significant change from the original conception of market research. Market research began as a discipline based on the model of psychology, sociology, anthropology, and other social sciences. Its start point was the classic notion of 'research': detached, objective, and keeping as close as possible to the agreed principles of social science-based inquiry. It was a model that worked hard to differentiate professional survey research, from canvassing and selling (under the guise of research). This tradition gave us an industry with a sound set of research practices and a well-established code of ethics.

Factoring intuition into the analysis process

This approach was right for its time and the 'detachment' model has much to commend it. However, users of market research are now looking for an approach that is better equipped to handle the 'messiness' of today's data, and one that 'engages' more with the end decision-maker. Classic, objective analysis of single-survey findings is only the start point. Today's researcher has to both make use of the best of traditional survey methods and also embrace more intuitive inputs into decision-making.

The arrival of 'new' market research

In describing how 'new' market research is different from the 'old' modus operandi, it is helpful to think of market research as operating on the following four fronts. First, how the quality of each piece of evidence will be assessed (robustness). Secondly, the extent to which the new incoming information will be assessed relative to relevant and related past evidence (context). Thirdly, the techniques used to evaluate the meaning and significance of each item of data (evaluation). And fourthly, the way in which the research findings are presented to the client (application).

It is possible to characterize old market research as being represented by the inner shaded area of Figure 1.1. This illustrates how old market research typically functioned on each of the above four fronts:

Robustness: the emphasis, in the past, was on working with orthodox concepts, such as 'validity' (is the evidence measuring what we think we are measuring, and free from any systematic bias?), and 'reliability' (how likely is it that the data will hold good over time, and that we will be able to reproduce our results?).

Context: in the past, most market researchers would get no further than checking their new incoming study against, maybe, one past related research report.

Evaluation: this would inevitably focus on examining one data set and involve the application of (classic) statistical tests.

Application: the study would conclude with a presentation of the research findings, possibly with some recommendations for action (but would not be closely related to the subsequent decision-making process).

New market research takes us into new territory. This is summarized by the activities shown in outer white panel in Figure 1.1:

Robustness: the emphasis today is on 'compensating' for the imperfection in the varied data sources that market researchers now draw upon.

Context: the availability of marketing information systems usually means that new market research evidence will be set in a much richer context than ever before.

Evaluation: orthodox statistical analytical methods will be employed alongside frameworks aimed at factoring prior management knowledge (intuition) into the data analysis process, with this involving the analysis of multiple, not just single, data sets.

Application: new market research goes beyond simply presenting research findings and making recommendations, with market researchers now much more closely involved with decision-facilitation.

The methodological challenges for new market research

There are three distinct methodological challenges in building the new holistic market research approach outlined above:

The first focuses on finding actionable frameworks to help combine qualitative and quantitative evidence when tackling business problems.

The second centres on how to develop frameworks to incorporate management intuition into the formal data analysis process.

And the third involves synthesizing what we know about the overall market research 'craft' - what we know from experience does and does not work - into a form that can be made accessible to data analysts, so that this can enrich their interpretation of the data.

We briefly examine these three issues below, but we also return to these major themes throughout this book.

Integrating qualitative and quantitative data

Holistic researchers tend not to think of qualitative and quantitative research as separate disciplines. The emphasis is on finding ways to integrate the two forms of evidence. Holistic researchers recognize the power of the rigorous statistical analysis of quantitative data, but they also see merit, on occasion, in analysing quantitative data in a qualitative way. Similarly, the holistic researcher understands the benefits, where appropriate, of not only examining qualitative data in a thematic way, but also subjecting the evidence to a more quantitatively-orientated analysis.

Distinguishing the qualitative 'method' from the qualitative 'mode'

In interpreting the increasingly blurred methodological boundaries between qualitative and quantitative evidence, it is helpful to draw a distinction between the qualitative 'mode' and the qualitative 'method': that is, to explicitly delineate the idea of the qualitative mode of analysis from the qualitative method of data collection.

Most are quite comfortable with the - albeit blurred - distinction between qualitative and quantitative data collection. But what happens to that data is a different matter. We argue that the qualitative mode - an open and flexible way of thinking about data - should not be restricted only to the qualitative method. It should be extended to apply also to the quantitative method.

Certainly, the idea of defining 'qualitative' as a way of thinking that can be applied to all forms of data is consistent with dictionary definitions of the terms 'qualitative' and 'quantitative':

Qualitative: involving, or relating to, distinctions based on qualities, constituents, or characteristics.

Quantitative: involving, or relating to, considerations of quantities - amount or size.

In sum, the qualitative mode is a powerful concept, one that should not be restricted to the qualitative method. It can add insight to almost any piece of information. In fact, there is no reason why the qualitative mode of analysis cannot be expanded to encompass all forms of marketing evidence. In the matrix in Figure 1.2, we illustrate the way the qualitative mode of thinking can apply to either qualitative or quantitative data.

Many will argue that, working across the qualitative/quantitative 'divide' is what they already do, and in some cases, this may be true. But the point is that the qualitative mode has only been sporadically articulated as a skill set in its own right, separate from the business of collecting data - the qualitative method. We believe that in promoting the holistic school of data analysis, the articulation of our concept of there being a qualitative mode, not just method, concentrates the mind and helps us to define what holistic data analysis is all about.

This observation challenges quantitative researchers to bring the same level of attacking interpretation to the numbers as qualitative specialists routinely deliver based on fewer, but deeper, observations. The challenge to qualitative research is to take what we define as the 'qualitative mode of thinking' out of its rather introspective methodological box.

Developing analytical frameworks to embrace prior management knowledge and intuition

In the past, it seemed that business problems were tackled via two, almost mutually exclusive, channels of thought.

Continues...

---

Excerpted from The Art & Science of Interpreting Market Research Evidence by D. V. L. Smith J. H. Fletcher Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

---