Giving Your Research Data a Good Workout
Have you ever wondered whether you’ve managed to squeeze the last pips out of your research data? Considering the human efforts made and expense we often incur reaching respondents and persuading them to answer the burning questions of the day, this is often a moot point. The truth is that better ROI from research can be accomplished easily, with a little foresight and small contingency in the analysis budget.
There is no doubt that data tabulations and top-level reports are useful for summarising the overall message. Percentages of those with a certain opinion and behaviour are summarised and compared across subgroups, average (mean) scores on attitudinal scales are compared. Questions are typically evaluated one at a time. While this approach has its merits — it is easy to communicate and can be used to summarise key take-outs — it is easy to miss subtler relationships which can really enhance our understanding of a topic.
Additional lines of enquiry which add value include:
- What attitudes and behaviour drive survey respondents to answer a question in a particular way or give high/low ratings? How can these be prioritised? What is the nature and direction of the relationships between these responses?
- What combination of subgroups is most/least likely to possess some characteristic?
- How can decision makers be grouped according to how similarly they respond to a series of questions?
- How can the similarities between a set of question responses be summarised?
- How are certain characteristics and perceptions related to certain brands or products? Which brand/products are seen as similar/dissimilar?
All of these can be easily accomplished with a little foresight and planning. The analysis techniques employed use statistical methods to investigate the direction in which question responses move around their average (mean) score. They are concerned with similarities and differences in the direction of question responses, rather than averages within groups.
This is very important for identifying underlying behaviours, which may be captured by a collection of question responses rather than a single response. For instance, you might determine that decision makers, who buy direct from a particular channel, might have specific technology requirements, be price sensitive and consider themselves technology literate. Other decision makers might come from different sized organisations and be willing to pay a premium for a different channel, with more consultancy sales techniques.
Reducing Your Data into Something More Useful
This kind of profile can be created using a collection of analysis techniques which can generically be called “data reduction” – which reduce and summarise a large collection of data points into underlying themes. This data reduction can involve inferring “composite” measures of an underlying attitude or behaviour from a larger number of actual responses.
Commonly, respondents can be clustered into groups based on these underlying responses to create new segments. This is the approach used in the illustration below:
This is from a hypothetical customer satisfaction survey for a Software Solutions Vendor, using completely made-up data. We have clustered on a series of questions looking at the level of satisfaction with the service, frequency of use, level of ICT literacy and general service requirements. In this case a 5-cluster solution was seen to best fit the data collected. All clusters are inferred from natural relationships between the questions.
We can assign names for these different “typology” groups and formulate strategies for each. For instance, the Vendor might investigate how they can improve satisfaction in Clusters 3 and 4, given that these are regular purchasers and potentially high-value accounts. It might also look at ways of offering better value to Cluster 2, who are more likely to look for a “no-frills” service. Cluster 1 is also of great interest. It is the largest cluster, has a lot of good will, but is not buying frequently. Targeted communications around “added value” services could tap into this segment.
Rather than creating new segments or clusters, the goal might have been to create some composite measure of perceptions of added value-ness (for want of a better phrase) to determine the extent to which this drives satisfaction with the service. Other techniques such as Factor Analysis and Latent Class Analysis facilitate this. This kind of composite measure can be useful as a summary or to rank customers on this scale, for targeting with particular communications perhaps.
Prioritising Business Improvements
Another common business need is to determine what needs to be done to drive up some measure of business success, such as overall satisfaction or sales. In this instance, the question is “What buttons do I need to press to drive satisfaction (or sales) up?” In other words, “Which areas should we prioritise for budget investment, given limited resources?” Moving to a research agency example, and assuming we have customer satisfaction data on individual projects, we might be facing a situation similar to that shown below:
The questions are — should the budget be invested in better training for the researchers, improving turnaround time, sharpening up our market knowledge, or improving our pricing …. etc? The potential drivers might be responses to attitudinal agree/disagree scales of the type:
TO WHAT EXTENT DO YOU AGREE?
“the results were delivered in a timely manner” on the following scale? (circle response)
- Strongly disagree
- Tend to disagree
- Neither agree nor disagree
- Tend to agree
- Strongly agree
Or the indicators themselves might be composites across a number of related questions. Again we are interested in whether movement in these indicators around their averages, from case to case, influences movement in overall satisfaction around its average. For instance, does positive response across the scale for timeliness cause a corresponding increase in overall satisfaction?
A family of techniques known as Multiple Regression is used to answer these questions. It establishes the extent to which measures are linearly related to each other. Given that the indicators are often related to each other as well as the measure of performance, we need to partition the impact of each on overall satisfaction into a unique and share impact. The results can be converted into a straightforward bar chart as shown below.
This kind of analysis can be carried out across all customers, or just across particular subgroups of interest. In this instance, it is obvious that market knowledge, price and added value are key to higher satisfaction. Improving market knowledge would have the biggest impact though. If we were looking for a single priority, this would be the one. This technique is commonly known as Key Drivers Analysis.
Getting More Sophisticated
We have illustrated just a couple of examples of the simpler techniques available for use with market research data. It can get a lot more sophisticated. Perhaps we might want to better understand the relationship between employee satisfaction, customer satisfaction and business performance for different units? It is possible to build very complex models building on these and other techniques, but still present them in an intuitively easy to understand way.
We might also want to understand how to maximise the share or revenue from a product, service or process, through optimising its design and constituent parts. There are other techniques available for modelling discrete choice “buy/not buy” responses which can be brought to bear on this kind of problem.
These are perhaps better left as a subject for other articles, but all are achievable through survey research, with often only a modest additional investment.
Data Mining as a Lifestyle
I strongly believe that data analysis isn’t just a back-end process. The questionnaire and data collection methodology need to be choreographed to ensure that all of the useful analysis tools in our kit can be brought to bear in the reporting process. However, it is still surprising, considering the total costs of a research project that a modest allowance often isn’t built in for simple data mining techniques, which can add so much value to the research.
Given the scrutiny that marketing budgets have faced over recent years, there is a convincing case for spending just a little more, given the huge potential returns from the valuable knowledge that can be gained.
This article was written by Gary Bennett, Managing Director of Logit Research. Gary works with Business Advantage as a market research and statistical consultant.