On Thursday July 22nd, 2010 over 118 participants signed up for Survey Analytics and Dorian Simpson of Planning Innovations presentation on: How to Use Conjoint Analysis In the Innovation Process.
Conjoint Analysis is a powerful and often under-utilized marketing research tool that can provide powerful insight into how your customers actually think. The resulting information can be used to prioritize features, develop pricing strategies, and estimate market share… all before you develop your product or spend valuable marketing dollars.
Participants posted the following questions and both presenters, Dorian Simpson of Planning Innovations and Esther LaVielle of Survey Analytics, responded to each one.
1) What are new innovative ways to gather data and analyze it using Conjoint Analysis? What kinds of tools are available in market to perform conjoint analysis?
SA – At Survey Analytics we offer a robust yet easy to use Discrete Choice Conjoint Analysis tool. Guidelines are provided to ensure data is concise and accurate. We also provide a market segmentation tool, which offers you an opportunity to “test” new product ideas against your current data to help predict possible market share.
https://www.surveyanalytics.com/conjoint/index.html
2) Many times consumers don’t take surveys etc seriously and just complete surveys for the sake of it. How can we take that into account when applying CA?
DS – It’s important in the lead in that you let the respondent know that YOU are taking the survey seriously and that you would appreciate if they do also. This is less of a problem if you’re using your own databases. You should also try to screen out responses that are obviously completed just to finish, such as never varying their response.
SA – I agree with Dorian. Respondents always appreciate an introduction that is upfront with your intentions. Be honest with how long it would possibly take and provide an incentive that appeals to your targeted sample. In my experience of working with internal databases, you will become “familiar” with those who are not truthful or do not take your surveys seriously and can remove them from future surveys.
3) What should be the minimum sample size to conduct conjoint…any lower and upper limits and implications of sample sizes…for calculating utility values…(help me on the sample size limitations) ?
SA – This depends on your target market. The larger your target market, the larger your sample should be for statistically significant data. The general rule of thumb for Conjoint Analysis is usually a minimum of 200-300 completed surveys. This, however you can go down to 100 completed surveys if your target market is relatively small.
4) Most use cases of conjoint focus on consumer electronics/durable goods. Is there a case for using conjoint in the FMCG/CPG industry?
SA – There is an example of a packaged goods study- Trail Mix: https://www.surveyanalytics.com/t/ADvnXZIA2S
As you can see in the results Dry Fruit had the highest relative importance compared to other ingredients whereas Nuts Type 1(sesame seed and sunflower seeds) did not make an impact on choice.
5) As attributes and levels are important in conjoint what should be appropriate “no” on attribute levels?
SA – It would depend if the feature is something you may want to add or not.
For example, if you wanted Trail Mix with/without Crackers you would set up the following:
Features: Crackers >> Level: Yes, No
6) Did you ask the “why” questions such as frequency and power questions in a study after the conjoint study?
DS/SA – It has been investigated in other research and will be tested again further.
7) How is conjoint used in the launch of a service business versus a product launch?
DS – Can be used similarly. E.g. Instead of price it may be price/mo., etc. You must identify attributes and levels similar to a product.
SA – A fun example is a hair salon. What kinds of services will you offer to your clients and at what price do you think they would pay for it?As Dorian said you must identify attributes and levels similar to a product.
8 ) With 6 attributes and multiple levels, how long was the [example] survey? I assume that you used experimental design to shorten the length of the survey?
SA – The case study survey that was used during the presentation took respondents on average 15 minutes to complete.
9) Do we see conjoint analysis used often in the food industry…specifically for product development?
SA – Yes. Conjoint Analysis can be used in any industry that is interested in doing a trade-off analysis of some type. Whether it is on a medication a pharmaceutical company is trying to develop or a new kayak model that would appeal to families with young children, Conjoint Analysis can be used to provide guidance in those industries.
10) Is there max levels else of options to ask in choice task?
SA – The minimum is 2 levels per feature/attribute. The standard is to stick to no more than 3-4 levels per feature/attribute. Every once in a while going up to 5 may be needed depending on the feature needed to be test.
11) Can I use conjoint in B2B surveys where sample size will be usually low almost 100 customers. In what cases can I use?
DS – You’ll want to keep the number of attributes and levels reasonably low.
SA – The fewer the respondents being surveyed the fewer attributes and levels should be used. At this point in your research you should have highly defined features and levels that would fit your targeted sample size.
12) Is there any relation between the number of attributes and minimum number of respondents required to get the results from Conjoint Analysis?
SA – From a technical standpoint, the system does NOT impose any limitations. You can have unlimited attributes and unlimited levels within each attribute.
However, from a practical standpoint, it is unreasonable to have more than 4-6 attributes, and about 3-4 levels per attribute. Our suggestion would be to keep the number of attributes to under 5 and try and seek about 3 levels for each attribute.
13) What is the ideal task count?
SA – Our experience has shown that there is a precipitous dropout rate after about 15 tasks. Unless there is a strong personal incentive for the end-users to complete the survey, we would suggest keeping the number of tasks to fewer than 15 especially in cases where users are volunteering to take surveys. Please keep in mind that conjoint product selection is a little more involved than simply “answering a survey question” — users have to comprehend each of the attributes/concepts and then make a choice.
On the lower side, we would suggest that 5-8 tasks be the minimum for a conjoint model with 3 attributes. The more attributes you have, the more number of tasks users has to fill out.
14) In the cases study in the webinar price was one of the key features. I didn’t get how the results are interpreted. Can you explain it again?
SA – See screen shot below:
15) When you say “market share”, you mean “share of preference”, right?
SA – Yes, that is correct.
16) Don’t we need any intelligence in the tool when designing the conjoint study? The tool may generate a profile which has worst features but its price is highest.
DS – This is true, but this is part of a conjoint analysis to understand what your customers deem which attributes and levels are the worst. I don’t think you want to limit options for a high price and low attribute levels.
SA – We have built intelligence into our conjoint tool such as the prohibited pairs tool to ensure certain combinations that are not possible will ever show up. We must be careful in using this tool because the idea is not to limit the profiles based on what the client will not do, but to find out what resonates higher with your audience. We also provide a concept simulator that will calculate the number of times an attribute will be shown given the approximate number of people who will complete the survey.
17) I have QuestionPro enterprise account. Do I have access to conjoint analysis tool?
SA – No you do not have access to this tool. To access the Discrete Choice Conjoint Analysis tool you must upgrade to Survey Analytics.
https://www.surveyanalytics.com
18) Is it possible to get a copy of the slides from the presentation?
Slides from Survey Analytics:
Slides from Planning Innovations:
http://docs.google.com/present/edit?id=0ARLS1YfnuC-fZGhzYzVnamdfMTIxaGI2cDZ2aGI&hl=en&authkey=CJeq3dgN
19) When you changed the price for the subwoofer [in market segmentation simulator], you used $800 (a level that existed in the study). Can you use a price that is not in the study, such as $900?
SA – No you cannot. Before begin starting the conjoint analysis tool it is critical to have highly defined features in attributes before starting. In the case of our example in the presentation, we have established that $600, $700, $800 are the price points to test. Any other price points would require another conjoint study to be run before using the market segmentation simulator.
20) What kind of conjoint analysis is better: adaptive or choice based?
SA – This depends on what you are interested in retrieving data for. If you are looking for data that mimics the purchase process then the Choice Based (Discrete Choice) Conjoint Analysis is the better bet. Adaptive Conjoint Analysis (ACA) is a computer-administered, interactive conjoint method designed for situations in which the number of attributes exceeds what can reasonably done with Choice Based Conjoint Analysis.
Survey Analytics specializes in Discrete Choice Conjoint Analysis.
21) If we have a product with 15 attributes and 5+ levels, is there a way to figure out it?
DS– It’s important to focus on the most important attributes that really drive decisions. You’ll probably want to do preliminary research such as interviews, focus groups or short quantitative to narrow it down.
22) You have said market share is a result from the utilities obtained, but this isn’t correct. Is it?
The market simulator uses aggregate utility values to project the probability of choice and hence the market share
23) What other kinds of conjoint exists? How can I compare advantages & disadvantages with method seen today?
SA – Here are other conjoint tools that you can review and compare:
Adaptive Conjoint Analysis
Choice Based Conjoint Analysis
Discrete Choice Conjoint Analysis * Survey Analytics specializes in Discrete Choice Conjoint Analysis*
Full profile Conjoint Analysis
Adaptive Choice Based Conjoint Analysis
24) In your experience what was your worst result in a conjoint study? I mean when results were not logical or useless?
DS– Can’t really say. Most studies have had some “interesting” results that needed further investigation, but they’ve never been useless unless the company didn’t do the upfront work to understand the right attributes and levels.
SA – With Survey Analytics we appoint a dedicated account manager who will help with conjoint studies to ensure statistically significant data.
About the Presenters:
Dorian Simpson founded Planning Innovations in 2002 to help technology-driven companies launch successful products and services through focused innovation management and planning. He has significant experience in both engineering and marketing to help build the bridge between these two critical innovation functions.
http://www.planninginnovations.com
Esther LaVielle is a Senior Account Manager at QuestionPro and Survey Analytics, which was started in 2002 in Seattle and is now one of the fastest growing private companies in the US. Prior to her adventure at QuestionPro she spent 3 years as a Qualitative Project Manager at the Gilmore Research Group.
https://www.surveyanalytics.com
