What is a concept test?
It’s a tough job to manage the launch of your company’s products and services. You’re already facing several statistics that paint a grim picture of the reality of product launches:
- Only 55% of all product launches take place on schedule, and 45% of product launches are delayed by at least one month, according to a 2019 product manager survey from Gartner.
- More than 25% of total revenue and profits come from launching new products across industries, according to a McKinsey research survey.
- Preparing is the biggest problem when launching new products, according to a Harvard Business School report.
Under this pressure, you want to make sure your product connects with your customer before it goes to market. By doing the right product research, you’re checking that your assumptions about your customer are correct.
Concept testing is early market research that maximises the odds of you launching a product or service that the public wants to buy. You explore the viability of a product or service with its target audience early on, and improve its development from the feedback.
This uses your time and money wisely, so that you can learn actionable insights today and impress the public with the improved results later.
What concept testing is not…
Concept testing is not a replacement for the ideation phase of a development cycle. Instead, it tests the basic idea that comes out of the ideation phase – whether it is fully formed or not. In this way, your target audience might vary as the idea develops.
It’s also different from brand testing, advertising or marketing campaigns – these promote the product or service when it’s near-complete or fully developed.
What are the benefits of concept testing?
Traditional market research requires audiences to choose from a company’s predefined criteria to provide their feedback. It is one-way and ends when the company gets the answers they want.
Concept testing is different. It’s based on listening to the audience’s views and exploring how viable the concept is for them – without bias. You get their direct feedback on what they want, which might shed light on your blind spots. It’s an open learning process that goes both ways – you can continue to engage with the same audience to get their take on your developments as they happen.
It’s also a way to show good Experience Management. At a product launch, your brand’s perceived value and the strength of your offering is directly linked to your profit and loss. Likewise, a disappointing loss can hurt internal staff morale, investor relationships and future business ventures.
Poor test-and-learn execution has been known to hobble a company’s fortunes for years to come.
An essential tool for product managers and executives
Why is this an essential tool for product managers and senior executives?
1. Concept testing can be a cost-effective and flexible solution
You can set-up a simple and quick survey if you want high-level feedback, or you can delve deeper to understand more detail.
Concept testing is also a great money-saving technique as it prevents launching a faulty concept. And in comparison with the cost of outsourcing to external market research programs, internal concept testing research is cheaper.
2. Concept testing can help you optimise the product or service
The insights are evidence that can make you feel empowered to act with authority.
You can uncover or identify information that will have a true impact on your development decisions. Some examples are:
- How effective the concept’s brand equity, brand image and benefits are to the customer.
- The customer’s perception on the concept’s price range, buying preference and how they might use it.
- The concept’s status among its market competition.
- The concept’s weaknesses (E.g. Is there unclear communication, low customer value or an unmemorable product personality?)
You can also test different concepts at the same time to see which works best at solving your customer’s problem. If a concept doesn’t work, invest in the concepts that do make a difference.
3. Concept testing acts as a quality-assurance check over time
It’s common for there to be several rounds of concept testing, allowing product managers to check in with target audiences and optimize the offering.
Vary your audience to get different insights. You can use the same audience for reflections on your improvements, or a new audience to get fresh insights on your design and developments.
4. Research can help you build up good customer relationships
You can build brand loyalty and increase your brand equity value, by including your potential customers in the concept’s design and development.
As a brand, you look transparent and open to innovation. It’s also a great way to show you value the opinions of your customers.
5. Companies that product concept test can avoid costly consequences
In 1957, Ford spent the equivalent of $3.1 billion in the “car of the future,” the Ford Edsel. While Ford conducted consumer polls, they failed to implement feedback and many of the touted new features were unreliable.
Coors made a similar blunder when they attempted to enter the bottled water market in 1990. The packaging and branding of Coors Rocky Mountain Spring Water was similar to Coors beer, which frightened many of Coors’ target demographic.
Three examples of successful product concept testing
Some brands have leveraged product testing to make decisions that have tremendous payoffs:
- Emerging fashion brand Shinola used customer research to verify which watches to feature in their spring 2018 collection.
- Yamaha used concept testing to make a product design decision about using a knob or a sliding fader in a new electronic keyboard. Understanding customer preferences has helped Yamaha maintain its top keyboard in the industry.
- Chobani used product testing to adjust packaging in one of its new products, Chobani Oats and Ancient Grains. This helped the brand increase the user-friendliness of the product and maintain the top spot as Australia’s preferred yogurt brand.
Ready to make your own product concept test?
We’ve put together a step-by-step guide to get you creating the best concept tests for your products and services. We cover these topics:
- Choose the right concept test survey methodology
- Choose the right survey components for your concept test
- Choose the right flow for your concept test
- Identify the most promising concept
- Concept testing common mistakes and how to rectify them
Or, you can also identify your most promising product concepts using our Concept Testing Tool. It’s an expert solution that tests and reveals what people really think of your company’s concept ideas.
Step 1. Choose the right concept test survey methodology
What do you need to consider first to set-up a successful Concept Test? Before designing your concept testing survey, consider which methodology will best fit your needs. Some of the most common survey methodologies include:
- Single Concept Evaluation (Monadic) – Respondents complete a full evaluation of a single concept.
- Multiple Concept Evaluation (Sequential Monadic) – Respondents complete full evaluations for multiple concepts.
- Concept Selection (Comparative) – Respondents choose which concept they like best.
- Concept Selection and Evaluation (Comparative Monadic) – Respondents choose which concept they like best and complete a full evaluation of that concept.
Each product testing methodology comes with its own positive and negative trade-offs. Get a closer look at our pros and cons table below, here.
Step 2. Choose the right survey components for your concept test
Concept tests can easily be constructed once the researcher has determined the key components for the survey. Of course, these components will vary depending on the purpose of the concept study.
Carefully consider the objectives of your concept test to make sure the measures used will successfully answer your team’s questions.
The major components needed for a concept test:
- Measurement for the overall concept reaction
- Concept need or relative improvement over current method of doing things
- Overall reaction to the concept (acceptability, desirability, interest)
- Likelihood to purchase concept
Components for detailed concept analysis evaluation
- Likes and dislikes about the concept
- Attribute list evaluation
- Awareness of competing products
- Awareness of substitute and complementing products
- Superiority over other existing products
Components for use situation evaluation
- Likelihood of use in specified situations
- Current use of similar / competing products
- Frequency of product use
Components for value analysis
- Estimate product value
- Price sensitivity analysis
- Preferred method of purchase
Component for segmentation analysis
- Market segments most likely to use (order and prioritise)
Step 3. Choose the right flow for your concept test
The flow of your concept test survey ensures that audiences understand and buy into the survey they’re taking.
In the introduction section, make sure to introduce your survey in clear language. Qualify the respondent with a few suitability questions to make sure they fit your audience preference. Ask demographic questions to give the respondent initial easy-to-answer questions to get them started (you can use the results to also filter your results later).
Next: The main section. Here you can start to add in the core survey questions. For example:
Establish a baseline of respondent’s buying history
- What is your awareness of this product brand?
- What brands have you purchased (within the product class) in the past three months?
- Identify purchase dimensions: number of bottles, frequency of purchase, frequency of consumption in a day
Introduce the concept
Evaluate the concept as a whole
- What is the perceived value of the product concept?
- How innovativeness is this concept?
- Is this relevant to you?
- What do you think about the packaging?
- What do you want to be different?
Explore the concept’s individual attributes
- What do you think about this X attribute? (Continue for each attribute)
- What are the benefits? Further probing questions might be:
- Is it good value for the money?
- Would it help you to… (do something more easily or better)?
- Would it improve your health?
Compare the concept
- What’s the concept’s power to replace the current brand you purchase?
Ask about respondent’s buying response now
- What is your likelihood of purchase?
- Where might you purchase from?
- Do you believe in the concept?
End the survey by thanking the respondent. You could also use this opportunity to remind them that their data will be stored securely, and it’s only accessed by the research team.
Check out one example of a concept testing survey below. You can access this template for free, and begin using it immediately here.
Step 4. Identify the most promising concept
When you have carried out the concept test or tests, you’ll receive a lot of data back. If you’re using a turn-key system like the Qualtrics Concept Testing Tool, you can pre-assemble data together quickly into reports. If you’re sorting the data by hand, use a spreadsheet software like Microsoft Excel to begin to sort, filter, compare and pivot-chart your results. You can additionally try to pull information into a visual format with the in-built chart tools.
We recommend splitting your analysis into two sections:
- Overall results will give you a high-level view of which concepts performed the best.
- Individual results will let you dive into each concept and understand how they performed and why. This is where demographic segmentation (e.g. by gender or age) can help you make assumptions from the result, which can be tested further.
Where qualitative analysis for open-ended questions has been captured, try parsing out common themes, strengths, and weaknesses. Using a tool like Qualtrics Text iQ can make this analysis quicker.
Concept testing common mistakes and how to rectify them
Some common mistakes when creating your concept tests are:
Mistake #1: Running one concept test on an idea in isolation
You choose to run a concept test to get your initial insights, but stop there. This might provide you with the answers you wanted, but you may be missing out on the full benefit of comparing research data over time.
Solution: In advance, think about what you’re measuring and how you’ll use that information when you get it. This might direct you to use a different methodology or suggest whether you need to repeat the exercise to see if your product’s development is on the right track.
Mistake #2: Over-stuffing lots of ideas into one concept test survey
You may want to use the tool to compare lots of ideas in one survey, and ask a long list of questions for each, to save time. Even if you explain this in the introduction, it could make your target audience’s experience long, tiring and confusing.
Solution: It’s important to respect the time and energy of the targeted audience. For one idea, use the Monadic approach, which goes into depth on a single idea. For multiple ideas, try using multiple surveys that explore one idea each, and think about how many survey questions are reasonable to ask. This way, your target audience is clear about the focus of the survey and more likely to finish them.
Mistake #3: If you don’t get the answers you wanted in the concept test, you automatically cancel the idea
You find out that there is no demand or interest for your product idea with your target audience. That could be a reason to stop investing your time and energy into the product’s development. However, this may be short-sighted as there may be potential profit in changing it.
Solution: The purpose of the research is to see what target customers think and feel about the idea. If it’s negative, then it could mean that the target audience is not interested at this time, or they are not the right audience. You may find more by testing more widely, or trying again after some time. A bad result doesn’t mean it’s a bad idea.
Mistake #4: You assume the audience thinks like you do
You’ve got an in-depth understanding of the product idea, and you want to share this in the survey introduction. You may also want to share marketing and brand ideas now, as you’ve been thinking of them already.
Solution: Avoid using technical jargon or acronyms that won’t be immediately understood by your audience. It can be intimidating for survey participants and cause confusion.
Also, avoid making a strong marketing pitch. Adding this information now can suggest the idea is set in stone – remember that marketing and brand activities come in at a later stage when the idea is close to or fully formed. This could also prevent the opportunity for honest feedback about what they like or don’t like.
Mistake #5: You don’t adjust the survey to cater to all geographies
You may have an audience you haven’t considered. Not trialing the concept test in different countries could mean you miss out on expanding your product idea’s reach.
Solution: Where you want to widen your audience pool, consider translating the survey into the native language. In addition, you can use images as survey answers, displaying them together to see which is preferred.
As we’ve seen, concept testing is more than getting feedback using a likert scale. It involves considering the methodology, components and flow of your survey for the maximum impact.
When done right, it saves a lot of time, money and energy. Most importantly, this essential, money-saving tool lets you know you’re on the right track, and gives you the insights you need to move forward with confidence.