The complete guide to product testing

What is product testing?

Product testing is a research methodology that involves investigating potential customers’ reactions to a product or service being developed, before launching it onto the market. Typically, a sample of your target audience uses, tries out or consumes the product and tells you what they think of it. You use both quantitative and qualitative data to collect and analyze their feedback as part of the product testing process.

Why is product testing important?

If you don’t test a product before it goes to market, there’s no way of knowing how reliable it is, nor how well it’ll cope with ‘real world’ challenges.

The fact is, while products are developed to solve certain problems and used in a certain way, customers may end up using them differently — adding their own habits in the process.

With this in mind, you have to ensure that your product is robust enough to survive under real-world conditions and flexible enough to accommodate different methods of operation.

For example, car manufacturers use product testers and product testing sites to not only test the car itself, but also expose it to various real-world scenarios. This way, they can test the structural integrity of the car in the event of an accident, and how it might perform in various conditions.

The reason they test the car in this way is because it’s made up of thousands of components, each one essential to its operation. These components must meet demands over tens of thousands of miles without faltering, and also withstand the pressures of different driving styles (some drivers will be more aggressive than others).

Product testing each individual component ensures they can individually withstand the environment they’re in, and then testing the full product (the finished car) ensures your component and final product will also meet the standards you expect.

Here are a few of the main benefits of product testing:

• Ensure quality and reliability in your products: If you test products rigorously before launch, you can be sure they’ll be able to stand-up to ‘real-life’ situations and you can identify defects early on
• Reduce overall costs: Once you’ve started to develop and test products with customers you can start to see how they work in a real-life scenario and ultimately identify how to develop products while reducing costs
• Understand your product’s life expectancy: Using the car example again, it’s important to understand how long your product is likely to last before you experience problems or customers require support. Once you understand your product’s lifespan, you can start to identify where the likely maintenance stages are going to be.
• Reduce your customer’s aftercare needs: Again, product testing allows you to understand potential problems with your products and where you might need to conduct maintenance, or understand problems and fix them before launching the product. Using proper product testing you can reduce the need (and costs) of customer aftercare.

Ultimately, product testing can have several benefits, and you should aim to test products using qualitative research, quantitative research, focus groups and technical product testers. Incorporating a comprehensive feedback cycle, composed of these elements, will enable you to get the insights you need to improve your products.

The best product testing companies implement feedback cycles at every stage to ensure that their products meet the needs of their target market!

Get the concept right – from the start — with effective product testing

Concept testing, which happens before you’ve started designing and creating your product, helps you explore how viable your initial product idea is.

You’ll be able to ‘take the temperature’ of various concepts by running them past test panels of multiple potential customers to review products before you invest substantially in developing them. You could even hire the services of a product testing company to test your concepts.

Using online surveys, you can gather target audience feedback on one or more of your product’s rough concepts either before or after it goes through the hands of a product tester. Those concepts that particularly resonate with your product testing audience can then be further developed into something closer to the finished product.

The refined product can then go back out for retesting.

Find out more on idea screening and product concept testing

Conjoint analysis is your best friend

Conjoint analysis is a powerful method for comparing multiple product attributes during the product testing process.

You will have used it to ‘compare similar products’ when shopping online, choosing which features matter most to you or your customers.

Using surveys, target audience respondents trade off the different product features and choose which ones are most valuable to them.

There’s a wide range of conjoint analysis methods available, to help you collect and assess your data in different ways and with different levels of detail.

Depending on the variant, you can ask respondents to:

• choose between profiles
• rank them in order
• rate them on a scale

You can show them the complete range of profiles to choose between, or sets of two at a time.

Which conjoint analysis method you choose will depend on how many product attributes you are working with, your data collection method, and how robust your results need to be.

Self-explicated conjoint analysis is one of the simplest and most robust conjoint analysis methods. It’s built into Qualtrics CoreXM as a standard feature.

Self-explicated conjoint analysis is a three-step process.

Respondents:

1. Eliminate features that they would never accept in a product
2. Evaluate the desirability of the remaining features against each other
3. Rank the most desirable levels of the attributes according to their importance

Using research to choose product features

Once you’ve developed your product concept, you’ll need to find out which features your customers truly value, and which ones matter more than others. This is known as customer needs analysis.

Because it’s the customers who will ultimately determine your product’s success or failure, its features should ideally be a direct reflection of your customers’ requirements.

Means-end analysis is a powerful method for assessing what customers really want in a product. It begins with a survey to gather information about a customer’s purchasing decisions.

The survey will gather responses on subjects such as:

• price
• perceived value
• performance expectations

It can also begin exploring deeper-level benefits such as:

• status
• trust
• your brand’s compatibility with a customer’s personal values

Depending on the depth of your study, you may choose to include survey questions on:

• Usage context– this includes the physical location of your customer, as well as the ‘mode’ they are in when using the product (relaxing, socializing, working, etc). It’s highly valuable for marketing and advertising, as well as feature research.
• Top of mind imaging– this is a snapshot of a customer’s positive and negative associations with the product concept and your brand, and the reasons they give you for those associations.
• Product category– this involves placing the product in context with its competition. You can ask the respondent to group similar and dissimilar choices together or to consider substituting one brand or product for another. You note which features make a product more or less ‘swappable’. Product category process is also useful in your competitor analysis program.

How market research helps with product testing

As well as researching your product concepts, using a panel for product development and testing research, e.g. how well it works, key features and functions required, general thoughts and more, can help drive your product’s success.

Using online surveys to help with product testing

Asking product testers to rate your product by completing surveys is a great way to uncover the general thoughts and feelings towards your products.

A product test could involve using the same products (with all the same features) and asking customers generally what they think and where the improvements could be. But they could also involve customers being asked to test items using different variations of your product and asking product testers which they prefer and provide feedback.

Just be wary that some people who are paid to test products could have some bias and may give answers based on what they think you want to hear (so ensure you provide incentives for honest criticism).

Using qualitative analysis for product testing

As well as completing surveys and generating quantitative responses, you should also look to get some qualitative responses from real customers about their specific experience of using your products.

Cosmetic products are often tested using focus groups, and tech companies might invite a small group of consumers to test new products, such as a new mobile device or laptop to get their feedback. This qualitative input can help you hone your product ideas.

Use qualitative and quantitative research when testing products

Ideally, your product testing market research should use a combination of qualitative and quantitative analysis. To get insights at scale, you could send out a survey — as mentioned above — and then validate your findings with a focus group. Alternatively, you could do it the other way around, invite a sample group to assess your product and then see if their opinions are representative of the whole.

You could incentivize participation by offering free products to those who complete the survey. This practice is well-established in the cosmetics industry as brands often send out beauty products or free samples to focus groups for the evaluation.

Of course, incentivizing testing products by offering free products can influence whether or not you get honest reviews, so exercise caution and don’t give away too much free stuff.

Defining your price point

You’ve developed a winning combination of product features, and your research shows the concept is really hitting the mark with your target customers. But how much would they be willing to pay?

This is your price point: it’s your product’s suggested retail price, set to be competitive with other products, and able to be altered depending on competition and demand. The factors involved in pricing a product go way beyond knowing the target customer’s budget.

Pricing research uses a wide range of techniques to assess responses to price, from direct questions about the monetary value to more nuanced tests that compare acceptable prices in different buying contexts.

Psychological factors are also key. Price a product too high, and a respondent won’t even think about buying it. Too low, and they will question its quality.

Find out how to price your products for maximum profitability

Use great surveys, develop great products

To get the right insights from your product testing surveys, you have to use the right tools, strategies, delivery methods and questions.

From question format to flow and funneling, the way your product testing survey is designed will directly influence the quality of data you collect. Make sure the tools of your trade are as carefully crafted as your product experience.

How to put together a successful product concept assessment using Qualtrics

Product testing is more than just giving your product to a few customers or a product tester to use for a while and report back. It needs to be a thorough and defined process to ensure you get the results you need.

From understanding the aim of the product testing and concept assessment to understanding the types of tests you can carry out and how to get the best results, product testing requires a lot of planning throughout.

And this is where Qualtrics can help.

Whether it’s concept testing, panel management, market assessment, conjoint analysis, or survey design, with Qualtrics product experience management software, you have a single system of action for everything you need: ProductXM.

With ProductXM™, you can accelerate your product development cycle with experience data on one platform, you can also quickly spot gaps in the market and launch new products as a result.

Take advantage of powerful, easy-to-use surveys to uncover how your target market feels about your products and concepts, and then use that information to refine your offerings and your overall product development process.

At the same time, with our ultimate listening tools, you can capture customer feedback — wherever it is — and turn every comment into an opportunity to deliver a better product experience.

Finally, leverage smart analysis — including concept testing and conjoint analysis — to incorporate and act on user feedback at every stage and launch your products with certainty.

And if you need expert support, the Qualtrics research services team can help you to uncover deeper and more targeted insights from your data.