How a Design Agency Turned Creativity on Its Head With Conjoint Analysis & Ad Testing
UK-based design agency Linney is making waves in the industry with a new approach that uses conjoint analysis to give their clients certainty when it comes to knowing how their campaigns will perform.
If you’ve ever worked at a creative agency, you’ll know the scene. You’re gathered around the table at a pitch, the team has worked through the night, every night, for the last 3 weeks and finally, you’re all really excited with what you’re about to show your client. Your plane ticket to Cannes is booked and your dinner jacket is at the dry cleaners, ready to grace this summer’s award ceremonies.
There are two creative routes on the table, but they’ll LOVE the first one. They must.
You all love it – it’s bold, it’s on brand and it tested so well with that focus group of 10 consumers you showed it to.
But alas, they choose route two – the sacrificial lamb only ever on the table to draw them towards the first one.
We’ve all been there – many of us on multiple occasions.
But at one creative agency in Nottinghamshire, UK, those days are becoming far less frequent since they brought design and data together in their creative process.
Linney is a multi-channel marketing agency with a client roster most would dream of – think McDonalds, PayPal, Carlsberg, Mars, and Nivea to name a few.
And in their creative agency they have a secret weapon that’s helping to make days like that a thing of the past – an in-house research team with the express desire to push data analysis into creative territory.
“I’ve always been interested in where the world of data science meets the world of creativity,” says Matt Geeleher, Quantitative Research Lead at Linney.
“In the past, creative has been very much a subjective area, but we were really interested to go beyond that and think about the business outcome: If we choose this design, how will that affect sales? If we redesign the packaging this way or that way, will people pay more for it? There are so many questions we can answer before the work even goes to the client.”
CREATIVITY MEETS DATA SCIENCE
Linney started small, armed with its new research team and a Qualtrics license. Using Qualtrics, they would create up to 10 different designs with each project and test them with the target customer to see which worked best.
Come the client presentation, they could go in armed with data to show how the design impacted the metrics their clients cared about most from intent to buy and click-throughs to the price their customers would be willing to pay.
But then came a watershed moment when they introduced automated design software into the process.
“Our clients are wowed by it,” says Matt. “When you have a client with numerous key stakeholders, each with their own opinion on what works and what they want, it can be hard to manage – but when you show them ‘this design will increase brand recognition by 6%’ or ‘this packaging will allow you to charge 12% more and still maintain your sales figures’ it’s pretty compelling.
“It starts with a few creative routes and then the software breaks each one down into its composite parts – say a headline, hero image, price roundel – and automatically creates multiple variations of each one that we can test in Qualtrics.”
It wasn’t just a step forward, it was a leap – the team at Linney could scale their research to test millions of variations of a design to get to the perfect one.
“We used to only be able to test about 90 concepts,” says Matt. “But with Qualtrics we can scale that up to tens of thousands.”
A campaign for a major high street retailer for example saw 23,468 poster designs tested. Carlsberg tested 10,800 designs as they looked to revamp their SKU images given to supermarkets to use online.
Using Qualtrics, Matt and his team feed the output from the automated design software into the platform to create conjoint analysis models – a statistical method that allows them to test how people value different attributes.
The results then allow Linney’s creative team to build the perfect piece of creative, piecing together the right combinations of different composite parts to get to a final design that they can be confident in.
“Conjoint analysis really allows us to link the creative back to behavior – it tells you exactly what’s driving the consumer’s decision and it quantifies it.” says Matt.
THE FISH WITH THE MILLION DOLLAR SMILE
Have you ever seen a goldfish smile? Thought not.
But take a look at the packaging for Aquarian Goldfish Food next time you’re in a pet store and you will. Plus, you’re going to pay an extra 10% for the pleasure.
It’s all down to the work Linney did with Aquarian’s parent company Mars last year. Hired to come up with a new packaging design for the fabled fish food brand, Linney turned to a conjoint analysis tool to test various iterations of the package design and the price consumers would expect to pay.
What they found was that of all the elements tested, one simple change had the most surprising impact ‑ when you Photoshop a smile onto a goldfish, consumers perceive the product to be worth more than when the fish has its usual (and natural) pout.
DESIGNING THE PERFECT SANDWICH WITH EXPERIENCE DATA
Another great example comes from Linney’s work with one the the UK’s biggest retail chains to identify the perfect line-up of pre-packaged sandwiches – a market worth some £3.3bn.
When you think of supermarket sandwiches your mind goes straight to the classics – BLT, New Yorker on Rye, Ham and Mustard, and Egg Mayonnaise. Look beyond those however, and there’s a whole untapped world of sandwich combinations – 2 million of them to be precise!
So Linney created 2 million iterations of the humble sandwich, with different combinations of fillings and different types of bread creating the mother of all conjoint projects.
Of course, not all of these were tested – when modeling with conjoint analysis they don’t need to test every single combination, but can focus on the individual elements and allow the tool to use that to build the ideal one.
And so the list was whittled down, and the Linney team was able to test hundreds of sandwich combinations, identifying which combinations consumers liked and how much they would expect to pay for different ones.
Conjoint methodology cleverly presents sandwich options and asks respondents to choose their favorite. After a few choices, advanced modeling creates a profile of each respondent’s preference of ingredients.
The survey experience replicates a real shopping experience, asking consumers to make a quick decision between the options, and the analysis provides powerful insight.
“Most shopping decisions are grab-and-go. It’s quick, subconscious system 1 thinking, and if you asked a customer to analyze why they made their choice, you’re looking at it the wrong way — they won’t actually know. With conjoint, they don’t need to know because the tool automatically does that based on the choices they make in the moment,” says Matt.
The results proved invaluable – they had a crystal ball into the future behavior of the target consumer, allowing them to craft the perfect line-up of sandwiches in store that they could be sure would sell well.
TAKING CONJOINT EVEN FURTHER
Linney is only just getting started on its mission to bring data science and creativity together. Going one step further, they’re now starting to look at applying conjoint analysis to animated video.
The same theory follows – break it down into all its constituent parts, create multiple variations and test, test, test!
It’s a formula that’s helping them add value to their clients and put the business outcome at the heart of creative decisions – an approach that’s setting new standards for the creative industry. If you want to learn more about ad testing, download our ebook below!
eBook: Test and Launch Effective Ads