Despite the increasing amounts of time and money spent on research and data, the truth is that most people still make decisions based on gut instinct rather than evidence.<\/p>\n\n\n\n
But making choices driven by personal motivation is far from the best option for brands looking to grow. In reality, it risks making their proposition worse.<\/p>\n\n\n\n
A\/B testing is often used as a way to validate changes a brand was going to make any way \u2013 changing design elements, testing headlines, or even adding new functionality.<\/p>\n\n\n\n
But its potential is far greater. Experimentation lets brands test out their boldest ideas \u2013 that they may be otherwise too nervous to roll out.<\/p>\n\n\n\n
That\u2019s why A\/B testing has become a way to inform product strategy, and it\u2019s relatively quick and easy to do.<\/p>\n\n\n\n
In short, it involves creating different versions of a brand\u2019s website to see what customers respond best to, and takes away one of the riskiest elements of launching a new product.<\/p>\n\n\n\n
Many brands spend months or even years developing a product and bringing it to market, just to see it fail miserably because it wasn\u2019t what people wanted. Even with research to back it up, brands often realise that the insight may be out-of-date or misleading.<\/p>\n\n\n\n
By testing it out on audiences first, the customer unknowingly plays an active role in the decision-making process, helping brands determine what works and what doesn\u2019t.<\/p>\n\n\n\n
Testing multiple ideas at the same time allows us to pick the highest performer, and if something doesn\u2019t work, we can turn the test off without really having lost anything.<\/p>\n\n\n\n
GAIN<\/a> recently conducted research into buying behaviour for British shirt maker T.M.Lewin. The brand has always offered a number of ways for people to customise their shirts, including the somewhat baffling \u2018sleeve length\u2019 choice.<\/p>\n\n\n\n I say baffling, because in all honesty, how many men in the UK actually know what their sleeve length is? According to our YouGov survey, 92% do not, and it\u2019s fair to assume that the remaining 8% are lying.<\/p>\n\n\n\n With multiple choices for sleeve length, we decided to see what would happen if we marked one of the options as \u201cregular\u201d and one \u201clong\u201d. Half of the people visiting the website would see this version and the other half would see the usual version without any explainers.<\/p>\n\n\n\n The result was a 7% uptick in sales<\/a>, without the brand having to make any other changes to its website or its marketing strategy<\/a>. All it had to do was add a couple of words to the sizing options to offer clarity for customers.<\/p>\n\n\n\n Another fun example is when we tested out new pizza flavours for a leading pizza restaurant. Launching a new product is a huge undertaking for the brand that typically takes 12 months and involves market research, focus groups, taste tests, sourcing ingredients and working with its supply chain.<\/p>\n\n\n\n To speed things up, we tested consumer demand for new flavours by adding one new pizza to the online menu, chosen at random from five different options. The catch was \u2013 none of these products existed. We just wanted to see if customers would show interest. <\/p>\n\n\n\n Half of the people browsing the website would see them and half of them wouldn\u2019t, and then we analysed how many people tried to buy them. Those who did were told that the product in question wasn\u2019t available yet. <\/p>\n\n\n\n This is a world apart from traditional focus groups. We didn\u2019t bring people into an artificial environment, feed them pizza and ask for feedback. Instead, we tested on real customers who were \u201cin the moment\u201d \u2013 hungry, on their sofa, on a Friday night. There\u2019s no better form of evidence \u2013 and it\u2019s immediate, too.<\/p>\n\n\n\n There are multiple benefits to A\/B testing, and the fastest growing companies out there, the likes of Amazon, Meta, and eBay, do a huge amount of experimentation.<\/p>\n\n\n\n The good news for smaller businesses is that they don\u2019t need a big budget to get started, as long as they have enough website traffic to get statistically significant results. Any brand spending a decent amount on paid search and paid social can and should be experimenting.<\/p>\n\n\n\n What we\u2019ve found is that only around 20-30% of tests are successful \u2013 that is, they generate a statistically significant improvement in performance. This means that brands that are running them constantly are gathering user data and making safer choices around how they improve their website and their products.<\/p>\n\n\n\n It also means that brands who don\u2019t<\/em> experiment using A\/B testing will fall behind. Considering that only 30% of changes have a positive impact, that means the brands who aren\u2019t<\/em> experimenting will still be making these changes \u2013 they just don\u2019t know what\u2019s helping and what\u2019s harming. And a lot of their efforts will have no impact at all.<\/p>\n\n\n\n Most importantly, once people realise that the test can be turned off if it doesn\u2019t work, it helps them think about the big, bold changes that they\u2019ve been scared to roll out.<\/p>\n\n\n\n Instead of settling for the safe option, they feel empowered to test something much more aggressive, something their rivals wouldn\u2019t do. And that\u2019s how they get a competitive advantage.<\/p>\n","protected":false},"excerpt":{"rendered":" Stephen Pavlovich, CEO \u2013 Experimentation at GAIN, explores why A\/B testing doesn\u2019t only help brands create better products, it also allows them \u2013 and their agency partners \u2013 to try out their boldest ideas.<\/p>\n","protected":false},"author":480,"featured_media":23393,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"apple_news_api_created_at":"2025-08-05T08:29:49Z","apple_news_api_id":"a9efe432-9889-4728-8e0e-2a4cd3cf26d8","apple_news_api_modified_at":"2025-08-05T08:29:49Z","apple_news_api_revision":"AAAAAAAAAAD\/\/\/\/\/\/\/\/\/\/w==","apple_news_api_share_url":"https:\/\/apple.news\/Aqe_kMpiJRyiODipM088m2A","apple_news_cover_media_provider":"image","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_cover_video_id":0,"apple_news_cover_video_url":"","apple_news_cover_embedwebvideo_url":"","apple_news_is_hidden":"","apple_news_is_paid":"","apple_news_is_preview":"","apple_news_is_sponsored":"","apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":[],"apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[3],"tags":[],"topic":[650,651],"class_list":["post-23392","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-the-interface","topic-digital-strategy","topic-people-culture"],"acf":[],"apple_news_notices":[],"yoast_head":"\nCase study: Testing the market, once slice at a time <\/em><\/strong><\/h3>\n\n\n\n
Smaller budgets, bigger impact <\/em><\/strong><\/h3>\n\n\n\n