A new term recently surfaced, and it will undoubtedly be part of the digital marketing lexicon for a while. It’s a decision technology being referred to as “multi-arm testing.” You may have even heard it referred to over the past few years as “bandit testing.”
Predicting behavior
So, exactly what is it? It’s A/B testing on steroids; a “preferential science experiment.” In this case, it gives digital marketers the ability to test lots of images, copy and headlines all at once to determine which potential combination will be most likely to influence an end-user to take the desired action.
Think of a slot machine. You pull the lever and a graphical icon combination is presented. Lemons, kiwi, whatever. If it’s not a winner, you drop another coin, pull the lever again and see more fruit.
Quicker is better
Imagine you had lots of slot machines and could pull all the levers at the same time, you’d greatly increase your odds of winning, in a fraction of the time.
That’s the theory behind multi-arm “bandit” testing. With normal A/B testing it sometimes takes a year or more to get good, solid data. And if it’s not the result you want, you’re back to testing for another year. Nobody has that kind of time. With “bandit” testing, you’re running a multitude of variations in order to find the most successful verbiage in the fastest amount of time.
Human-machine interface
By leveraging machine learning and using highly developed mathematical models, you can test multiple “arms” to find which images, copy and layout options convert. When humans can use machines to do the heavy lifting, the end product is a much more efficient process and you can have an answer 80% quicker.
Iteration can literally take months instead of years, because there is no need to wait for a single, winning variation. By using the multi-arm approach, it is possible to reduce the (typical A/B) testing routine to mere months.
How can I get my hands on this?
If this sounds like just the tool for you, here are some of the available options:
- Optimizely’s Stats Accelerator
- Split Metrics’ Bayesian Multi-Armed Bandit
- Presado’s Message Machine
Big time adoptions
If this sounds a little too “Big Brother”-ish, maybe it is, but today, when everything is expected to move at the speed of sound, it’s a super smart move.
Because of its ability to shift gears during a campaign, bandit testing typically works best in the following situations:
- High traffic volume
- Targeting
- Site or feature will be short-lived
- Testing can be automated
- Headlines and short-term campaigns
It should come as no surprise that this testing methodology has been adopted by certain corporate giants. Currently using “bandit testing” to hone strategy are Zillow, Microsoft, Google, Amazon, Netflix and McDonald’s. In fact, McDonald’s embraces this technology so completely, it bought Israeli start-up Dynamic Yield and is now personalizing customer experience both online and in-store.
About Michael Taylor
