Multivariate, or A/B testing, from the AirBnB website

When A/B testing don’t forget the answer may turn out to be C) You’re an idiot.

posted in: process | 0

Title: The tyranny of testing over design
Context: Data doesn’t lie. But honesty is not a prerequisite of intelligence.
Synopsis: Testing is good. But like anything else, too much of a good thing may not be so good. The law of diminishing returns tells us that (to borrow a perhaps incongruent metaphor) putting all of your eggs in one basket is not the smart long term play. In UX, using, specifically, multivariate testing as the final arbiter of all design decisions is a single tool, not a panacea. We must balance the benefit of side by side feature comparisons with our learned opinion and accumulated skill knowing when each should be the definitive word on a design decision. Testing can be be both insightful and valuable, but only when necessary, and it is our job to know when and where to wield it for maximum efficacy or we will only get minimum value out of it’s overuse.
Best Bit: “In my experience as a designer performing plenty of usability testing sessions, most of the times I can foresee what the outcome of the testing will be, and that’s probably because I am quite good at doing what I do.”

via medium.com

Leave a Reply

Your email address will not be published. Required fields are marked *