Why product teams fail

When I first started working in startups I was the design world’s equivalent of John the Baptist.

I really f**king loved the whole thing. The talks of apps, of app stores. Of data, of that really cool new tool everyone is talking about. Of prototyping, of user testing.

I, for want of a better word, drank the cool-aid & queued up for more.

There was so much authoritative information out there, on Medium & design blogs, that I cut my teeth in the design world by reading that information & practicing it out in the real world.

And, because it was all so authoritative, I never stopped to question it.

I mean, it had loads of likes. And people left encouraging comments. And it was part of a publication. And the guy that wrote it looked smart & had worked for some big company.

But recently, a thought has persistently come to mind & refuses to be left unanswered:

That maybe product teams have a key element to their approach all wrong?

Questioning common design practices for the first time Specifically, that the evangelism surrounding user testing may be predicated on an unsubstantiated claim:

That we can determine the outcome of a product change by testing a basic version of it (a prototype) on users before actually committing to a product change.

That our obsession with being data-driven has caused us to find any data to substantiate a claim, regardless of its validity or relevance.

And user testing, when you really think about it, is in many ways invalid — at least it would in no way stand up to the scientific rigours of academia.

Jason Fried, CEO of Basecamp, doesn’t even bother with user testing with the Basecamp team because he doesn’t see it as worth the time.

Not only is user testing incredibly time-consuming, but, in his view, it also tends to be inaccurate because:

1. Users act very differently in the context of testing: They sit, wholly focused on the task you ask of them with no external pressure put upon them. Constrast this with the real world, with a deadline, the constant distractions of office life, their boss chasing them up for something else, etc. You’ll likely find they use that product very differently in the real world.
2. A prototype, unless you have the resources of Airbnb or Google, will never be like the real thing. At worst, you’ll be testing grey wireframes on somebody who doesn’t even know what a wireframe is. At best, you can test an interactive prototype that takes you from screen to screen, but is unlikely to have functionality.
3. User testing falls victim a large number of user biases that mean they don’t say what they really mean. For example, users will confirm your assumptions to be polite, saying the product is “awesome”, regardless of what they truly think

Rather than having these long discussions about what to test, how to test it & then what to build, would it not just be better to build things faster & launch them in the wild, like Basecamp? Would we need all those meetings & discussions if that were the case? Would we not reach a conclusive solution faster?

And that isn’t to say that we give up on being data-driven. Data is the truth. I’ll still be singing its praises & evangelising until the day I give up on the startup world & head off to Bhutan to live a more simple, tech-free life.

It just means that we should remember to value different types of data by the accuracy of such data.

Furthermore, the assumption that we can test our way to a solution tends to mean we brush over one really big, important problem: That, in many cases, we are creating solutions to a problem that is potentially not even a problem in the first place, or is the wrong problem to be solving.

Refusing to acknowledge you may be going after the wrong problem As Seth Godin suggests, all the obvious problems in the market have now been solved, because the market is so competitive & anyone can start a business with under $100 nowadays.

So jumping to conclusions & settling on a problem too quickly is likely to mean you have failed to grasp the core of that problem or its complexity (otherwise it would have been solved already, right?)

Those who will be able to make the big leaps, to come up with the really innovative solutions, those are the ones that really understand what problem they are solving:

They are the ones who take empathy to a higher level.

They are the ones that come up with profound insights from living as their user, from being their user, from spending days being them to really, deeply understand what it is like to be them.

If you’re unable to deeply empathise with your user, you can test all the solutions you want, but you’ll never resolve a core problem for them because you don’t really understand what that problem even is.

The failure rate of startups is so high that it has led me to believe that, in many cases, product teams approach product development all wrong.

What if, rather than rushing to build a solution, your team really stopped to work out what the actual problem is? And then, rather than spending weeks discussing, arguing, assuming, why not just get something out there? Just test it in the real world? See what the results are?

You may find that you’re able to make those big leaps in product decisions whilst also moving quickly. What’s not to like about that?