MEX

Mobile devices and wireless networks are inherently suited to lowering the cost of user experience testing, providing a viable channel for remote testing. However, there has traditionally been a certain stigma attached to this approach and a notion it is less valid than studies conducted face-to-face in a lab environment.

I sense that mood is changing and those who commission user experience studies are realising the optimum approach is a hybrid of qualitative and quantitative studies, in-person and remote testing, controlled and natural environments. In my opinion, this shift in attitudes is long overdue.

As testament to this, I am seeing growing interest from user testing practitioners in presenting new approaches for user testing at our MEX Conference. I am also seeing growth in usage of remote behavioural monitoring tools among network operators.

In addition, companies such as Mob4Hire, are harnessing crowdsourcing techniques to bring user experience testing to a much wider range of clients and for a broader range of projects than has previously been possible. I sat down with Stephen King and Paul Poutanen from Mob4Hire this morning to talk over how their platform for remote user testing is evolving.

The concept is simple: Mob4Hire signs up users all over the world, on a diverse range of networks and devices, who make themselves available for mobile testing projects. A client can send them a general survey or a specific application to test and receive direct feedback on how it performs in a real world environment, from real users on real networks. Each user is able to put forward a bid for how they’ll charge to do a test (typically between USD 5 and USD 10 for looking at a new application) and the client can accept or reject their offer.

By using crowdsourcing techniques, Mob4Hire is able to provide small developers with a usability testing reach they’ve never previously enjoyed, at a fraction of the traditional cost. While this will never replace the in-depth contextual understanding from detailed ethnographic and immersive studies, it is a cost-effective and useful addition to developers’ tool boxes.

A particular benefit is the exposure to all the vagaries of real world conditions on an international scale. With so many variations of handset and network, it is impossible to predict where the chain of user experience will break down in a controlled environment – the Mob4Hire approach gets you direct to the street-level conditions which will make or break the UX of your application.

I was interested to learn from Poutanen and King that a recent survey of users found 83 percent of people only download applications with at least a 4 star rating in app stores. The message to developers: you only get one shot to ensure your initial users have a great experience before your application is rapidly voted down the app store rankings. You need to test, test and test again using whatever methods at your disposal to ensure you don’t fall at the first hurdle.