Testing a complex product design for early adopters?
We are building a platform for the live music industry. We have used the latest in web technology in an attempt to provide the most up todate and exciting experience for our potential customers.
We are close to launching the product but I really want to get early adopter feedback and test some of our design decisions. What principles should I focus on when running user focus groups or performing design tests with early adopters?
-
Hey I've edited this question so that it now fits in this forum rather than stackoverflow can someone take a look and see if I was successful. Thanks! – Ross Mann 11 years ago
-
@RossMann, that's a much better fit for this site, but perhaps eliminate part about asking for experiences, and just ask "what principles make for the best focus groups?" instead. This is also a very different question than the original, and I'd love to hear from MuseJam if this is still something he/she would be interested in asking about. – rbwhitaker 11 years ago
-
Done. Yeah its pretty different but not 100% different and still valuable information for what he is about to go through. – Ross Mann 11 years ago
-
@MuseJam, the Q has been edited significantly from your original Q. Is the revised Q still something you would be interested in? If not, you may try re-asking the Q with a different focus, as long as it is not focused on technical aspects. Note that answers.onstartups is for business related questions, not technical questions, which is why your original question is off-topic. – Zuly Gonzalez 11 years ago
1 Answer
It's really tricky to run user focus groups well: what people say is often not what they do. You also have very interesting group dynamics that come into play.
The first piece of advice is that you really don't want groups, you want individuals. Sit down with each person one-on-one and then you will get their opinion unvarnished by group dynamics, people will speak more openly, etc. It is far better for you to invest the extra time in a series of one-on-one meetings than to do groups. I can't stress this enough (I've participated in both).
Second, it is important to show them designs (ideally interactive so they can use it in front of you: use a mockup tool like Balsamiq if needed to do this, though it sounds like you have a product already). You want feedback on a specific design alternatives rather than just asking them questions about what they want. This gets over the 'people don't know what they want until you show it to them' problem.
The third, related point of advice is that you really want to watch them using your site to accomplish a task they are interested in. Giving advice on whether a site 'looks good' is very different than watching them actually try to locate an artist, etc.
Fourth, as much as possible, you want to watch them trying to use your site unprompted. When you talk people through the different alternatives you get very different feedback from when you watch them try to accomplish tasks on their own on your site. I am often shocked at how what is obvious to us in our designs is incomprehensible or hard to use by users (or vice-versa, when users quickly and intuitively adopt something that my partners thought was a weak design). You only find this out by keeping your mouth shut.
Fifth, remember that people are different: what works for one person may not for someone else. Don't take quick decisions based on the feedback of a couple of people only.
Sixth, make sure that your focus group are representative of your actual customers. If you, for ease of use, pick a bunch of university students for your focus group but actually your main target is going to be mothers of young children, you will get very different answers.
Lastly, and perhaps most importantly, far and away the best feedback you can get is money. People who say "this looks good, I would use it" may just be being courteous. People who actually pull out a credit card and place an order really do want it. If you can find a way to get pre-paid by some of your focus group members for your platform (assuming people will pay to use it), then you will have the best kind of feedback (and you will impress potential investors).
