|
Although focus groups and surveys can be a powerful tool in web site and application development, you shouldn't use them to drive your user interface design.
At best, traditional market research like focus groups and surveys often tells you what users think with your web site or application, but they are less useful at pinpointing what to do about it.
Focus groups and surveys assess what customers say they do or want and not the way customers actually use the web site. Usability testing, on the other hands, tells you precisely where the problem lies and how to fix it.
"Unfortunately, focus groups are a rather poor method for evaluating interface usability. It is thus dangerous to rely on them as your only method in a Web design project." Jacob Nielson, The Use and Misuse of Focus Groups
-
The Focus Group Session: Drop-down's are good.
One of our clients conducted a focus group with their target users. During the session, one guy mentioned that the site should have a drop-down menu navigation. Because he was insistent, soon everyone in the group said "Sure, why not?".
-
Our client's reaction: Let's have drop-down menus.
We knew from research that drop-down menus annoy users and we warned our clients against it. However, our clients went ahead and 3 weeks later, drop-downs were implemented for their new prototype.
-
Usability tests were conducted: Let's test the navigation.
We conducted usability tests on the prototype with the same group and asked our clients to sit in. The first test we did was with the same person who insisted that drop-downs were good.
We asked him to search for "Home Loans". He scanned the page but couldn't find it, so he started to use the navigation. While he was using it, he was visibly getting annoyed.
-
Test findings: In reality, drop-downs annoyed users.
This is because, by nature, drop-downs suffer from poor readability. Secondly, drop-downs are quirky because the menus tend to disappear when you move your mouse pointer to try to click on an item in the drop-down list. So the user has to try again.
Eventually, the user got so frustrated that he unwittingly burst out, "Stupid, drop-downs. I hate it!". This was the same user that advocated drop-downs in the focus group session. The minute the user made this remark, he gave us a sheepish look because he remembered what he said.
-
Our client's reaction: Drop-down menus were removed.
Two days later, after seeing actual user behavior, our clients completely removed the drop-down navigation menus.
"There is value to surveys and focus groups, but the danger [with focus groups] comes when you start interpreting information. For example, in a focus group, you hold up an interface. One guy says he's not sure if he likes it. People are influenced by this comment. Behind the scenes, everyone runs around changing things. I trust behavior more. It's a stronger indicator of whether an interface is effective or not." - Fischler, Agency.com
As with any method based on asking users what they want, focus groups and surveys can produce inaccurate data because users may think they want one thing but the reality may be completely different. The better method is to measure or observe how they actually use things.
During the conceptual phase, there are lots of questions and a lot of assumptions are made about what users may want or what could work. Instead of actually testing out the concept, organizations tend to try and gather more and more feedback. Instead of gathering more and more opinions, start gathering actual usage data by conducting usability tests or evaluations.
| About the author |
 |
Gul Amir Khan is the Chief Usability Consultant for MicroUsability and the current President of the Usability Professional's Association (Singapore Chapter). Gul has been trained in Game Theory and Strategic Behavior Analysis and has incorporated these techniques in web usability engineering. He has conducted numerous usability projects and usability workshops across Asia. |
| |
|
