Ultimate Usability Testing Toolkit

I’ve written and spoken on website usability testing plenty of times. But recently, our team at Notre Dame has begun to seriously investigate other methods, tools, and concepts to improve every part of our projects.

Special thanks to Kate Russell, our information architect and usability coordinator, who has uncovered a wealth of tools and is helping create an entirely new process around one of the biggest weaknesses for so many websites.

This post has also been translated to Belorussian.

Methods and Tools

Let’s start with the specific testing methods you might employ throughout a project.

Gut Reaction/5 second test

Quick—what’s the purpose of this site? Take five seconds to look at a site and list the things you noticed about it. This is something of a free association exercise, but it’s useful for evaluating the overall impact of a site without getting into a ton of detail. You don’t need a fancy tool to do this, but since there is one (and it’s free and easy to use) you might consider it.

Tool	Platform	Cost
fivesecondtest.com	Web	Free

Card Sorting

A card sort asks the test subject to help you organize the site in a way that makes sense to him or her. This is a formative or summative type of test, allowing users to help create the solution rather than simply validate the solution you have provided. A card sort can be performed with index cards (print the name of each page on a card) or with specialized software.

Tool	Platform	Cost
OptimalSort	Web	Free to $559/year
websort.net	web	Free to $2400/year
UXSort	Windows	Free
xSort	Mac	Free

Information Architecture (IA) Testing

This type of test uses the same kinds of tasks, but examines it within the context of an information architecture or sitemap. IA testing can be done using an outline, flowchart, index cards, or specialized tools.

Tool	Platform	Cost
TreeJack	Web	Free to $559/year

Task Testing

Identify common user actions and ask the test subject to do them. These can be phrased as a story:

You are interested in getting updates from this blog when there are new posts. How might you do this?

Task testing can be applied at many different stages in a project, but is most useful when done on a mockup or a working prototype of the website. Too early (wireframes, grayscale mockups) and it’s hard for users to imagine interacting with it, or too late and it’s tough to fix deep-rooted critical mistakes.

Tool	Platform	Cost
Chalkmark	Web	Free to $559/year
Loop11	Web	$350/test
Silverback	Mac	$49.95
Morae	Windows	$1,495
UserTesting.com	Web	$29/user

Voyeur Analytics

I call these tools “voyeur” tools because they let you watch actual users interacting with your website. These track the mouse movements and clicks of individual users. I’m not sure what the right name is for this one, but it’s a neat idea. It does bring up some privacy questions, but that will play differently depending on your organization and the tools you use.

Tool	Platform	Cost
Userfly	Web	Free to $1,200/year
Clixpy	Web	Free for captures, $5 for 10 captures
ClickTale	Web	Free to $5,688/year

Analytics, Measurement, and Surveys

Once a site is live, there are any number of analytics and surveys you can use. This goes beyond the scope of “testing,” per se, but it’s a valuable part of evaluating the site’s success. Look beyond traditional stats packages (Google Analytics, WebTrends, etc.) and consider Heatmaps (CrazyEgg or ClickHeat), A/B testing such as Google Website Optimizer and surveys or site feedback tools such as UserVoice.com.

When to Test

When do you test? Test early, test often. Don’t wait until the end, or it’s too late to test.

Download this graphic (PDF)

Before You Begin

What to Test: Current website, competitor websites.
Methods: Task tests, IA testing
Tools: Silverback, TreeJack, Chalkmark

Creating a Site Organization

What to Test: Proposed sitemap or collection of
Methods: Card sort
Tools: websort.net, OptimalSort

Testing the New Site Organization

What to Test: Proposed sitemap
Methods: IA testing
Tools: TreeJack

Initial Layout – Wireframe/Paper Prototype/Mockup

What to Test: Wireframe/Paper Prototype
Methods: Task testing, gut reaction
Tools: Silverback, Chalkmark, fivesecondtest.com

Full-blown Mockups

What to Test: Flattened files
Methods: Task testing, gut reaction
Tools: Silverback, Chalkmark, fivesecondtest.com

A Working Site (templates and content)

What to Test: The development or pre-production website.
Methods: Task testing
Tools: Silverback

The Website is Live!

What to Test: The live website.
Methods: Analytics, surveys, voyeur analytics
Tools: Web analytics software (Google Analytics, StatCounter, WebTrends, etc.), Userfly

Where to Learn More about Usability Testing

• My Usability Bookmarks
• All Usability Bookmarks
• Usability.gov
• Jakob Nielsen
• Steve Krug