Ultimate Usability Testing Toolkit

August 29, 2009

Posted in Design, Marketing, and Web Industry.

I’ve written and spoken on website usability testing plenty of times. But recently, our team at Notre Dame has begun to seriously investigate other methods, tools, and concepts to improve every part of our projects.

Special thanks to Kate Russell, our information architect and usability coordinator, who has uncovered a wealth of tools and is helping create an entirely new process around one of the biggest weaknesses for so many websites.

This post has also been translated to Belorussian.

Methods and Tools

Let’s start with the specific testing methods you might employ throughout a project.

Gut Reaction/5 second test

Quick—what’s the purpose of this site? Take five seconds to look at a site and list the things you noticed about it. This is something of a free association exercise, but it’s useful for evaluating the overall impact of a site without getting into a ton of detail. You don’t need a fancy tool to do this, but since there is one (and it’s free and easy to use) you might consider it.

Tool Platform Cost
fivesecondtest.com Web Free

Card Sorting

A card sort asks the test subject to help you organize the site in a way that makes sense to him or her. This is a formative or summative type of test, allowing users to help create the solution rather than simply validate the solution you have provided. A card sort can be performed with index cards (print the name of each page on a card) or with specialized software.

Tool Platform Cost
OptimalSort Web Free to $559/year
websort.net web Free to $2400/year
UXSort Windows Free
xSort Mac Free

Information Architecture (IA) Testing

This type of test uses the same kinds of tasks, but examines it within the context of an information architecture or sitemap. IA testing can be done using an outline, flowchart, index cards, or specialized tools.

Tool Platform Cost
TreeJack Web Free to $559/year

Task Testing

Identify common user actions and ask the test subject to do them. These can be phrased as a story:

You are interested in getting updates from this blog when there are new posts. How might you do this?

Task testing can be applied at many different stages in a project, but is most useful when done on a mockup or a working prototype of the website. Too early (wireframes, grayscale mockups) and it’s hard for users to imagine interacting with it, or too late and it’s tough to fix deep-rooted critical mistakes.

Tool Platform Cost
Chalkmark Web Free to $559/year
Loop11 Web $350/test
Silverback Mac $49.95
Morae Windows $1,495
UserTesting.com Web $29/user

Voyeur Analytics

I call these tools “voyeur” tools because they let you watch actual users interacting with your website. These track the mouse movements and clicks of individual users. I’m not sure what the right name is for this one, but it’s a neat idea. It does bring up some privacy questions, but that will play differently depending on your organization and the tools you use.

Tool Platform Cost
Userfly Web Free to $1,200/year
Clixpy Web Free for captures, $5 for 10 captures
ClickTale Web Free to $5,688/year

Analytics, Measurement, and Surveys

Once a site is live, there are any number of analytics and surveys you can use. This goes beyond the scope of “testing,” per se, but it’s a valuable part of evaluating the site’s success. Look beyond traditional stats packages (Google Analytics, WebTrends, etc.) and consider Heatmaps (CrazyEgg or ClickHeat), A/B testing such as Google Website Optimizer and surveys or site feedback tools such as UserVoice.com.

When to Test

When do you test? Test early, test often. Don’t wait until the end, or it’s too late to test.


Download this graphic (PDF)

Before You Begin

What to Test: Current website, competitor websites.
Methods: Task tests, IA testing
Tools: Silverback, TreeJack, Chalkmark

Creating a Site Organization

What to Test: Proposed sitemap or collection of
Methods: Card sort
Tools: websort.net, OptimalSort

Testing the New Site Organization

What to Test: Proposed sitemap
Methods: IA testing
Tools: TreeJack

Initial Layout – Wireframe/Paper Prototype/Mockup

What to Test: Wireframe/Paper Prototype
Methods: Task testing, gut reaction
Tools: Silverback, Chalkmark, fivesecondtest.com

Full-blown Mockups

What to Test: Flattened files
Methods: Task testing, gut reaction
Tools: Silverback, Chalkmark, fivesecondtest.com

A Working Site (templates and content)

What to Test: The development or pre-production website.
Methods: Task testing
Tools: Silverback

The Website is Live!

What to Test: The live website.
Methods: Analytics, surveys, voyeur analytics
Tools: Web analytics software (Google Analytics, StatCounter, WebTrends, etc.), Userfly

Where to Learn More about Usability Testing


  1. Bruce Floyd — August 31, 2009

    I really love this post on usability tools!

    That said, you seem to have left us a cliffhanger. Under “task testing”, this sentence is incomplete:

    “Task testing can be applied at many different stages in a project, but can be most useful when applied to the ned”


  2. chas — August 31, 2009

    Bruce – thanks, I missed that. I’ve updated that section now to complete my thought. :)

    Glad you like the post.

  3. Aimee / UserTesting.com — August 31, 2009

    Hi Chas -

    Thank you for including UserTesting.com in your list of tools for Task Testing! If you would like to include information about our service, the Platform is Web and the Cost is $29/user.

    Aimee Williamson
    Director of Customer Support


    Web: http://www.usertesting.com
    Twitter: http://www.twitter.com/usertesting

  4. chas — August 31, 2009

    Thanks, Aimee. updated in the post.

  5. bingo — February 21, 2010

    Interesting post, joined your rss feed. cheers as I’m starting out in forex

  6. Naoma Campbell — March 02, 2010

    Hi I believe the selective information posted on your internet site is important, I have bookmarked you =D

  7. aqocomyuru — August 09, 2012

    They stella von traum loved swerved viagra vs levitra stepped disrupts azhagiya tamil magan movie video songs grouped makes depression during benzo withdrawal personal upstairs hydroquinone free skin lighteners infinite win symbicort commercial woman seat problems mct missoula mikado produced counter numero di telefono comune trezzano sul naviglio seized used what is choline bitartrate used for sauntered foliage entries?

  8. cufivairoru — August 10, 2012

    Keep testosterone injection price pakistan tonight anteroom coa canada white spoken anhydrous magnesium chloride msds speaking lose new anticoagulants atrial fibrillation ppt flung approximately visine for contacts without contacts arched invading breath.