Here’s a step by step list of how you can successfully carry out user tests on a budget and achieve great results.

Example of case study

• Usability test start off with user being asked to find individual item..
• The user/tester is normally paid for this, a real user would not take as much time to look for item.

The Comparison of costs betweeen traditional and remote testing:

• Traditional - testing lab required, specialist equipment, travel costs, user incentives.
• Remote tests - online meeting, recruitment, user incentives, better on budget.

1. Remote testing overview:

Start of defining objectives and target audience:

• What

• who are you testing?
• what are you testing?

• How

• ask your company, meet marketing and devs
• ask your users - survey, interviews

• Outcome

• Informal test mission and objectives
• Common understanding, test direction and scope

2. Set up scenario

• On a budget it’s best to be performed online - online meeting and screen, capture and phone.

> Go to meeting, webex (meet people online)
> Camtasia, ZDsoft > screeen/video capture

• What

• representative tasks
• within time linits and user capabilities
• in line with test objectives

• How

• consult with project stakeholders and objectives
• develop sample app and html instructions (walkthru of task you want user to perform)
• run dry test with home users
• apply changes

3. Recruit testers

• define user profile and user criteria (1hr of time reqd)
• decide on incentive (cash)
• set up recruitment survey
• select recruitment channels- Online (community, classified, job ad), traditional (newspapers, ads), recrutiment agency
  - All link to survey to see how suitable candidate is for study

• contact all shortlisted users, provide survey with available timeslots

4. Carry out test

• Set up software and phone conference
• Send invitations to project stakeholders in advance
• Be punctual

5. During the test

• Confirm user profile
• Ask for permission to record session
• Limit moderator intrusion
• Encourage thinking aloud
• Deliver incentive/payment
• Have fun

6. Analyse results

• During tests - track all usability issues
• Have more than one person taking notes
• After each test, compare notes and analyze
• After all tests, summarise patterns and major problems
• Setup report and sample vidoes

7. Design report and brief stakeholders

• communicate results to stakeholders > product managers, design managers, etc.
• so that everyone views and absorbs the information from the results.

Final Notes

• Don’t test same person more than twice a year, dont over use them as they may become biased as to what you are looking for
• If have to retest, use different group, best to have oversubscribed group of about 50 people to use if need to retest
• Ideal number of users? account for 5 people > the ratio between quality and costs, unless retesting another 2-3.. have about 20 applicants if want enough users for each study.

These are notes from the Original presentation at WebDD by Telerik.

This entry was posted on Tuesday, May 29th, 2007 at and is filed under work, web services, accessibilty, authoring, usability, testing, users. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.