ncelled. This was disruptive
to the development teams, which were relying on the tests to
answer important questions about the interface. But I've
never had to abort a usability test due to a problem with a
paper prototype. I've even done paper prototyping during a
Paper prototypes give you complete control over how the
interface behaves -- this is useful for short-notice demos as
well as usability tests. Paper prototypes don't use databases,
networks, servers, or any other form of technology, which
eliminates most of the things that can go wrong. You don't have
to worry about someone putting a change up on the development
server that breaks the very thing you're trying to test (and
this also means that development doesn't have to come to a
complete stop every time you want to do a usability test).
You may be thinking, "But this is my application. I'm the one
writing the code." Yes, but until you're well into the
debugging stage, your code (or code written by others that
yours depends on) may not be stable enough to allow users to
do meaningful work without crashing. Bugs happen -- there's a
big difference between the pleasant fantasy of &147;These ones
and zeros obey my every whim and the painful reality of
I'd be better off if I hadn't come in today. I
hope I'm not the only software developer who's ever said that!
Even people who recognize the benefits of paper prototyping
may have some concerns about the technique. The questions I
commonly hear fall into three categories: validity,
professionalism, and resources.
- Validity ("Does it find real problems?")
I'm often asked whether paper prototyping might have bad
side effects, either in terms of missing important problems
or finding false ones. The short answer is that paper
prototyping does find most of the same problems that you'd
find by testing the real thing.
To my knowledge, there is only one published scientific
study (see Resources) of the
validity of paper prototyping. Three researchers at
Verizon (formerly GTE Labs) compared the type and
number of problems found with a low-fidelity
(i.e., paper) prototype as compared to a working
prototype. They found that there was a significant
degree of overlap between the problems found using the
two different methods. Although the working prototypes
did uncover a few more problems, they also took
significantly longer to develop than the
low-fidelity ones -- weeks instead of days. These
results indicate that there are diminishing returns
from taking the additional time to develop a polished prototype.
My own practical experience backs this up. I have
conducted several hundred usability tests of software
and Web sites, including more than 100 using paper
prototypes. With only a few exceptions, I find the same
kinds of problems with both.
- Professionalism ("What will others think of it ... and us?")
The thought of showing an unfinished -- or even flawed
-- design to outsiders makes some developers uncomfortable.
Many of us are perfectionists by nature and/or training,
and our jobs reward this trait. It's natural to be concerned
that others will perceive our work as incomplete or sloppy.
Fortunately, that's not what happens. Users (and coworkers)
respond positively to a paper prototype, provided that
they've been briefed appropriately. When participants
arrive for a usability test, I explain that we know the
design has some rough edges (sometimes literally!) but
that we want to get their feedback before we've invested
a lot of effort to build the wrong thing. I've found that
users understand this, and I've never heard a user make
a disparaging remark about a paper prototype. On the
contrary -- users appreciate knowing that their input
is being requested while there's still time to incorporate it.
- Development resources ("Do we really have time to do this? Is it just extra work?")
Those who have never done paper prototyping (or even
usability testing) often overestimate the amount of time it
will take. If you're already doing usability testing and
only the paper prototype part is new, estimate no more
than 1-2 days for proto