Prototypically Speaking – The Software and Requirements Prototyping Blog

In Requirements Prototyping, when a designer creates an initial prototype design for a customer, it becomes the starting point for a cycle of evaluations and feedback sessions.  These sessions can (and should) involve both the users who will work with the final system, and project stakeholders.  Though not necessarily at the same time.

What I’d like to touch on is this little dance of ‘build a design:evaluate’ – in many ways it’s one of the most fundamental cogs in the wholeprocess of requirements prototyping-led software projects.  We need to get it right and understand it so that it delivers real value.

In an ideal world of infinite resources and time, we would repeat this little dance as often as it takes until we had the perfect design.

However, this utopian world just doesn’t exist so we need to consider how to organise our evaluations so that we can get as much out of them as possible.  We need to think about the actual organisatinal cost of getting people together for feedback sessions, and make sure that we don’t just include everyone for the sake of it.

Jakob Nielsen has suggested that five is an optimal number for testing a design.  His arguments are that the majority of usability concerns are identified by the first few participants, and thereafter most new participants tend to repeat observations with the odd exception.  His suggestion is that, rather than blow the entire budget in getting everybody to test a design in one phase, it is better to have a number of smaller test phases and an iterative refinement of that design between them.  This is precisely what we should be doing when running feedback sessions with users.

What’s different about feedback sessions as opposed to Nielsen’s usability testing is the fact that feedback sessions are generally:

• Collaborative: we’re not looking for individual isolated reports, but engaging discussion about the general design;
• Objective-led: we’re not specifically testing the usability of a design, rather its suitability to meet the objectives set out at the outset;
• Cross-disciplinary: not restricted of any one type of person, feedback sessions are most usually composed of a cross-section of interested parties, including users and business stakeholders.

The objective is different as well – usability being, well, about how usable a design is, as compared to general feedback which aims to help steer and shape the direction of an ongoing design effort.  Different beasts.

That said, restricting any feedback group to a smaller number of participants helps with focus and coordination, and it’s also much easier to find a time to get everyone together and a place to meet when the group is a fraction of the size.

So, a recommendation here is to keep the numbers low – perhaps with a representative from each section of the project: sponsor, business domain expert, designer, manager, user.  Ensure that everyone actually understands what it’s all about and is willing to make an effort to be there; there’s absolutely no point in forcing someone to get involved in this process against their wishes.

Each feedback session should set out with the stated objectives and try to assess the latest design proposal or prototype against those objectives.

Previous efforts should be used as yardsticks to gauge progress and to ensure that things don’t end up moving away from what’s required.  Nominate a chair for these sessions, someone who will be responsible for maintaining control and ensuring that the session is focused and organised.

What we need to get out of each session is a clear picture of how well the latest design meets the objectives, and whether further refinement is required.  It’s about proving the suitability of a design from different perspectives.  It’s also a good progress indicator, something which can be fed back into the project plan.

The next thing is to try to ensure that these feedback sessions are reasonably regular – say, once every other day or once a week – whatever works for the project.  Of course, how often depends on how much design change is going on, and it’s reasonable to expect the amount of that change to decrease as the design firms up.

In the next instalment we’ll look at what constitutes a good feedback session, how best to organise it, what outputs to expect from it and how to feed those into the next iteration and into a project plan.