As with all things in life, sometimes we learn the most when we fail. Early success can be a harbinger of bad design. Imagine if the first generation cars America made had never been involved in crashes. Would we have begun crash testing and safety design or just assumed that our vehicles wouldn’t crash? Think about how deadly crashes would be on modern automobiles if we hadn’t failed so many times to date.
When we’re talking about modern products and services, the question becomes: how do we design for failure before our products are ever released to the public? It’s no longer sufficient to rely on the user to fail in order for the manufacturer to learn. All of the testing the airline industry puts into the components of a new jet is a testament to good product development, designing for failure under the worst conceivable conditions.
Computer software should be no different. Too often today, the phrase ”beta testing” is synonymous with “product marketing”. This plays a significant role in forcing underdeveloped prototype products into the hands of unsuspecting users too quickly, and I believe it’s what lies at the heart of our most detested software.
With the proper amount of requirements, documentation, and planning the time required for beta testing can be reduced. Nonetheless, even the designs of Leonardo Davinci and a software programmer with the foresight of Sun Tzu couldn’t possibly predict how every single user is going to interact with today’s complex software applications. Testing software will always be a necessary part of development.
So if testing is so critical, how should it be carried out? I would argue that it ought to be a controlled, continuous, and iterative process. You cannot have a handful of your test users sit down at any given time, test for an hour, write down their suggestions, and leave. Nor can you test with all of your users at once, as that would overwhelm your team’s capacity to evaluate and implement their suggestions.
If the US Constitution were drafted and ratified with the involvement of every member of every state government sitting in a giant meeting hall, we’d likely have no Constitution and no country at all. Moreover, if it were created behind closed doors solely by Thomas Jefferson and James Madison, we’d have arrived at the same result. Planned, iterative, and controlled development is a reasonable and prudent process.
Therefore, my software testing solution at present is to bring in groups of users and continually increase the size of the test group as changes are implemented and testing is carried out. Representatives from your group of critical stakeholders, the people who have the most to lose from a poorly designed application, are the first to test. After that, I increase the size of that group and add in a few non-critical users. The third and final phase is to open the door to all users for testing with the most refined and development prototype of the software at hand.
It’s important to note that users need to be encouraged to actively participate. Often they tend to wait until the last testing date and submit everything at once. Also, prior to opening the system up to all users, it’s beneficial to get the word out about your product. Much like the Federalist Papers did for the Constitution, you have to create an argument for your product and explain it to the least familiar and most opposed of your users.
Coupled with good planning, good testing can create some of the most innovative and user-friendly software in the computer industry. The developer can fail often and do so before the product arrives in the public’s hands. What this process amounts to is the re-humanization of computer software design. It forces the developer to adapt applications to people naturally, instead of forcing people to adapt to the applications.
