Click here to Skip to main content
15,792,540 members
Articles / Operating Systems / Windows

Usability Testing (Chapter 2)

Rate me:
Please Sign up or sign in to vote.
4.85/5 (31 votes)
22 Jan 200214 min read 85.3K   57   7
Usability: The myth of efficiency.


Before I begin, I just want to say that I do not consider myself to be an expert in usability testing, user interface design or just about any other area. I do, however, have some experience with these areas (both positive and negative), and I decided to write this article to share my experience with others. The concepts and ideas I discuss in this article are not earth-shattering and many of you will already know all of this information.

Also, I want to acknowledge my boss (that sounds very odd I know). I have learned from him a great deal concerning usability and "efficiency". Much of the information I cover in this article is knowledge I gained by working with him and watching how he works with users. I have managed to benefit from his help and knowledge and hopefully build on that knowledge.

If you ask just about any software developer how they define efficiency, they will tell you that it is a measure of how fast a function/algorithm/program performs and how few resources it uses. If you ask the average user what it means, they will tell you that it is a measure of how quickly they can do their jobs. These two definitions are not always compatible.

I titled this article "the myth of efficiency" because I feel that as a software developer I have been mislead. Just about all the books, programming journals, web sites, etc. that I have read have focused on efficiency from the programmer's perspective. I am not saying that this perspective is not important or even the right perspective for many problems, just that very often it is counter-productive and should not be the main focus.

Let me just say that I am a big fan of efficient algorithms and I like to see my code perform well. It is difficult to predict how a particular function/algorithm will be used in the future, and making the code efficient often expedites things in the future. Unfortunately, this has often been my primary focus. I have neglected the need the user has to get their job done in the most efficient manner possible. This has resulted in me optimizing the wrong algorithms. The key is not to be concerned with making all or a particular algorithm efficient, the key is to make the right algorithms efficient.

Where to Start

As I said in the introduction, the key to efficiency is making the right algorithms efficient. To do this, we need to understand how an application needs to be used by the user. (I say "needs to be" instead of "will be" because "will be" assumes we are going to decide for the user.) The only way I know to do this is to work with users before development begins and while development is in progress. The key to this is understanding the user's workflow.

User workflow is how the user performs their jobs in the absence of software. Most people have a basic job description which involves a certain number of responsibilities. They also may have a set of requirements related to accounting for their work. They usually have a set of tasks that they perform in repetitive fashion along with a few tasks they perform highly sporadically. In addition, there are typically a set of tasks that they only perform on a very rare occasion.

Many software applications fail to recognize that users already have a defined workflow process before the software was available, and basically try and force the user to alter their workflow to conform to the software. This may work when the software is a critical part of their jobs, but will probably result in decreased workflow efficiency and user dissatisfaction.

To develop a truly "efficient" application, it is necessary to speak with potential users about what they do and how they do it. This discussion is probably best done in the absence of software, prototypes, or screen shots. The user may be confused or misunderstand the software, and fail to provide the most valuable information. One place where software can help, though, is when there is an existing system in place you are going to replace. The user may be able to point out "what they like and dislike."

The goal is not to have the user develop the software for you, just to have them explain how they work. It is the developer's responsibility to convert this into software. With this information, it is possible to better design the application to meet the user's needs. It is also easier to decide what parts of the application need the most optimization and which ones are less important.

Case Studies

The best way I can think of to illustrate the problem and its solution is by using real-world examples.

Workshop management

A recent project I worked on involved replacing an MS Access database with an updated UI and reporting system. The old system had many "features" (I now refer to them as warts). One of the "warts" was a "workshop management system". I hesitate to call it "management system", but that was its intended purpose.

The system had three basic objects. These were workshops, members and registration records. There were basically just three forms in the system. One form was used for member data, one for workshop data, and the other form was used for registration records. Pretty simple and uncomplicated (technically). To navigate among the workshops, the user could use record navigation (i.e., First, Last, Next, Previous, Add, Delete) or a combo box located in the upper right-hand corner. From this screen, they could print a registration list, attendance roster, etc. Navigating members was basically the same, but instead of a combo box, there was a box where the user could enter a member ID and go to that record.

The next step was to ask the user how they did their job before they had the MS Access database. The user informed me that each time a new workshop was created, they would create a document in Word and make multiple copies of it for each date it occurred on. Then they would tack those sheets to a wall. When a person called or came by to register for a workshop, all they had to do was look at the wall to see what workshops were available and what the dates were. Then they just wrote the user's name, phone number and other info on the sheet and re-tacked it to the wall. A week before the workshop, they would take down the sheet and call each registered person to remind them of the workshop. On the day of the workshop, they took down the workshop and gave it to the presenter of that workshop to record attendance.

From this, it was obvious what their "natural workflow" was. It was also obvious that the MS Access database forced them to work differently. With the "sheets on the wall system", all of the needed information was right before their eyes and they did not need to know the person's member ID. The MS Access database forced the user to work in an unnatural workflow. To determine what workshops were available for a certain date/time, they had to navigate through multiple records on the workshop data form. Then once they found the workshop the person wanted to register for, they had to close the workshop form, search for the member ID (usually the person does not know their ID), and finally they had to open the registration form, find the right workshop, and register the user.

I have not attempted to scientifically calculate how much time was lost by using the MS Access database, but I believe it is clear that the cost was high. (In my work, I usually don't get concerned about statistical validity as much as user acceptance.)

My next step was to look around at off-the-shelf applications which were used to perform similar tasks. Many of them worked the same or worse than the one they already had. Some were better but not great. Then I looked at QuickBooks Pro 2000. Although the problems are different, the solutions cross over well. QuickBooks uses an HTML style interface with a few obvious components. (I just want to say that I think QuickBooks implementation is a little overwhelming. There is too much info on many of the screens.)

After playing with the design a little bit, I was able to come up with what I thought was a good start for a workflow solution. My UI solution was to provide the user with an HTML style interface broken down into four areas. (See the screen shot below:)

Image 1

This UI was similar to the old "stick it on the wall" system because all of the information the user had to look for was placed right before their eyes, not hidden in multiple forms, and record navigation. The user can click on any upcoming workshop to access its data form (and from there they can register also). They can click on one of the icons beside the name to either register people for the workshop, cancel the workshop, or print a roster, reminders, etc.

This UI improved on the "stick it on the wall" system because the user could easily filter the information on the screen by the most important criteria (when it occurs). The graphic in the upper-left assists less experienced users by visually representing the workflow process and providing one-click access to the most commonly used functions. (The icons have been whitened out because they do not belong to me, sorry. It looks better with the icons.)

In the end, the users are pleased with this system and feel more productive because of its design. As you can see from this problem and its solution, the efficiency the system needed was not improved performance in the record navigators, improved search criteria, etc. By developing the system to work with the user's workflow, user efficiency is maximized.


In my first article, I discussed the "quick search" feature I developed for this project. I am now going to elaborate a little on this feature.

In this same database, the user often needed to search for a member in the database. (The database is fairly small, about 3000-4000 members.) Also, there are < 1000 organization names in this database. Users need to search for these also. Sometimes, the user needs to search by name (first, last), phone number, address, member ID, organization name, etc.

In the MS Access database, there were multiple forms for the various "types" of searches users performed. There was a form for searching for a member by name, another form for searching for facilities by name, and another for searching by name/address (1 each for members/facilities). Searching by phone number was not available even though the user often needed to do this.

One common approach to dealing with this problem is to present the user with one search form containing all the possible search fields. There would be two forms, one for members and one for organizations. The user could then enter the name, address, ID and phone number, and perform their search against one or all of the available criteria. From what I have seen, this is a pretty common user interface.

The problem with this approach is that most of the time the user does not have all of the information and is unsure, when first approaching the form, whether all of the fields are required to fulfill the search. Also, usually with this type of form, it is unclear to the user whether pressing ENTER on a search field will activate the search or if they need to click a "Search" button. Users tend to attempt to enter as much information as they know and this usually results in very poor search results due to misspellings, missing, incorrect or out-of-data data, etc. This can quickly result in users adding new members/organizations to the database even though they are already present. This is exactly what had happened in the MS Access database. Users could not find people in the DB so they just added them back into the system.

The other problem with this approach is that it forces the user to make a decision about what type of object they want to search for. Although this seems simple enough, this can reduce efficiency because the user just wants to find some record in the database, not navigate through the application's user interface.

The solution I devised for this problem was to provide a single search edit box located on a navigation bar docked to the left side of the screen. The edit box can also be placed on the toolbar when the navigation bar is not present. This search incorporated the "quick search" methodology I discussed in my first article. Basically, this one search control could be used to search for members or organizations and could search by name, address, phone number, ID #, etc. The control has some basic logic to determine what type of info the user enters (i.e., is this a phone number, ID #, text, etc.).

Then the control searches all reasonable columns and tables in the database for matches. The control also has some basic logic to deal with situations like "Mount Saint Hellens", "Mt. St. Hellens", "MT saint hellens", "MT st helens", etc. It also has logic to improve address searches by expanding or removing directional and numeric portions of addresses, street type indicators, etc.

Because the control is located on the navigation bar (and the app uses almost no modal dialogs), the user can perform a search from anywhere in the application quickly and easily. They do not have to decide what they are searching for, they don't have to worry about providing too much info or too little, and it does not interfere with what they currently have on the screen.

Also to deal with more advanced search requirements, two forms exist which the users can use to search members/facilities as described above. These forms are not used very much.

The result of this implementation is that most searches are fulfilled on the first attempt. This reduces database hits, user confusion, and duplicate data being put into the database. Users can do their jobs more efficiently and with less intrusions forced upon them by the software.

What to Expect

When designing software for user efficiency instead of computer efficiency, your skills of user experience design will be tested. You will need to look at other peoples' attempts at solutions, and you will need to experiment with different methods. You will also need to make a concerted effort to understand the user's natural workflow.

I expect failures and problems. The searching algorithm I described above took many iterations to get to where it is. I spent a tremendous amount of time optimizing this algorithm (not for speed) but for user experience. I played with numerous algorithms for expanding and reducing names, addresses, etc. It took me many iterations to achieve a workable algorithm and even now I am still tweaking it a bit.

The rewards of developing software which produces efficient users instead of efficient algorithms are many. These are just a few.

  1. Efficient users - Efficient users are usually happier users. They make better references and are easier to work with. They provide better feedback to management and this can lead to more work, pay raises, etc.
  2. Better applications - No one cares if the XYZ function of an app works amazingly quickly if the XYZ function only gets used once a year. I have spent countless hours optimizing code that barely ever gets used. This can be very frustrating when you reach the end of a project only to find that the user is struggling with the user-interface instead of waiting on XYZ function.
  3. Fewer bugs - Often, optimizing code introduces subtle bugs. Had a particular search been done as a simple linear search instead of a binary search, a serious bug could have been avoided. I am not saying not to optimize code, just optimize the most important ones the most.
  4. Reduced technical support calls - Because the software works the way the user works, users have an easier time learning how it works and what to do. They also tend to remember how to use it better also.
  5. Pride in your work - I find that making applications that are efficient for the users is just as rewarding as making a particular function/algorithm efficient.
  6. Competitive advantage - If your application is easier to use and works better with user's natural work-flow than your competitors', that is a great selling point. If your competition is already doing this, you are at least as good as they are.


This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Written By
Web Developer
United States United States
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

GeneralDatabase comparison [modified] Pin
Lokanatha Reddy20-Dec-06 20:48
Lokanatha Reddy20-Dec-06 20:48 
GeneralAOM and WrokFlow Pin
Christopher G. Lasater14-Apr-05 9:47
Christopher G. Lasater14-Apr-05 9:47 
GeneralThere is basically 3 components of software product: GUI, application logic and data... Pin
Vasek1-Feb-02 8:08
Vasek1-Feb-02 8:08 
I understand why Matt wrote this article. I don't want to argue he had met an awful application just labeled by 'management of somewhat'.

There is many good ideas mentioned in hes article, but all of the ideas listed could be applied to almost any domain of human being. Let me introduce an example.

You want to buy a car. First, you must decide what car is suitable for you needs. According those needs and your financial abilities you choose a suitable car for you.

The problem with software is all the same. Some people wants just something, somehow to register something in their PC, for example. For this purpose the MS Access could be a suitable tool. Namely, if you do not have enough money to pay for:
1] Off-the shelf product or
2] To hire highly experienced professional developer, to be asked to deliver a suitable solution.

People want to buy cars, which are easy to use, without the need of to be an expert in driving and maintaining cars. Of course, there is one steering wheel, switch for lights and many BASIC automobile controls, which can be improved in further by enhancing their features to provide more comfort.

Matt is focusing on the role of user interface in software product. I'm getting suspicious he is overestimating the role of this software layer in the possible software project success. What I really hate is "smarter-than-needed" computer widgets, which tends to make wrong decisions. Soon you can realize the world of your bussiness (or the bussines you are supporting) has changed and your smart component is very difficult to modify, just because it handles such complex task, like a decision whether this number should be automatically interpreted as ZIP number, or telephone routing number.

Searching a database more than one time could be a reasonable simplification for the customer, but only if you have very few information to search for. I have already used this approach. The only difference is that I have bound the decision mechanism as close to data as possible. That means, the mentioning decision mechanism does not reside on UI-side. On the bussiness logic side it is accomodated rather. But still you have to decide if the single dropdown box in front of your search textbox would bother the potential user more than possibility of waiting very long time (more than 10 sec is always very long time). The problem could be escalated by adding more than one user: what about 65 client workstations connected to central enterprise system consisting of one, two, or more servers? Would still be applicable your approach of "simple, clever search"?

Seriously, I am NOT SURE!

As I mentioned above, there are many simple ways how to ask user without bother him decide which search form is the best. First of all, there is a dropdown listbox choice which can be settled in front of your search box. I will be more satisfied as a user, when knowing the machine is searching for the names of my bussiness partners (people) acctually, rather than trying to browse institutions the sought people are working for.

The next possible solution is to show just different forms with different options when the initial dropdown listbox is selected.

The third is to use wizard, or wizard-like web pages.

I am not sure the people tends to fill everything in search form. The more they fill, the shorter will be the result, because the more specific information leads to the more specific search which obviously ends with fewer results. This is very simple mathematic the non-computer people understand.

The application efficiency does not mean only to have handy user interface. The user interface is just ONE of MANY MORE software layers. The data storage component (like a database) must be efficient when storing and retrieving data. The bussiness logic must handle data very carefully, thus optimizing database access (fewer means better). And finally, the user interface must be for users, not for computer specialist.

Why I posted this message to this article? Just to say: I AGREE with many issues listed here, BUT I want to warn against excessive simplification.

That's all. Any of your oppinions are welcome.

VB6, C#, MS DNA, MS .NET programmer
GeneralRe: There is basically 3 components of software product: GUI, application logic and data... Pin
Matt Gullett1-Feb-02 9:27
Matt Gullett1-Feb-02 9:27 
GeneralRe: There is basically 3 components of software product: GUI, application logic and data... Pin
Dag Oystein Johansen3-Mar-04 7:15
Dag Oystein Johansen3-Mar-04 7:15 
GeneralRe: There is basically 3 components of software product: GUI, application logic and data... Pin
bradw2k28-Apr-04 10:19
bradw2k28-Apr-04 10:19 
GeneralRe: There is basically 3 components of software product: GUI, application logic and data... Pin
gfoot21-Dec-04 1:42
gfoot21-Dec-04 1:42 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.