Expert Review | academic rigour VS real world practice

I recently completed an internship at a usability copmany for my Msc. During the internship, it came to my attention, the notable difference between the teachings of usability research methods in universities, and actual real world practice. Having had exposure to how things work on both sides of the fence, I found that the differences were particularly evident in the case of expert review (i.e., heuristic evaluation).

The difference is rather straight forward. The method currently being thought in the universities encourage that no stone should be left unturned in ones heuristic evaluation, & that Nielson’s ten heuristics are (amongst a few other sets, such as Keith Innstone’s) ‘THE’ best practice guidelines to evaluate a site against. Students are thought to be meticulous in their evaluation, running each element of an interface past each heuristic and asking the question; is this heuristic violated in any way here? I have attempted this pain staking process and any honest man or woman who has also attempted this will tell you; it just doesn’t work that way. It is not human nature to work that way. In fact, I don’t even believe that Nielson believes it works that way.

In reality (i.e., the workplace), it is more so human nature, to make your first pass over an interface and already have an idea of the problems you will be reporting. You are then quite likely to discover problems independent of the heuristics, before then trying to find a home for the problems within the heuristics. One might now argue that there is no need for greatly specified rules of thumb, where in fact, a set of high level guidelines would suffice.

My argument is that we as ‘experts’, who are conducting ‘expert’ reviews should be taught to work this way in the first place. I mean, we are experts aren’t we? It would seem that Nielson’s intended method is more suited to those who are not experts. Why should an ‘expert’ not be encouraged to identify problems using their ‘expertise’ rather then following someone else’s rule book. Sure, teaching this scientific method will instil rigour in students. But this needs to be done in some other way that will prevent people like me having to unlearn things I have been taught, once I reach the workplace.

Has our relatively young industry already fallen fowl to the same pitfalls of general web design, where students in universities (particularly in Ireland) courses are not being thought cutting edge practice?
What has this got in hold for the industry? Even more so, what has it in hold for students currently pursuing HCI as a research area? Will students like me quickly have to unlearn the methods being thought –OR- should they/we encourage this rigour in practice?

If it's not broken, why fix it?

My regular post-lunch trip to the canteen today to get water was dampened some what (excuse the pun) by what greeted me on my arrival. The old water machine had been replaced by a new shinier & streamlined version of its predecessor.

I was at first, I have to say, impressed by the new dispenser. It caught my eye immediately and stood out with authority. However the problems began when I unscrewed the lid from my water bottle and bent down to get the water. I placed the neck of the bottle between my forefinger and middle finger and tried to press the button with my thumb.

The problems with the new machine

Problem the First: It was not possible to see where the water would be coming from, so i had to do some guess work as to where to position my bottle, before I pushed the big shiny button.

Problem the Second: The distance created between where i held the bottle and where my thumb needed to be to press the button was too great. As a result my hand became unsteady and the water by passed the nozzle of the bottle and, instead, took a detour onto my trouser leg via my wrist and arm.

Problem the third: The speed of the water coming from the dispenser was notably slower then that of the old one. This made the entire experience of holding a soaking wet, unsteady hand underneath a water nozzle that I could'nt see a little bit uncomfortable!! Not to mention the fact that my trouser leg was now a little bit too wet for my liking.

The whole experience left me thinking on the way back over to the office, “why did they get rid of the old one?”

In reality, it is a typical example of where style, flashy buttons and a curvy design prevailed over simple, honest, good usability.

What became of the old machine then?

So what had come of our beloved water machine that had suddenly disappeared? Well, after a little investigation work i was directed to the skip behind our offices and low and behold, there it was, sulking in the skip, having done nothing wrong whatsoever except try its best!