Category Archives: Usability

Brainstorming really is dumb

The dreaded memo has come around again – management has called another brainstorming session. They are getting people together to solve a problem or, more commonly, to discuss future trends or challenges for the organization.

You cringe when remembering the previous awkward sessions, when too many flip charts were filled with half-baked “ideas.” But the managers seemed to be pleased that “people were involved.” If you are an introvert, you remember the sessions as downright painful. And those flip charts that were so important that day – they were soon forgotten as people returned to their usual work.

But everyone says they had a fun day, so we know the next memo will come around soon enough.

Brainstorming has been described as the placebo in the management medicine kit – everyone believes that it works despite clear evidence that it does not. A recent Fast Company article has declared, “brainstorming is dumb.” Brainstorming continues to be popular even though major studies have shown that positive results are rare, at best, and viable alternatives are readily available.

Getting people together to propose new ideas seems, on the surface, like a good thing. However, as the Fast Company article describes, “just because you throw people together doesn’t mean wonderful things happen.” Others have been more critical, describing brainstorming as nothing more than executive entertainment.

So, what does the research say, and what are the alternatives to brainstorming?

To dig into this, we first have to review where brainstorming came from, and how it was supposed to work. Alex Osborn, an advertising executive, made brainstorming popular — most notably in his 1953 book Applied Imagination. Brainstorming was described as a wonderful group creativity technique that could be used to generate great ideas or solve hard problems. The term came from the idea of storming a problem like a group of commandos, with each “stormer” attacking the same objective.

Osborn was quite detailed in describing how brainstorming should be done. He established instructions for brainstorming sessions, many of which are now ignored. Osborn said that brainstorming sessions should:

  • Involve groups of about 12 people, or perhaps less
  • Avoid judging ideas
  • Strive for idea quantity, because quantity leads to quality
  • Promote divergent thinking and wild ideas
  • Welcome combinations and improvements on ideas

Perhaps the most overlooked principles in Osborn’s technique were:

  • Carefully choose the topic to be amenable to idea generation and as specific as possible
  • Provide pre-session training on the technique
  • Provide background information and orientation to the problem before the session

Osborn claimed that brainstorming, conducted with these principles, produced 44% more ideas than people working alone. With this evidence, and the attractiveness of the sessions, brainstorming took off and was widely adopted in many organizations.

Almost immediately, there were criticisms. Soon after Osborn’s book was published, researchers reported that, contrary to the doctrine, individuals often performed better than groups. Promoters of the technique argued, however, that those studies used questionable designs — one widely quoted study compared people working in groups versus individuals working alone, but all participants received brainstorming instructions. So, they argued, this was not a fair test of the technique.

Questions about how to properly test brainstorming remain today. Should we compare?

  • Brainstorming groups vs. brainstorming individuals
  • Brainstorming groups vs. other groups
  • Number of ideas vs. quality of ideas
  • Variety of ideas vs. novelty of ideas
  • With facilitator vs. without facilitator
  • Pre-existing groups vs. random groups

With so many open questions, it should be no surprise that studies that test brainstorming have come up with inconsistent results. But very often, the results are negative, suggesting that brainstorming may not actually work as advertised.

A review by Scott Isaksen examined 90 different studies and found that brainstorming was often not effective. The review also showed that many of the principles proposed by Osborn were not followed in the studies, leading to questions about the validity of the tests. Other studies have often found that individuals produced more, better ideas. Researchers have even proposed that working in groups could actually inhibit the creative process, perhaps due to nervousness, free riding by some participants, and “group think” where teams fixate on a few early ideas.

With the tests of brainstorming showing mixed results, people have started to consider modifications to the original Osborn principles. Some have argued that the lack of criticism, something that Osborn thought was crucial for allowing creativity to flourish, might actually be a problem. Their studies showed that adding debate and criticism to the brainstorming sessions actually led to more ideas.

Other people have started to question the role of the group interaction. With recent tech developments, researchers have started to examine electronic brainstorming, where people work at computers or online. When online, people can work anonymously while still participating in groups, perhaps using a shared editing space or chat service. Working electronically might reduce some of the inhibiting social factors of group work, such as nervousness or introverted personalities.

Another derivation of the original brainstorming technique is brainwriting where, instead of sharing ideas out loud in a group session, people write their ideas down and pass them around. Others can read the ideas and perhaps build on them, while continuing to work on their own ideas. The Fast Company article suggests that brainwriting, done best when writing both alone and in groups, can be a lot less dumb.

So, when the memo comes around again, propose an alternative? Try brainwriting or electronic brainstorming. Groups should exchange ideas when working on a problem, but they should also work alone, probably before they work in groups. The 1950s-style, awkwardly social, too-man-flip-chart sessions are not the best way to come up with good ideas.

Suggested Reading

Brainstorming Is Dumb.

Groupthink: The brainstorming myth.

10 New Rules For Brainstorming Without Alienating Introverts.

Good writing is like a dirty joke, but…

Here is an interesting collection of tips for how to write well. Some are obvious, and some are rather novel and interesting.

Remember that the model of the drama is the dirty joke. This joke begins: “A traveling salesman stops at a farmer’s door” — it does not begin: “Who would think that the two most disparate occupations of agriculture and salesmanship would one day be indissolubly united in our oral literature? Agriculture, that most solitary of pursuits, engendering the qualities of self-reliance and reflection, and salesmanship, in which…”

Source: This Is How To Improve Your Writing: 7 Easy Expert Secrets

CFP: Workshop on Usable Security, due Nov. 6

NDSS logo

I am helping out on the Program Committee for the  USEC’14 workshop.  The workshop will take place as part of the Network and Distributed System Security (NDSS) Symposium in February 2014 in San Diego. Consider submitting your work.

The workshop on Usable Security invites submissions on all aspects of human factors and usability in the context of security. USEC’14 aims to bring together researchers already engaged in this interdisciplinary effort with other computer science researchers in areas such as visualization, artificial intelligence and theoretical computer science as well as researchers from other domains such as economics or psychology.


Nov. 16, CapCHI event, David Barrera on usability and security of Android

photo by laihiu

David Barrera will be speaking on Usability and Security  of Android, Google’s Open Source Smartphone System

Date: Tuesday November 16, 2010
Time: 6:00pm
Place: TheCodeFactory, 246 Queen St., Ottawa

The adoption of Android-based smartphones is growing at a rapid pace (nearly 200,000 activations per day) which has placed Google among the top smartphone system vendors worldwide. Despite Android’s open source nature, there are a number of security and usability issues that have yet to be addressed. This talk will cover issues related to security prompts and notices on the device, permission granting, smudge attacks and application security. We will discuss how these issues affect other platforms as well, including Apple iOS, Blackberry, and Symbian.

David Barrera is a 1st year Ph.D. student in Computer Science at Carleton University under the direction of Paul Van Oorschot. His research interests include smartphone and mobile OS security, data visualization, network security and IPv6.

International Conference on Trust and Trustworthy Computing

TRUST2011 is scheduled for June and the Call for Papers is out. I am on the program committee for the socio-economic strand and papers are due February 15, 2011.

TRUST 2011 is an international conference on the technical and soci-economic aspects of trustworthy infrastructures. It provides an excellent interdisciplinary forum for researchers, practitioners, and decision makers to explore new ideas and discuss experiences in building, designing, using and understanding trustworthy computing systems.

Symposium on Usable Privacy and Security (SOUPS 2011)

SOUPS logoThe Call for Papers for SOUPS 2011 is now out. It is my pleasure to be on the program committee again.

The symposium will bring together an interdisciplinary group of researchers and practitioners in human computer interaction, security, and privacy. The program will feature technical papers, a poster session, panels and invited talks, discussion sessions, and in-depth sessions (workshops and tutorials). This year SOUPS will be held in Pittsburgh, PA.

Papers are due March 11, 2011.

Apps4Ottawa: Build a cool app for your city

contest logo

Ottawa people involved in human-computer might be interested in this…

Do you have a cool idea to mash up some data about Ottawa? Well, the city is running an apps development contest, and the main judging criteria are right up our alley: usefulness, inventiveness, usability, and accessibility.

via CapCHI » Apps4Ottawa: Build a cool app for your city.

Symposium on Usable Privacy and Security (SOUPS 2009)

soups2009SOUPS 2009 is underway in lovely Southern California. Google is hosting the conference this year.

SOUPS is the major conference in the field defined by the intersection of usability and security. The conference includes two tutorials, which took place yesterday, 15 technical research papers, a panel, break-out discussion sessions, and a keynote address.

Check out the conference program at the web site:

SOUPS – Symposium On Usable Privacy and Security

The science of waiting lines

I hate waiting in lines. If I go to a restaurant that has anything more than 5 people waiting, I leave. The thought of queuing up outside a store for one of those big holiday sales is completely foreign to me. I often arrange my daily schedule to avoid waiting in lines: I drive in off-peak hours, eat when others are working, and avoid crowded shopping centers in favor of small locations.

The one thing more frustrating that waiting in a line is waiting in a bad line. You know, those lines that are poorly designed, without clear guidance about where to wait, or what you are waiting for. Or lines where it is clear that there are not enough services at the end (cashiers, clerks, or kiosks) to handle the number of people waiting.

What makes this so frustrating is that waiting lines don’t have to be so terrible. There is a science of waiting lines, and if organizations follow the science they can make their lines more efficient and more pleasant. Waiting times are a necessary evil. Having some form of a queue is often the best way to ensure that a system or service works as efficiently as it can. Queues make sure that each resource, be it a cashier or a customs official, never goes idle waiting for the next customer. Queues that are too long can drive customers away, but queues of the right length can increase efficiency and save money.

Donald Norman, a popular usability guru, has recently outlined some valuable design principles for making good waiting lines. Many of these are common sense, but when they are not followed the effects are usually obvious, and unpleasant.

One of the most important principles is that human emotions dominate the experience of waiting in lines, and that emotions are affected by context. Setting a pleasant context for a wait in a line can make all the difference. A location that is warm, bright, cheerful, and welcoming will be far better tolerated. This Disney theme parks, which are synonymous with long lines, are very good at setting a good context for a waiting line, often having costumed characters entertain people while they wait.

My son, who has more patience than me, recently got into one of those multi-hour waiting lines for a Boxing Day sale at an electronics store. The store did many things to make the wait as pleasant as possible: they gave out coffee and snacks, they gave people a number so they could take a break and return to the line, and they held raffles for free merchandise. This is also a good example of a related design principle: keep people occupied. Peoples’ perceptions of time and space are influenced by their surroundings. Many service environments, such as waiting rooms, now have TV’s and advertising displays to keep people occupied.

Emotions spread from person to person. So, in managing a good waiting line, it is important to watch for and address any negative emotions. This means paying attention to the most upset customers so that their negative emotions do not spread to the other people waiting in the line.

Another design principle is to eliminate confusion by making it clear and unambiguous where the line starts, where it ends, and how it works. Perhaps the worst waiting experience is finding out you have been waiting in the wrong line. Some of the best lines I have seen at airports are where one employee at the head of lines provides reassurance that people are entering the correct queue.

Providing feedback on how the line is progressing is also important. Nira Munichor and Anat Rafaeli from Isreal recently conducted a study of telephone waiting lines. They had participants either listen to music while they waited for a telephone service, or the people heard music and one of two spoken messages. The first message apologized for the wait and asked people to remain on the line. The second message provided information about where the person was in the line (e.g., “You are currently third in line”). Call abandonment rates (the frequency by which callers hang up) were 50% lower with the informative position information than with the music or apology conditions, and the customer experience was described as much better. It is possible to design good waiting lines, even for the telephone.

Waiting lines also need to be appropriate: long waits for trivial reasons or trivial goals won’t make any sense. Waits also have to seem fair. If there are a lot of people waiting and yet only a few service locations open, such as at a bank, then this appears to be unfair.

A lot can also be done to improve the waiting line experience by changing the way the line works. Certain kinds of layouts can make lines more efficient, faster, and therefor more pleasant. You may have seen cashier stations at cafeterias, for example, that have two sides for serving customers on the left and the right. This is because the process of cashing out a customer can be inefficient, with time spent while the customer puts items on the counter, finds money, packs up the items again, etc. While a customer is busy doing those things on one side, the cashier can serve another customer on the other side. For situations like this, this kind of arrangement makes the lines work much better.

Another example of changing the way a line works can be seen in drive-through restaurant windows. These restaurants often have one place for giving a food order, another place for paying for the food, and a third place for picking up the food. And, these windows are often far apart. This physical distance means that, as long as there are a few customers, there are built-in delays in the process. This is important because preparing the food takes time, but by designing the line in this way customers are seldom left waiting for their food to be prepared — by the time they have visited all the windows the food is ready.

Another design decision is having multiple servers with multiple lines, as is usually done in super markets, or multiple servers and one line, as is usually done in banks. One line can appear to move faster and appear more fair, and a single waiting line automatically adapts if one of the servers is slow or encounters an unusual circumstance (one of those people with dozens of coupons).

Of course, sometimes the best design is to avoid the line in the first place. Reservations are a method of avoiding waiting lines. I recently traveled to Europe and visited a number of museums and art galleries. I did a lot of research to make as many reservations as I could, even if it cost more, to avoid the lines. Even though every guide book I read explained how to do this, I was amazed at the thousands of people who ignored the advice and waited in line after line. If you get to Rome and want the visit the Vatican museum, make a reservation for one of their wonderful guided tours and don’t wait in the infamous lines (and don’t miss the Borghese Gallery, where you have to have a reservation).


Donald Norman, D. (2008.) The Psychology of waiting lines.

Munichor, N. & Rafaeli, A. (2007). Numbers or apologies? Customer reactions to telephone waiting time fillers. Journal of Applied Psychology, 92 (2), 511–518.

My favorite comedian, John Pinette, also hated waiting in lines: