evaluatingdesign.com

I’ve got an interesting post about FIFA 07 and human error, but wanted to post a quick link to an article that I’m looking forward to reading - called “The effect of available choice on cognitive processing of pictures”:

http://journalism.missouri.edu/news/2007/07-12-online-content-study.html

This study (soon to be published in the journal Computers in Human Behavior) argues that users allocate attention to a small array of pictures as compared to a large array. Participants were asked to select three pictures they would like to examine more closely from a set of either 6 (small array) or 24 pictures (large array). After looking at one of the group of pictures below, participants performed a quick distraction task, then were given a picture recognition test.

The study has two main effects - one for visual attention, and another for memory. Here’s the gist:

1. Participants who looked at the smaller picture array showed a cardiac orienting response (a short term change in heat rate) while selecting pictures. Participants who looked at the larger picture array showed no such response.
2. After the distraction task, participants who chose from the smaller array remembered 98% of their pictures, while participants with the larger array remembered 89%.

These two findings are used to argue that there is a lack of available mental resources - the researcher categorizes this as the difference between “getting there” and “being there”. Quoted from the article:

“At some point, our mental processing resources become overloaded and cannot efficiently process new information without sacrificing old information. More mental resources were utilized when participants selected from 24 pictures than from six pictures, and this left participants who selected from the 24 pictures with fewer mental resources to devote to encoding the pictures they selected,” Wise said. “When the process of ‘getting there’ requires greater cognitive effort, fewer cognitive resources remain to encode content while ‘being there.’”

Let’s think about both of the main effects in this study:

Cardiac Orienting Response

That COR shows a difference is interesting, but what does this actually mean? We know that COR is affected when attention is captured, but what does this mean for the comparison of picture arrays? Is COR a generalized measure of attention, or a focused measure of attention? If focused, could it also be that this physiological measure is capturing different strategies users leverage when faced with more information? It would certainly seem so, based on other visual research.

Peterson, Boot, & Kramer (2003) showed that participants showed near perfect memory when searching a display of 12 items, while displays that lacked as many environmental cues showed a memory capacity of near 4 items. A feature-rich display can provide evidence to guide the search process. However, landmarks seemed to lead to smaller memory spans.

Consider this in the context of visual search and focused attention. If participants are being guided by environmental cues and not focusing their attention, perhaps COR would not have as strong an effect as a limited environment which encouraged the capture of focal visual attention, versus scanning of the environment (which may use more generalized visual attention). If COR is the capturing of focused attention, perhaps a second measure should be used to define visual search. It could be that users are implementing a different strategy for a different environment - more on this below.

Limitations in Memory

The Rule of 7 is well known in the human memory - essentially, users seem to be able to process and hold 7 +/- 2 items in memory at a given time (It’s likely that memory is closer to 5 +/- 2, but that’s another post). More than that, and the human is going to be limited in their recall.

Let’s think of this in terms of this study. One group is presented with 6 icons and asked to remember 2-3. They’re then tested, and come out at 98%. Pretty good, and definitely within the bounds of what we know about human memory.

The second group is presented with 24 icons, and asked to remember 2-3. They’re tested on their icon memory, and show about 89%. Also not bad! Memory can be disrupted a large number of factors; workload, interruptions, and environmental noise. This difference in memory may be a simple as users being required to do a larger amount of scanning to determine the pictures they would like to select.

At this point, there is also the question of statistical significance versus practical significance - if your users are picking up on 90% of all information, and reducing your array by 75% only gains you another 8% in accuracy, is it worth reducing the number of items being presented? Especially when both group’s accuracy is so high?

There is no “There”

My interest is in the concept of “getting there” versus “being there” (which sounds suspiciously like the difference between scanning and encoding information). I’m not sure (from the information presented on the web page) whether we’re really talking about a trade-off in mental resources affecting performance, or if we’re talking about two groups adjusting the actual task they perform based on the environment the task is to be performed in. If it is a difference in tasks, then perhaps there is no “there” - that is, no resource-depleting bottleneck that trades off mental resources to perform a similar task to get “there”, but rather a set of separate tasks that are employed dependent on the environmental demands. More on this once the full article comes out - I’m very interested to read more on this study!

References

Diao, F and Sundar, S.S., (2004). Orienting response and memory for web advertisements: Exploring effects of pop-up windows and animation. Communication Research, 31,(5) 537-567.

Kostiuk, K. (2007). “Less is More” Online. Accessed from http://journalism.missouri.edu/news/2007/07-12-online-content-study.html on July 19th, 2007.

Miller, G. (1956). “The Magical Number Seven, Plus or Minus Two” Some limits on our capacity for processing information. Psychological Review. 63. 81-97.

Peterson, M. S., Boot, W. R., & Kramer, A. F. (2003). Environmental cues modulate memory during visual search [Abstract]. Journal of Vision, 3(9):626, 626a, http://journalofvision.org/3/9/626/,

If you enjoyed this post, make sure you subscribe to my RSS feed!

I was looking at a couple of articles this morning about “starting off on the right foot” with a new position. Here’s a couple of articles I recently checked out:

http://www.careerjournal.com/jobhunting/jungle/20070627-jungle.html?cjpos=h

http://hbswk.hbs.edu/item/4860.html

Both articles stress the importance of making an initial good impression. Also, both articles emphasize accurately perceiving the implicit qualities of the work environment. A great example of this from the aforementioned Wall Street Journal article:

• “The work world is more ambiguous than school, where students get explicit instructions and regular feedback. On the job, some bosses explain assignments and culture, but many don’t. Often, the onus is on the new hire to figure out how to behave.
  Career advisers recommend that new grads observe carefully during their first few weeks on the job. It’s better to watch and learn at first, they say, than unknowingly blurt out something that causes offense — such as criticizing a project that turns out to be the boss’s pet. Watch co-workers to learn how people dress, the hours they keep, how long they take for lunch and how they interact with each other and their supervisors. Even something as simple as how loudly co-workers talk can be important.”

This type of advice, however, assumes that you’re on-site, and available to pick up all of these implied cues. What happens when you’re telecommuting? How do you integrate into the culture of an office when the culture isn’t immediately accessible? How do you communicate your own professional image when you’re usually a voice over the phone?

A quick online search pulled up the following tips:

• From “Making Telecommuting Successful: A Guide for Employees“.

  • Ask for a volunteer “office buddy” to email you office news
  • Request weekly feedback on how the arrangement is working
  • Make calls, send memos, and don’t let people forget you
  • Keep in touch to “keep in the know” through maintaining your relationships
  • Send out emails and updates of your projects
  • Be flexible with your schedule and the office schedule
  • Establish “office hours” and ask colleagues to contact you during those hours
  • Keep a watch on the perception of your in-office co-workers
  • Know their schedules so you can find them when needed
  • Take credit where credit is due — for yourself, your office co-workers, and your boss

However, this information seems like general advice on being a decent employee (regardless of whether you work on-site or not). So what do we know specifically about integrating into a company’s culture when you’re not on-site?

One solution may be a mentoring relationship; a 1 on 1 interaction with a senior employee that allows for the transfer of relevant information about the culture of the company from more senior members to incoming employees who are attempting to perceive the values and requirements of a new workplace.

The Wall Street Journal article on adjusting to a new workplace mentions the value of mentorship, and how it can help to understand a new workplace.

I found this powerpoint online that summarizes many of the main points of a mentor/mentee relationship that apply to new workers. This seems like the type of interaction that could allow you develop some understanding of the implicit structure of an organization & it’s informal culture. More on this as I move forward.

If you enjoyed this post, make sure you subscribe to my RSS feed!

Just got an email from the Air Accident Digest, who ran a story on the FAA Airport Design Award we won in May (this link includes the full article).

Below is a picture of a scenario from the design. For more information on the competition and our award, click here.

Here’s a link to the Air Accident Digest Article.

If you enjoyed this post, make sure you subscribe to my RSS feed!

Just listened to an interesting podcast on the role of price point in the ratings score of a game. This is specifically in the context of Shadowrun, a new Xbox 360 FPS that developed some new tweaks for multi player gaming, but did not feature a single player campaign. Mitch Gitelman (who’s definitely passionate about the game and his work) makes some interesting points about game reviews, and how ratings affect users’ perception of game value (listen to the podcast, I’m not giving it away here. It’s a great interview about price, value and some great insight into how proximity.

• One of the underlying themes is that the reviewer’s ratings may have been skewed downwards by the game’s price point, rather than reflection only an evaluation of the game features.

• The other is the question of what does a game rating actually mean, and how do you quantify between multiple review sites that are giving different ratings?

One of the underlying issues here is the discrepancy between user’s ratings of subjective preference (Ex.: What would you rate a game on graphics on a scale from 1-10?) and subjective value (Ex: Given a limited amount of resources to spend on a game, how much would you spend, and on what?)

But how to address the difference between preference and value?

One interesting project I’m collaborating on with Julius Najab is quantifying the relationship between user’s preference of game qualities (how important the qualities are) and their perception of the same qualitie’s subjective value. For example, if you only have 60$ to spend on a game, what game aspects would you invest your money in?

The hope is to address exactly the types of issues that are brought up in the podcast - to standardize game reviews by introducing measures of subjective value. I’ll link more results as we analyze some early data.

If you enjoyed this post, make sure you subscribe to my RSS feed!

One of the new iPhone commercials does an excellent job of displaying the beauty of task-based design, as well as the value of using scenario based design to create novel interfaces.

The danger we face, as designers, is creating a design that can respond to the scenarios we don’t anticipate - the designs that address the “What Now?” in the schematic below.

The commercial of the iPhone shows a relatively elegant approach to search and mapping a particular location. The real question is how well the interface handles a task that isn’t immediately distinguishable from the icons on the top level. The iPhone iconography in the commercial shows 10-12 icons - what happens when the task we need isn’t immediately addressed (or immediately identifiable) by the tasks or programs listed on the top level?

On a related note…

A recent iPhone usability study by User Centric noted many anticipated iPhone tasks performed very well. Several tasks that had potentially not anticipated the actions of the user (such as editing mid-message and fine control of Google maps) seemed to frustrate the user.

What do both of these have in common?

In both cases, success is directly related to how well the display and it’s functionality is mapped to the task at hand. The challenge is always to map tasks in a manner that allows the user flexibility. When the user encounters functionality that doesn’t “work” in an anticipated fashion, frustration is the natural reult.

So how do we analyze for both anticipated and unanticipated tasks?

Three methods come to mind:

1. Task Analysis - This requires modeling the task as the designers anticipating it being performed with the interface. The upside is that the models created can be directly applied into the design of the interface. This also helps to validate that the task is as simple as possible, and contains no redundant or misleading steps. The downside is that task analysis requires that you anticipate every possible task that may have occurred - if a task hasn’t been addressed in the analysis stage, the resultant design may not be optimized for the task it’s designed to support. (See Kirwan & Ainsworth, 1992, and Hackos & Redish, 2003).
2. Scenario Based Design - Oftentimes, analyzing tasks without considering the environmental factors the task must be performed in can lead to designs that do not adequately account for user or environmental interactions that may alter how the task is performed. (see Carroll, 1995 & 2003)You can think of any product or interface use as an interaction of the user, the task to be performed, and the product the task is to be performed with in the context of the environment. Another way might be to visualize it like this:
   

   

   Note: This framework comes from Gray and Altmann (2001), and was recently used in Debbie Boehm-Davis’ TIES article on product design.

   There are some limitations to scenario-based design, however - similar to the limits of task-based analysis, it requires that the user anticipate and develop the scenarios that the user will face. Similar to the iPhone commercial above: Scenarios are great if you’re switching from a movie to looking for seafood, but what if the scenario demands something other than that? How does the design repond to different scenarios - especially if the designer has not anticipated that particular co-occurence of task, product, and environment?

3. Work domain Analysis -Domain analysis is different in that it ignores the actions being performed and concentrates on the boundaries of the system the interface is to be designed to address. By concentrating on the abstract system properties that need to be addressed, designers can anticipate a range of activities that must be represented. For more information on using work domain analysis in design, see my recent work on WDA with a mobile device, and this introductory white paper.The caveats - the more complex the system, the more value this technique has. If you’re designing a simple interface, the time invested in the analysis may outweigh the design insights. Also, this technique does require some training (my own training took multiple years, several design projects, two workshops with the creators of the techniques, and I’m still far from an expert), which may inhibit many from implementing it correctly (to maximize insight generated).

In Summary

Any of the above techniques could be used (either separately or in concert) to create exceptional designs. There are three main questions to ask when creating designs using any technique:

1. Is the design optimized to support the tasks or actions the design is created to support?
2. Does the design adequately account for the interaction of user, task, and environment?
3. Has the design adequately accounted for the environmental constraints that define the user’s interaction?

Answering these questions can lead to a good design (or in the case of the iPhone above, a plate of good calamari).

If you enjoyed this post, make sure you subscribe to my RSS feed!

With the resounding joys of Bomberman Live being proclaimed far and wide across the internet, I was piqued into dropping some MS Points on my very own copy of the game. After reading reviews here, here, and here, I assumed that I’d have a game that could best the fun of the Turbo-Grafx version.

I’ve had several friends over for 4-5 player sessions on the Wii, the thought of hitting it up on Xbox live had me excited. So Dhvani and I fired up the 360 and played a 1-on-1 match first.

After about 30 seconds of silence, I finally broke the ice with, “..umm..why is it so slow?” Turns out I wasn’t the only one feeling the lag. The new Bomberman requires you to start off at a speed akin of a snail. As it turns out, you have to speed up the proceedings with roller skates (similar to the earlier versions) - but instead of a categorical shift in speed, you now have a continuum. Doesn’t sound so bad, right?

Except that the beginning of the continuum is SLOW - completely non-fun. For those who played the version on the Wii, the beginning speed of Bomberman Live is similar to the original’s speed if you hit a skull and were diseased for a short time - imagine that for the start of EVERY match.

I feel like a minority given the glowing reviews online, but part of the fun of Bomberman was zipping around the map from the start of multiplayer, dodging predictable bombs (and occasionally eating one you weren’t prepared for). It was fun that fit with the manic feel of the game. The new game keeps the flow, but slows down the pace - which is NOT a good thing. bomberman Live feels more deliberate - bombermen move slower, & the bombs have larger range, forcing you to slow down and hide behind blocks. Is Bomberman fun when you’re sticking and moving - with the emphasis on the sticking? Play the Wii version and compare.

Under all of this is the issue in determining what truly makes a game fun - the core qualities of a particular experience that should be maximized for “fun” user experience.

It makes me wonder what qualities the game designers chose to maximize - i.e., what abstract qualities were maximized for the release.

• Was it customizability?
• Multiplayer?

What influenced the design decision on the pacing of the game?

How to Identify what’s Fun in a Game

A recent project I’m working on with several members of my lab is attempting to quantify the relationships that define what’s “fun” about a particular game. The hope is to create style templates on games, to help guide designers on what the “fun” aspects are of specific game types.

For example, different concepts might load on the concept of fun for a game like Bomberman. Imagine we use a forced choice scaling technique to generate a “loading scale” of how much each concept involved in the game influences the perception of what’s fun. Maybe bomberman would look something like this:

Given this (example) loading, should the default settings balance options over speed? Should the game have opted for a reward schedule of power-ups, or focused on high speed for all characters?

One of the hopes of the current research program is to provide specific loadings that designers can reference when asking the question of “how can we improve on a current game?”

More on this as we progress with our data collection and analysis. For now, I’m just going to avoid Bomberman Live.

Note to Designers - don’t make the floor, the obstacles, and the bricks less than a 1-4 contrast ratio from each other. Really.

If you enjoyed this post, make sure you subscribe to my RSS feed!

I got an email from a friend (Andrew C.) who has created his own drinking game based on Trauma Center: Second Opinion.

I thought of rewriting, but I think his email says it best:

“Dear Wii owners:

A few weeks ago I rented Trauma Center: Second Opinion. The gameplay was a little slow at first, but within 30 minutes we had a fast an furious drinking game on our hands. I had so much fun with it I had to share it.

Afterwards I went back to the game without booze, and found it was very fun after the first few “learning” levels. I definitely recommend renting it for a week. After that you’ll probably be done with it (though I’m thinking about buying it, I have very fond drinking game memories =) ).

When you (notice it’s not, “if you”) get Trauma Center: Second Opinion the following rules are in place to better your skills:

1. Whenever the doctor “excises a tumor”, drink.
2. Whenever a nurse says, “Doctor!” (outside the context of a sentence), drink.
3. Whenever a nurse scolds you, or makes an angry face, drink.
4. Whenever the doctor busts out his Public Enemy pose (you’ll know what I’m talking about), drink.

Above: The doctor from Trauma Center and his mentor, Flavor Flav.

I recommend each player has their own bottle of wine or a forty.
Expect a forty to be consumed in 1 to 2 hours of play.

Doctor!,
-Andrew”

If you enjoyed this post, make sure you subscribe to my RSS feed!

Good times when the New York Times posts up a feature about usability.

If you enjoyed this post, make sure you subscribe to my RSS feed!

When you’re a Human Factors student entering the workplace, where do you think you’ll find more assistance - your university, or your professional society? What services do human factors students value from their organizations, and how do these students perceive their current organizations’ effectiveness in preparing them for a career?

An article I collaborated on with Don Horvath and Peter Squire (featured in this month’s HFES Bulletin) addressed these issues via an online survey of human factors graduate students.

• Check out what students reported as the five most effective organizational services:
  

  Universities	Professional Societies
  1. Mentoring	1. Networking
  2. Networking	2. Job listings/searches
  3. Internships	3. Conferences
  4. Research opportunities	4. Mentoring
  5. Job opportunities	5. Career fairs/career information

There’s a large amount of overlap between services provided, but note that:

• internships don’t even make the list for professional societies.
• mentorship is ranked low among services offered by professional groups.

More interesting, however, is when we asked students about what services they’d like to have that aren’t currently offered. Their top five wish list is below:

• • 1. Mentoring
    2. Networking
    3. Job listings
    4. Internship opportunities
    5. Scholarship opportunities

Compare this list to the table provided above. The top three services students most wanted were services that are already offered!!

This suggests one of two things:

• either students want improvements to current services
  or
• students are simply unaware that the service already exists.

We ended with a challenge to our professional societies to address some of these issues in the coming year. Check out the article below:

Quick References:

Horvath, D.J., Smith, C.F., & Squire, P.N. (2007). From student to employee: What’s HFES got to do with it? HFES Bulletin, 50 (7). Human Factors and Ergonomics Society, San Diego, CA, USA.

If you enjoyed this post, make sure you subscribe to my RSS feed!

I was talking with one of my friends (and fellow Wii player) Daragh about how the Wii’s controls are an excellent example of the application of affordances. A quick breakdown for those unfamiliar with affordances; the term was first introduced by James Gibson as possibilities for action in the environment. For example, stairs might afford climbing. Don Norman used the term as action possibilities that can be quickly and easily perceived by the operator. (For more information on the differences, see Norman’s explanation.) This explanation has been expanded in usability circles to talk about how well a device “suggests” its purpose to the user. But what does this have to do with the Wii?

Check out the demo video below of the Wii game WarioWare: Smooth Moves:

Games like WarioWare rely on the ability of the user to rapidly perceive how an object is supposed to work. So using simple usability measures (time to completion, # errors, # of trials to success), we can quantify (and evaluate) how well an item affords a particular action.

Imagine how this could be used for future product design! Games like WarioWare use rudimentary graphics, opening up real potential for rapid prototyping. Furthermore, the Wii’s opera browser functionality could open the door for programming simple flash mock-ups of designs. These designs could be quickly evaluated before a product was even developed into a physical prototype, creating the opportunity for user evaluation and feedback in the design process to improve the final product.

I’m considering a quick usability study using the Wii and WarioWare to show off the possibilities. More on this as it comes around.

Couple of Quick References:

Gibson, J.J. (1977). The Theory of Affordances. In Perceiving, Acting, and Knowing, Eds. Robert Shaw and John Bransford.

Norman, D.A. (1999). Affordances, Conventions and Design. Interactions 6(3):38-43.

If you enjoyed this post, make sure you subscribe to my RSS feed!