How much can eye-tracking studies tell us about the ways that people read and react to search engine results pages? What is a search engine like Google doing when it comes to using eye-tracking?

I have to confess that I’m skeptical when I read about some of the studies that describe a golden triangle of Google and show nice heat maps which attempt to indicate how people view search results. Why the skepticism?

It began with one of the more interesting eye-tracking studies that I’d seen over the past few years, where the Poynter Institute used eye-tracking to test the way that people read news web sites. Excited by the results, I wanted to find out more, and started searching around.

I came across a blog post from Frank Spillers which looked closely some of the many issues involving eye-tracking, specifically within the context of that Poynter research – Eye-Tracking studies- Usability holy grail?

There are some great takeaway points from this blog post, and I think one of the most important is this one:

Bottom line: If you are using eye-tracking, to make it meaningful, you must:

…

  1. Match what users are doing and feeling with the eye-tracking data reports. Data is just data unless it is meaningful and informative.

My understanding of the “golden triangle of Google” research I linked to above is limited to what I’ve seen about it described at a couple of Search Engine Strategies sessions and in a few blog posts. Yet I’m left wondering why there’s only one heat map talked about and shown to describe the way people look at search results. And, there’s no insight into what people are doing and feeling when that golden triangle is shown.

Even on a simple level, shouldn’t the way someone looks at search results pages be different if the type of search was for different purposes? Say, for instance, a navigational search as opposed to an informational one, or a transactional one? See Andrei Broder’s A taxonomy of web search (pdf) for more on the differences between those types of searches.

So, how is Google attempting to use eye-tracking to understand how people look at their pages?

A paper prepared by a couple of Googlers, to be presented at the CHI2006 in Montréal this past April, gives a little insight into the backgrounds of Google researchers, and their “initial experiences with incorporating eyetracking into user studies at Google.” Laura Granka and Kerry Rodden wrote and presented Incorporating Eyetracking into User Studies at Google, and I’m fairly comfortable with their use of eye-tracking after reading their description of their approach.

Here’s why:

  1. Their statement that: “in general, existing eyetracking software lacks specialized features for analysis of studies where web pages are used as the stimuli, e.g., dealing with repeat visits to the same page, or page content that changes dynamically.”

  2. Google has access to log file information to tell them what people were doing on pages, and eye-tracking information can be used to help understand that behavior instead of being used to define it.

  3. A clear understanding and explanation of the importance of context, as shown in this statement:

While interpretation standards for the meaning of ocular indices are desired, another important issue to consider is that task significantly influences viewing behavior. Task differences most notably affect a user’s scan path, but they also lead to different interpretations of certain ocular indices. For instance, fixation duration can take on a different meaning based on whether a user is performing a reading task or visual search task. Furthermore, scan path and the number of fixations in specific areas of interest also differ based on the task a user was given.

  1. The desire to supplement eye-tracking information with other information, such as click-throughs and mouse movements, and any correlations between them.

A post from the Google Research Blog back in April describes some of the other Google usability testing presented at the CHI conference.