Sunday, March 30, 2008

Neural Network Gaze Tracking using Web Camera

Using a simple low resolution web camera without IR illumination David Bäck uses a neural network approach for eye tracking. Low accuracy (2-4 deg.) mainly due to the poor image quality from the web camera and no IR illumination. However, a nice introduction to the basics of technical implementation.

Abstract
"Gaze tracking means to detect and follow the direction in which a person looks. This can be used in for instance human-computer interaction. Most existing systems illuminate the eye with IR-light, possibly damaging the eye. The motivation of this thesis is to develop a truly non-intrusive gaze tracking system, using only a digital camera, e.g. a web camera.

The approach is to detect and track different facial features, using varying image analysis techniques. These features will serve as inputs to a neural net, which will be trained with a set of predetermined gaze tracking series. The output is coordinates on the screen.

The evaluation is done with a measure of accuracy and the result is an average angular deviation of two to four degrees, depending on the quality of the image sequence. To get better and more robust results, a higher image quality from the digital camera is needed."

Download paper as pdf

Mix08 Talk: New forms of interaction

Mainly on interfaces based on multitouch and speech, however many issues are relevant for gaze interaction as well.

Title:
Touch Me: Where Are Interfaces Going?
Speaker(s): Chris Bernard, Dale Herigstad, Daniel Makoski, Dave Wolfe, Doug Cook, Yoshihiro Saito
Description: The keyboard and mouse are aging input devices. For the future of computing and UX, where are interfaces going? Are these enough? Is touch-screen Surface/iPhone/iPod Touch just a gimmick? Where should Man Machine Interface (MMI) go?

WMV link: http://msstudios.vo.llnwd.net/o21/mix08/08_WMVs/PNL10.wmv
MP4 link: http://msstudios.vo.llnwd.net/o21/mix08/08_MP4s/PNL10.mp4

WPF and Windows Media Player COM control

The WMPLib offers direct control to Windows Media Player to your C# applications. This provides a rich set of events and controls that is really useful if you would like to create a custom media player.

The code I wrote scans the computer (My Music) and builds up a library of artists, albums and songs using the directory structure. Any images found are used to create folder icons. I later found out that it is possible to access the Windows Media Player media collection library to access this information so next time I´ll probably use that) However, this requires the albums to be imported/added to the Windows Media Player.

There is a few neat trick available when using the WMP. One is that it sends events when ever the player state changes (see list of events). This enabled me to initiate a progress bar that displays the current position in the song (code 3) as well as skipping to the next song when a the current playing song was completed (code 8).

To add WMP in you code

Using WMPLib;

WindowsMediaPlayer _wmp;

public MyApp()
{
_wmp = new WindowsMediaPlayer();
// The event below tells WMP to call your own MediaProcess code
// when the WMP player changes state
_wmp.PlayStateChange += new _WMPOCXEvents_PlayStateChangeEventHandler(MediaProcessChange);

}
private void MediaProcessChange(int newState)
{
//The integer "newState" contains the state of the media player, ranging from 1 to 11
// see http://msdn2.microsoft.com/en-us/library/bb249361(VS.85).aspx for mor infomation

switch(newState)
{
// case 1: // Stopped
// case 2 : // Paused

case 3 : // Song starts playing

progress.Maximum = _wmp.currentMedia.duration;
progress.SetValue(ProgressBar.ValueProperty, _wmp.controls.currentPosition);

BackgroundWorker _worker = new BackgroundWorker();
_worker.WorkerReportsProgress = false;
_worker.WorkerSupportsCancellation = true;

_worker.DoWork += delegate(object s, DoWorkEventArgs args)
{
while (_wmp.controls.currentPosition <>
{
// Dispatcher to update the U.I progressbar every two seconds
Dispatcher.BeginInvoke(DispatcherPriority.Background, (SendOrPostCallback)delegate { progress.SetValue(ProgressBar.ValueProperty, _wmp.controls.currentPosition); }, null);
Thread.Sleep(2000);
}
}; // End delegate
_worker.RunWorkerAsync();
break;

// case 4 : // ScanFordward
// case 5 : // ScanBackwards
// case 6 : // Buffering
// case 7 : // Waiting

case 8: // Song ended, play next

PlayNextSong(null,null);
break;

// case 9 : // Transitioning
// case 10 : // Ready
// case 11 : // Reconnecting
}
}

The "progress" object is a XAML added obj of the type ProgressBar. These handy U.I elements can be set to the maximum value (instead of 1 to 100) The maximum value I set to the entire lenght of the song. This way the current position scale to the bar.

There is a good tutorial for making a media player at Pauls Blog. Additionally, another custom media player by Sacha Barber. The MSDN network has extensive API info as always (but perhaps in a really boring format)

Friday, March 28, 2008

Gaze Media Player

The component I´ve been working on is now capable of the basics. Great feeling just looking a song titles and then skipping through the playlist by gaze =) There is room for improvements, would be nice have a component like a slider where one could go to a specific part of the song. (don´t know how many times I´ve been playing guitar to a song while learning it and going back and forth between mouse and guitar. Let´s see what a weekend could do =)

Screenshot of the music player component, layout not finalized.. Updated version has a song progression bar and volume controls.

The play-button houses another ellipse shaped menu with the regular controls for (next, previous, play, stop)

Thursday, March 27, 2008

RApid GAze-Based Interaction Techniques (RAGABITS)


Stephen Vickers at the Computer Human Interaction Research Group at the De Montfort University, Uk have developed interaction techniques that allows gaze based control of several popular online virtual worlds such as World of Warcraft or Second Life. This research will be presented at ETRA 2008, US under the title RAGABITS (RApid GAze-Based Interaction Techniques) and is espcially intented for users with severe motor impairments.

Selection method seems stable. None of the usual jitter can be seen. Nice!




Quote from http://www.ioct.dmu.ac.uk/projects/eyegaze.html

"Online virtual worlds and games (MMORPG's) have much to offer users with severe motor disabilities. It gives this user group the opportunity as entirely able-bodied to others in the virtual world. if they so wish. The extent to which a user has to reveal their disability becomes a privacy issue. Many of the avatars in Second Life appear as stylized versions of the users that control them and that stylization is the choice of the user. This choice is equally appropriate for disabled users. While the appearance of the user's avatar may not reveal the disability of the person that controls it, the behavior and speed or interaction in the world may do.

Many users with severe motor impairments may not be able to operate a keyboard or hand mouse and may also struggle with speech and head movement. Eye gaze is one method of interaction that has been used successfully in enabling access to desktop environments. However, simply emulating a mouse using eye gaze is not sufficient for interaction in online virtual worlds and the users privacy can be exposed unless efficient gaze-based interaction techniques, appropriate to activities in on-line worlds and games can be provided.

This genre of gaming (MMORPG's) is constantly evolving and regardless of the aim of the game they all involve common tasks such as, avatar creation, social interaction (chatting, IM), interaction with in world objects (pick up, open, shoot etc), navigating and walking around the environment. Our research involves analyzing these common tasks so that suitable gaze based interaction techniques to support them can be used in place of a mouse and keyboard. These will have different performance/effort trade-offs, and will include extended mouse/joystick emulation, gaze gestures, toolglasses and gaze-aware in-world objects. These techniques need to be integrated into a coherent and efficient user interface suited to the needs of an individual user with a particular disability. The research aims to model tasks inherent in using these worlds so that predictions can be made about the most appropriate gaze based interaction techniques to use. When these have been identified, they can be assembled into a front end or user interface. One possible outcome could be a software device for automatic configuration of a gaze-control interface for new games, which could use knowledge of a specific user's disability and the eye tracking equipment that they have."

Wednesday, March 26, 2008

Last week of prototyping

Being back from a short easter holiday the final week of developing the prototype is here. The functionality of the interface is OK but there is major tweaking and bug testing to be performed. The deadline for any form of new features is Monday 31st of March. I intend to use all of April for setting up the evaluation experiments and procedures. As always there is so much more that I would like to incorporate, every day brings new ideas.

The final version will include:
  • A configurable dwell button component. This enables drag and drop dwell buttons into Windows projects with individual configuration on dwell-time and icons etc.

  • A novel gaze based menu system that utilized saccade selection (two step dwell) This highly configurable interface component displays itself when activated. It aims at solving the midas touch problem while utilizing the screen real estate in a better way. The two steps in the activation process can be set to specific activation-speeds (dwells) More on this later..

  • A gaze-based memory game using a 36 card layout. By perfoming a dwell the cards "turn" over and shows its symbol (flag). The user the selects another card. If matching then remove them. If different, turn them back over. Got some nice graphical effects.

  • A gaze based picture viewer that zooms into the photos in gaze of the user. More on this later.

  • A gaze based media/music player. This component will scan the computer for artists, albums and songs. These items are then accessible by a gaze driven interface where the user can create play-lists and perform the usual functions (volume+-, pause, stop, next, previous etc.)

  • Perhaps just one little surprise more..

Eight weeks so far. Curiosity, dedication and plain ol´ hard work, nothing else to it.

Tuesday, March 18, 2008

Inspiration: Real-Time Facial and Eye Gaze Tracking System (Park&Kim, 2005)

While my main concern is gaze interaction it´s important to understand the underlying technology which makes it all possible (and the limitation it poses) Much work has gone into solving issues surrounding image processing that makes the "new" remote eye tracking systems we see today possible. One solution to battle the low resolution of cameras is presented in the paper below (using dual cameras) Additionally, the issues of reflections from glasses solved using dual IR-leds. Interesting paper, technical information on Kahlman algorithms etc. used to achieve a 15 frames / second eye tracker on an old Pentium IV 2.0GHz.

Abstract
The goal of gaze detection is to locate the position (on a monitor) where a user is looking. Previous researches use one wide view camera, which can capture the user's entire face. However, the image resolution is too low with such a camera and the fine movements of user's eye cannot be exactly detected. So, we propose the new gaze detection system with dual cameras (a wide and a narrow view camera). In order to locate the user's eye position accurately, the narrow-view camera has the functionalities of auto focusing/panning/tilting based on the detected 3D eye positions from the wide view camera. In addition, we use the IR-LED illuminators for wide and narrow view camera, which can ease the detecting of facial features, pupil and iris position. To overcome the problem of specular reflection on glasses by illuminator, we use dual IR-LED illuminators for wide and narrow view camera and detect the accurate eye position, which is not hidden by the specular reflection. Experimental results show that the gaze detection error between the computed positions and the real ones is about 2.89 cm of RMS error.


The paper is written by prof. Kang Ryoung Park who is heading the Computer Graphics & Vision department at the Sangmyung University, Korea (publications)

Co-author prof. Jaihie Kim is the head of the Computer Vision lab at the Yonsei University, Korea (publications)

Monday, March 17, 2008

Inspiration: Takehiko Ohno

Working out of NTT Cyber Solutions Laboratories in Japan Takehiko interests lies mainly in eye tracking technology and human computer interaction. He has published several papers on eye tracking technology and interaction methods. The QuickGlance selection method aims to solve the well known Midas-touch problem. The interface contains a specific selection area next to each choice/item which must be fixated to activate the function. There are two major advantages with this. First, the user can look around at the menu items without worrying about accidentally activating something. Second, advanced users can go for the activation area directly without even reading the menu text. Just like most people know the order/location of items on the Windows Start-menu. On the downside this means that all options are displayed on the screen all the time.

Takehiko additionally have published several articles on FreeGaze, a remote based system which allows the user to move around his head freely. The FreeGaze eye tracker at NTT has a 0.28 degree of accuracy and is based on a rather wide stereoscopic corneal reflection method using serveral image processing algorithms described in the papers, which are well written and worth reading.

His research highlights the importance of providing feedback to the user as the major method of reducing error rates. Something that I've taken to heart.

Thursday, March 13, 2008

COGAIN 2008: Communication, Environment and Mobility Control by Gaze

This years COGAIN conference will be held in Prague, 2-3 September. Detailed information on paper submission, program, dates, venues etc can be found at http://www.cogain.org/cogain2008

Themes this year are
  • Text entry by means of gaze
  • Gaze and environmental control
  • Gaze and personal mobility control
  • Direct interaction with gaze aware real world objects
  • User experience studies
  • Innovations in eyetracking systems
  • Low cost gaze tracking systems
  • Attentive interfaces and inferring user intent from gaze
  • Gaze-based interaction with virtual worlds
  • Gaze and creativity
  • Gaming using gaze as an input modality
  • Using gaze with other modalities including BCI

Wednesday, March 12, 2008

Eye Gaze Interaction with Expanding Targets (Minotas, Spakov, MacKenzie, 2004)

Continuing on the topic of expanding areas this paper presents an approach where the expansion of the target area is invisible.The authors introduce their algorithm called "Grab-and-hold" which aims at stablizing the gaze data and performs a two part experiment to evaluate it.

Abstract
"Recent evidence on the performance benefits of expanding targets during manual pointing raises a provocative question: Can a similar effect be expected for eye gaze interaction? We present two experiments to examine the benefits of target expansion during an eye-controlled selection task. The second experiment also tested the efficiency of a “grab-and-hold algorithm” to counteract inherent eye jitter. Results confirm the benefits of target expansion both in pointing speed and accuracy. Additionally, the grab-and-hold algorithm affords a dramatic 57% reduction in error rates overall. The reduction is as much as 68% for targets subtending 0.35 degrees of visual angle. However, there is a cost which surfaces as a slight increase in movement time (10%). These findings indicate that target expansion coupled with additional measures to accommodate eye jitter has the potential to make eye gaze a more suitable input modality." (Paper available here)

Their "Grab-and-hold" algorithm that puts some more intelligent processing of the gaze data. "Upon appearance of the target, there is a settle-down period of 200 ms during which the gaze is expected to land in the target area and stay there. Then, the algorithm filters the gaze points until the first sample inside the expanded target area is logged. When this occurs, the target is highlighted and the selection timer triggered. The selection timer counts down a specified dwell time (DT) interval. "

While reading this paper I came to think about an important question concerning filtering of gaze data. The delay that comes from collecting the samples used for the algorithm processing causes a delay in the interaction. For example, if I were to sample 50 gaze positions and then average these to reduce the jitter it would result in a one second delay on a system that captures 50 images per second (50Hz) As seen in other papers as well there is a speed-accuracy trade off to make. What is more important, a lower error rate or a more responsive system?

Monday, March 10, 2008

Application of Fitts Law to Eye Gaze Interaction Interfaces (Miniotas, 2000)

Fitts's law (often cited as Fitts' law) is a model of human movement which predicts the time required to rapidly move to a target area, as a function of the distance to the target and the size of the target. Paul M. Fitts (1912 – 1965) was a psychologist at Ohio State University (later at the University of Michigan). He developed a model of human movement, Fitts's law, based on rapid, aimed movement, which went on to become one of the most highly successful and well studied mathematical models of human motion. Fitts's law is used to model the act of pointing, both in the real world (e.g., with a hand or finger) and on computers (e.g., with a mouse) (Source: Wikipedia, 2008-03-11)


I became interested when I found the paper "Application of Fitts Law to Eye Gaze Interaction Interfaces" by Darius Miniotas (2000) at the Siauliai University, Lithuania. The study does only contain six participants. The task consists of keeping a fixation within 26mm x 26mm box continuously for 250ms. Knowing the noise and jitter in all eye trackers (the one I'm using is state-of-the-art 2008) the task might not be the best one for illustrating Fitts law while using eye trackers. Additionally, presenting a visual indicator of gaze position may appear as distracting due to often present offsets in eye tracking algorithms (somewhat off and moving around).

Abstract
An experiment is described comparing the performance of an eye tracker and a mouse in a simple pointing task. Subjects had to make rapid and accurate horizontal movements to targets that were vertical ribbons located at various distances from the cursor's starting position. The dwell-time protocol was used for the eye tracker to make selections. Movement times were shorter for the mouse than for the eye tracker. Fitts' Law model was shown to predict movement times using both interaction techniques equally well. The model is thus seen to be a potential contributor to design of modern multimodal human computer interfaces. (ACM Paper)

Inspiration: Professor Andrew Duchowksi

Andrew Duchowski is one on the leading authorities on eye tracking and gaze interaction working out of the College of Engineering and Science at Clemson University (South Carolina, U.S.) Andrew is involved in organizing the annual Eye Tracking Research and Applications Symposium (ETRA) conference.

Research interests
  • Visual perception and human-computer interaction.
  • Computer graphics, eye tracking, virtual environments.
  • Computer vision and digital imaging.
  • Wavelet and multi-resolution analysis.
Andrew has published the book Eye Tracking Methodology: Theory and Practice which is one of the very few titles within the field which is especially oriented towards practical research and technical development. Well worth reading if your aiming at developing gaze driven applications (deals with algorithms, experimental setup etc)

Eye Tracking Methodology: Theory and Practice, 2nd ed.
Duchowski, A. T. (2007), Springer-Verlag, London, UK.
ISBN: 978-1-84628-808-7


Resume, talks and publications

Inspiration: Professor Rob Jacobs (Mr Midas Touch)


Rob Jacobs, currenly at Tufts University, was early on interested in gaze based interaction. Jacobs have a long list of both honors, publications and presentations in the HCI field. Being the founder of the "Midas Touch" analogy and many other fundamental aspects of gaze interaction the guy clearly deserves an introduction.


From his homepage
"Robert Jacob is a Professor of Computer Science at Tufts University, where his research interests are new interaction media and techniques and user interface software. He is currently also a visiting professor at the Universite Paris-Sud, and he was a visiting professor at the MIT Media Laboratory, in the Tangible Media Group, and continues collaboration with that group. Before coming to Tufts, he was in the Human-Computer Interaction Lab at the Naval Research Laboratory. He received his Ph.D. from Johns Hopkins University, and he is a member of the editorial board of Human-Computer Interaction and the ACM Transactions on Computer-Human Interaction. He was Papers Co-Chair of the CHI 2001 conference, Co-Chair of UIST 2007, and Vice-President of ACM SIGCHI. He was elected to the ACM CHI Academy in 2007, an honorary group of people who have made extensive contributions to the study of HCI and have led the shaping of the field."

Research topics:
Human-Computer Interaction
New Interaction Techniques and Media
Tangible User Interfaces
Virtual Environments
User Interface Software
Information Visualization

Must-read:

R.J.K. Jacob, “The Use of Eye Movements in Human-Computer Interaction Techniques: What You Look At is What You Get,” ACM Transactions on Information Systems, vol. 9, no. 3, pp. 152-169 (April 1991) [link]

L.E. Sibert and R.J.K. Jacob, “Evaluation of Eye Gaze Interaction,” Proc. ACM CHI 2000 Human Factors in Computing Systems Conference, pp. 281-288, Addison-Wesley/ACM Press (2000). [link]

Inspiration: Dwell-Based Pointing in Applications (Muller-Tomfelde, 2007)

While researching the optimal default value for dwell time execution I stumbled upon this paper by Christian Muller-Tomfelde at the CSIRO ICT Centre, Australia. It does not concern dwell time in the aspect of gaze based interaction but instead focuses on how we handle dwell times while pointing towards objects and conveying this reference to a communication partner. How long can this information be withheld before the interaction becomes unnatural?

Abstract
"This paper describes exploratory studies and a formal experiment that investigate a particular temporal aspect of human pointing actions. Humans can express their intentions and refer to an external entity by pointing at distant objects with their fingers or a tool. The focus of this research is on the dwell time, the time span that people remain nearly motionless during pointing at objects. We address two questions: Is there a common or natural dwell time in human pointing actions? What implications does this have for Human Computer Interaction? Especially in virtual environments, feedback about the referred object is usually provided to the user to confirm actions such as object selection. A literature review and two studies led to a formal experiment in a hand-immersive virtual environment in search for an appropriate feedback delay time for dwell-based pointing actions. The results and implications for applications for Human Computer Interaction are discussed. "

I find the part about the visual feedback experiment interesting.

"We want to test whether a variation of the delay of an explicit visual feedback for a pointing action has an effect of the perception of the interaction process. First, feedback delay time above approximately 430 ms is experienced by users to happen late. Second, for a feedback delay time above approximately 430 ms users experience waiting for feedback to happen and third, feedback delay below 430 ms is considered by users to be natural as in real life conversations. "

Questions asked:
  • 1: Do you have the impression that the system feedback happened in a reasonable time according to your action? Answer: confirmation occurred too fast (1), too late (7).
  • 2: Did you have the feeling to wait for the feedback to happen? Answer: no I didn’t have to wait (1), yes, I waited (7).

  • 3: Did you have the impression that the time delay for the feedback was natural? (i.e., as in a real life communication situation) Answer: time delay is not natural (1), quite natural (7).

"This allows us to recommend a feedback delay time for manual pointing actions of approximately 350 to 600 ms as a starting point for the development of interactive applications. We have shown that this feedback delay is experienced by users as natural and convenient and that the majority of observers of pointing actions gave feedback within a similar time span."

Friday, March 7, 2008

Technology: Consumer-grade EEG devices

Not exactly eye tracking but interesting as a combined modality are the upcoming consumer grade Electroencephalography (EEG) devices sometimes referred to as "brain-mouse". The devices are capable of detecting brain activity by electrodes placed on the scalp.

The company OCZ Technology, mainly known for it's computer components such a memory and power supplies, have announced the "Neural Impulse Actuator (NIA)". While this technology itself is nothing new the novelty lies in the accessibility of the device, priced somewhere around $200-250 when introduced next week.

Check out the quick mini-demo by the guys at Anandtech from the Cebit exhibition in Hannover, 2008
This technical presentation (in German) goes into a bit more detail.


From the press release:
"Recently entering mass production, the final edition of the Neural Impulse Actuator (NIA) will be on display for users to try out the new and electrifying way of playing PC games. The final version of the NIA uses a sleek, metal housing, a USB 2.0 interface, a streamlined headband with carbon interface sensors, and user-friendly software. The NIA is the first commercially available product of its kind, and gamers around the world now have access to this forward-thinking technology that’s had the industry buzzing since its inception."




These devices do have the potential for taking the whole hands-free computing to the next level. They could be a feasible candidate for solving the midas touch problem by proving a device that enables the user to perform activations by subtle facial gestures etc. I have yet to discover how sensitive this device is and what the latency is. Additionally, does it come with an API?

I've tried research grade EEG devices as means for interaction while at the University of California, San Diego and pure thoughts of actions are hard to detect in a stable manner. It is well known in the neuroscience community that thought of actions activates the same regions in the brain as would be activated by actually performing them. We even have mirror neurons that are activated by observing other people performing goal-directed actions (picking up that banana) The neural activation of pure thought alone is subtle and hard to detect compared to performing actual movements, such as lifting ones arm. So, I do not expect it to be an actual Brain Computer Interface (BCI) capable of detecting thoughts (ie. thinking of kicking etc.) but more a detector of subtle motions in my forehead muscles (eye and eye brown movements, facial expressions etc.)

The firm Emotive has their own version of a consumer grade EEG which is named Epoc NeuroHeadset, it has been around for a little longer and seem to have a more developed software, but still mainly demonstration applications.


The Emotive NeuroHeadset

Inspiration: All Eyes on the Monitor (Mollenbach et al, 2008)

Going further with the Zooming User Interface (ZUI) is the prototype descibed in the "All Eyes on the Monitor: Gaze Based Interaction in Zoomable, Multi-Scaled Information-Space" (E. Mollenbach, T. Stefansson, J-P Hansen) developed at the Loughborough University in the U.K and the ITU INC, Denmark. It employes the gaze based pan/zoom interaction style which is suitable for gaze interaction to resolve the inaccuracy (target sizes increase when zooming in to them) Additionally, the results indicate that for certain tasks gaze based interaction is faster than traditional mouse operation.



ABSTRACT
The experiment described in this paper, shows a test environment constructed with two information spaces; one large with 2000 nodes ordered in semi-structured groups in which participants performed search and browse tasks; the other was smaller and designed for precision zooming, where subjects performed target selection simulation tasks. For both tasks, modes of gaze- and mouse-controlled navigation were compared. The results of the browse and search tasks showed that the performances of the most efficient mouse and gaze implementations were indistinguishable. However, in the target selection simulation tasks the most efficient gaze control proved to be about 16% faster than the most efficient mouse-control. The results indicate that gaze-controlled pan/zoom navigation is a viable alternative to mouse control in inspection and target exploration of large, multi-scale environments. However, supplementing mouse control with gaze navigation also holds interesting potential for interface and interaction design. Download paper (pdf)

The paper was presented at the annual International Conference for Intelligent Interfaces (IUI) that was held in Maspalomas, Gran Canaria between 13-16th January 2008.

Monday, March 3, 2008

Zooming and Expanding Interfaces / Custom componenets

The inspiration I got from the reviewed papers on using a zooming interaction style to developing a set of zoom based interface components. The interaction style is suitable for gaze to overcome the inaccuracy and jitter of eye movements. My intention is that the interface components should be completely standalone, customizable and straightforward to use. Ideally included in new projects by importing one file and writing one line of code.

The first component is a dwell-based menu button that on fixation will a) provide a dwelltime indicator by animating a small glow effect surrounding the button image and b) after 200ms expand an ellipse that houses the menu options. This produces a two step dwell activation while making use of the display area in a much more dynamic way. The animation is put in place to keep the users fixation remained at the button for the duration of the dwell time. The items in the menu are displayed when the ellipse has reached its full size.

This gives the user a feedback in his parafoveal region and at the same time the glow of the button icon has stopped indicating a full dwelltime execution. (bit hard to describe in words, perhaps easier to understand from the images below) The parafoveal region of our visual field is located just outside the foveal region (where the full resolution vision takes place). The foveal area is about the size of a thumbnail on an armslengths distance, items in the parafoveal region still can be seen but the resolution/sharpness is reduced. We do see them but have to make a short saccade for them to be in full resolution. In other words the menu items pop out at a distance that attracts a short saccade which is easily discriminated by the eye tracker. (Just4fun test your visual field)

Before the button has received focus


Upon fixation the button image displays an animated glow effect indicating the dwell process. The image above illustrates how the menu items pops out on the ellipse surface at the end of the dwell. Note that the ellipse grows in size during a 300ms period, exact timing is configurable by passing a parameter in the XAML design page.

The second prototype I have been working on is also inspired by the usage of expanding surfaces. The purpose is a gaze driven photo gallery where thumbnail sized image previews becomes enlarged upon glancing at them. The enlarged view displays an icon which can be fixated to make the photo appear in full size.

Displaying all the images in the users "My pictures" folder.


Second step, glancing at the photos. Dynamically resized. Optionally further enlarged.

Upon glancing at the thumbnails they become enlarged which activates the icon at the bottom of each photo. This enables the user to make a second fixation on it to bring the photo into a large view. This view has to two icons to navigate back and forth (next photo). By fixating outside the photo the view goes back to the overview.