Thursday, September 18, 2008

The Inspection of Very Large Images by Eye-gaze Control

Nicholas Adams, Mark Witkowski and Robert Spence from the Department of Electrical and Electronic Engineering at the Imperial College London got the HCI 08 Award for International Excellence for work related to gaze interaction.

"The researchers presented novel methods for navigating and inspecting extremely large images solely or primarily using eye gaze control. The need to inspect large images occurs in, for example, mapping, medicine, astronomy and surveillance, and this project considered the inspection of very large aerial images, held in Google Earth. Comparative search and navigation tasks suggest that, while gaze methods are effective for image navigation, they lag behind more conventional methods, so interaction designers might consider combining these techniques for greatest effect." (BCS Interaction)


The increasing availability and accuracy of eye gaze detection equipment has encouraged its use for both investigation and control. In this paper we present novel methods for navigating and inspecting extremely large images solely or primarily using eye gaze control. We investigate the relative advantages and comparative properties of four related methods: Stare-to-Zoom (STZ), in which control of the image position and resolution level is determined solely by the user's gaze position on the screen; Head-to-Zoom (HTZ) and Dual-to-Zoom (DTZ), in which gaze control is augmented by head or mouse actions; and Mouse-to-Zoom (MTZ), using conventional mouse input as an experimental control.

The need to inspect large images occurs in many disciplines, such as mapping, medicine, astronomy and surveillance. Here we consider the inspection of very large aerial images, of which Google Earth is both an example and the one employed in our study. We perform comparative search and navigation tasks with each of the methods described, and record user opinions using the Swedish User-Viewer Presence Questionnaire. We conclude that, while gaze methods are effective for image navigation, they, as yet, lag behind more conventional methods and interaction designers may well consider combining these techniques for greatest effect.

This paper is the short version of Nicolas Adams Masters thesis which I stumbled upon before creating this blog. A early version appeared as a short paper for COGAIN06.

Monday, September 15, 2008

Apple develops gaze assisted interaction?

Apple recently registered a patent for merging several modalities including gaze vectors for novel interaction methods. The direction of gaze is to be used in combination with finger gestures (or other input devices) to modify the object that the user is currently looking at. Will be interesting to see what types of devices they are aiming for. May not be high precision eye tracking since stability and high accuracy is hard to obtain for a 100% population in all environments.

From the patent document:
"There are many possible applications that would benefit from the temporal fusion of gaze vectors with multi-touch movement data. For the purpose of example, one simple application will be discussed here: Consider a typical computer screen, which has several windows displayed. Assume that the user wishes to bring forward the window in the lower left corner, which is currently underneath two other windows. Without gaze vector fusion there are two means to do this, and both involve movement of the hand to another position. The first means is to move the mouse pointer over the window of interest and click the mouse button. The second means is to use a hot-key combination to cycle through the screen windows until the one of interest is brought forward. Voice input could also be used but it would be less efficient than the other means. With gaze vector fusion, the task is greatly simplified. For example, the user directs his gaze to the window of interest and then taps a specific chord on the multi-touch surface. The operation requires no translation of the hands and is very fast to perform."

"For another example, assume the user wishes to resize and reposition an iTunes window positioned in the upper left of a display screen. This can be accomplished using a multi-touch system by moving the mouse pointer into the iTunes window and executing a resize and reposition gesture. While this means is already an improvement over using just a mouse its efficiency can be further improved by the temporal fusion of gaze vector data. "

TeleGaze (Hemin, 2008)

"This research investigates the use of eye-gaze tracking in controlling the navigation of mobile robots remotely through a purpose built interface that is called TeleGaze. Controlling mobile robots from a remote location requires the user to continuously monitor the status of the robot through some sort of feedback system. Assuming that a vision-based feedback system is used such as video cameras mounted onboard the robot; this requires the eyes of the user to be engaged in the monitoring process throughout the whole duration of the operation. Meanwhile, the hands of the user need to be engaged, either partially or fully, in the driving task using any input devices. Therefore, the aim of this research is to build a vision based interface that enables the user to monitor as well as control the navigation of the robot using only his/her eyes as inputs to the system since the eyes are engaged in performing some tasks anyway. This will free the hands of the user for other tasks while controlling the navigation is done through the TeleGaze interface. "

TeleGaze experimental platform consists of a mobile robot, an eye gaze tracking equipment and a teleoperation station that the user interacts with. The TeleGaze interface runs on the teleoperation station PC and interprets inputs from the eyes into controlling commands. Meanwhile, presenting the user with the images that come back from the vision system mounted on the robotic platform.

More information at Hemin Sh. Omers website.

Associated publications:
  • Hemin Omer Latif, Nasser Sherkat and Ahmad Lotfi, "TeleGaze: Teleoperation through Eye Gaze", 7th IEEE International Conference on Cybernetic Intelligent Systems 2008, London, UK. Conference website:
  • Hemin Omaer Latif, Nasser Sherkat and Ahmad Lotfi, "Remote Control of Mobile Robots through Human Eye Gaze: The Design and Evaluation of an Interface", SPIE Europe Security and Defence 2008, Cardiff, UK. Conference website:

COGAIN 2008 Proceedings now online


Overcoming Technical Challenges in Mobile and Other Systems
  • Off-the-Shelf Mobile Gaze Interaction
    J. San Agustin and J. P. Hansen, IT University of Copenhagen, Denmark
  • Fast and Easy Calibration for a Head-Mounted Eye Tracker
    C. Cudel, S Bernet, and M Basset, University of Haute Alsace, France
  • Magic Environment
    L. Figueiredo, T. Nunes, F. Caetano, and A. Gomes, ESTG/IPG, Portugal
  • AI Support for a Gaze-Controlled Wheelchair
    P. Novák, T. Krajník, L. Přeučil, M. Fejtová, and O. Štěpánková. Czech Technical University, Czech Republic)
  • A Comparison of Pupil Centre Estimation Algorithms
    D. Droege, C Schmidt, and D. Paulus University of Koblenz-Landau, Germany

Broadening Gaze-Based Interaction Techniques
  • User Performance of Gaze-Based Interaction with On-line Virtual Communities
    H. Istance, De Montfort University, UK, A. Hyrskykari, University of Tampere, Finland, S. Vickers, De Montfort University, UK and N. Ali, University of Tampere, Finland

  • Multimodal Gaze Interaction in 3D Virtual Environments
    E. Castellina and F. Corno, Politecnico di Torino, Italy
  • How Can Tiny Buttons Be Hit Using Gaze Only?
    H. Skovsgaard, J. P. Hansen, IT University of Copenhagen, Denmark. J. Mateo, Wright State University, Ohio, US
  • Gesturing with Gaze
    H. Heikkilä, University of Tampere, Finland
  • NeoVisus: Gaze Driven Interface Components
    M. Tall, Sweden

Focusing on the User: Evaluating Needs and Solutions
  • Evaluations of Interactive Guideboard with Gaze-Communicative Stuffed-Toy Robot
    T. Yonezawa, H. Yamazoe, A. Utsumi, and S. Abe, ATR Intelligent Robotics and Communications Laboratories, Japan
  • Gaze-Contingent Passwords at the ATM
    P. Dunphy, A. Fitch, and P. Oliver, Newcastle University, UK
  • Scrollable Keyboards for Eye Typing
    O Špakov and P. Majaranta, University of Tampere, Finland
  • The Use of Eye-Gaze Data in the Evaluation of Assistive Technology Software for Older People.
    S. Judge, Barnsley District Hospital Foundation, UK and S. Blackburn, Sheffield University, UK
  • A Case Study Describing Development of an Eye Gaze Setup for a Patient with 'Locked-in Syndrome' to Facilitate Communication, Environmental Control and Computer Access.
    Z. Robertson and M. Friday, Barnsley General Hospital, UK

Friday, September 12, 2008

COGAIN 2008 Video

Some highlights from the visit to COGAIN 2008 last week in Prague which was a great event. It demonstrates the mobile solution integrating a head mounted display and an eye tracker by Javier San Agustín. A sneak peak of the NeoVisus iTube interface running on the SMI IViewX RED. A demonstration of the Neural Impulse Actuator from OCZ Technolgies by Henrik Skovsgaard. Demo of the gaze controlled wheelchair developed by Falck Igel and Alea Technologies. Thanks to John Paulin Hansen for creating the video.