System and method for eye gaze tracking using corneal image mapping

US 20030123027A1
Filed: 12/28/2001
Published: 07/03/2003
Est. Priority Date: 12/28/2001
Status: Active Grant

First Claim

Patent Images

1. A method for eye gaze tracking, comprising the steps of:

creating a set of reference points in a reference coordinate system;

acquiring at least one image of at least one of a user'"'"'s corneas, said image having image aspects in an image coordinate system and including reflections of said reference points;

defining a mathematical relationship between said reference coordinate system and said image coordinate system;

mapping said image aspects from said image coordinate system to said reference coordinate system using said mathematical relationship; and

computing a point of regard from said mapped image aspects.

View all claims

4 Assignments

Timeline View

Assignment View

0 Petitions

Accused Products

Abstract

A system and method for eye gaze tracking without calibrated cameras, direct measurements of specific users'"'"' eye geometries, or requiring the user to visually track a cursor traversing a known trajectory. The preferred embodiment includes two uncalibrated cameras imaging the user'"'"'s eye and having on-axis lighting. The cameras capture images of a test pattern in real space as reflected from the user'"'"'s cornea, which is essentially a convex spherical mirror. The invention then extracts parameters required to define a mathematical mapping between real space and image space, including spherical and perspective transformations. The invention processes subsequent images of objects reflected from the user'"'"'s eye through the inverse of the mathematical mapping to determine a gaze vector and a point of regard. Alternately, a single calibrated camera may be employed with means for estimating the eye-to-camera distance. A head-mounted embodiment that may include a laser pointer is also described.

90 Citations

15 Claims

1. A method for eye gaze tracking, comprising the steps of:
- creating a set of reference points in a reference coordinate system;
  
  acquiring at least one image of at least one of a user'"'"'s corneas, said image having image aspects in an image coordinate system and including reflections of said reference points;
  
  defining a mathematical relationship between said reference coordinate system and said image coordinate system;
  
  mapping said image aspects from said image coordinate system to said reference coordinate system using said mathematical relationship; and
  
  computing a point of regard from said mapped image aspects.
- View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
- - 2. The method of claim 1 wherein said reference points include at least one of:
    - a printed pattern around a screen, an unobtrusively interlaced pattern in said screen, a set of controlled light sources around said screen, a set of controlled light sources on said screen, content displayed in said screen, a set of controlled light sources behind said screen.
  - 3. The method of claim 2 wherein said screen includes at least one of:
    - a computer monitor, a whiteboard, a desktop, a windshield, an advertisement, a television screen.
  - 4. The method of claim 1 wherein a laser pointer creates at least one new reference point.
  - 5. The method of claim 4 wherein said laser pointer creates said new reference point that reflects from said cornea at a pupil image center in said image coordinate system.
  - 6. The method of claim 1 wherein said acquiring step is performed by at least one camera focusing upon at least one of said user'"'"'s corneas, each said camera having a focal center, an image plane defining said image coordinate system, and an on-axis light source.
  - 7. The method of claim 6 wherein said image aspects are identified by subtracting a number of said images acquired during different phases of display screen refresh cycles.
  - 8. The method of claim 6 comprising the further steps of:
    - determining for each of said cameras an angle between said focal center, a user'"'"'s pupil center, and a point on a predetermined target surface where a virtual light source would create a new image aspect at a pupil image center in said image coordinate system; and
      
      defining a gaze vector as the bisector of said angle.
  - 9. The method of claim 8 comprising the further step of correcting said gaze vector for a foveal axis offset angle.
  - 10. The method of claim 6 wherein at least one of said cameras is head-mounted.
  - 11. The method of claim 1 wherein said mathematical relationship includes at least one of:
    - spherical transformations, perspective transformations, polynomial interpolation.
  - 12. The method of claim 1 wherein said computing step includes the further steps of:
    - mapping a target coordinate system to said reference coordinate system; and
      
      bisecting a line segment spanning an on-axis glint and a pupil image center in said target coordinate system.
  - 13. The method of claim 12 wherein said target coordinate system is said reference coordinate system.

14. A system for eye gaze tracking comprising:
- means for creating a set of reference points in a reference coordinate system;
  
  means for acquiring at least one image of at least one of a user'"'"'s corneas, said image having image aspects in an image coordinate system and including reflections of said reference points;
  
  means for defining a mathematical relationship between said reference coordinate system and said image coordinate system;
  
  means for mapping said image aspects from said image coordinate system to said reference coordinate system using said mathematical relationship; and
  
  means for computing a point of regard from said mapped image aspects.

15. A computer program product including a program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for eye gaze tracking, said program comprising:
- a first code means for creating a set of reference points in a reference coordinate system;
  
  a second code means for acquiring at least one image of at least one of a user'"'"'s corneas, said image having image aspects in an image coordinate system and including reflections of said reference points;
  
  a third code means for defining a mathematical relationship between said reference coordinate system and said image coordinate system;
  
  a fourth code means for mapping said image aspects from said image coordinate system to said reference coordinate system using said mathematical relationship; and
  
  a fifth code means for computing a point of regard from said mapped image aspects.

Specification

Resources

Litigation Campaign Assessment

Current Assignee
Tobii Technology AB
Original Assignee
International Business Machines Corporation
Inventors
Amir, Arnon, Flickner, Myron Dale, Morimoto, Carlos Hitoshi, Koons, David Bruce

Granted Patent

US 6,659,611 B2
Time in Patent Office

Days
Field of Search
US Class Current

351/209
CPC Class Codes

A61B 3/113 for determining or recordin...

G06V 40/19 Sensors therefor

System and method for eye gaze tracking using corneal image mapping

First Claim

4 Assignments

0 Petitions

Accused Products

Abstract

90 Citations

15 Claims

Specification

Solutions

Use Cases

Quick Links

System and method for eye gaze tracking using corneal image mapping

First Claim

4 Assignments

Subscription Required

Subscription Required

0 Petitions

Subscription Required

Accused Products

Subscription Required

Abstract

90 Citations

15 Claims

Specification

Subscription Required

Solutions

Use Cases

Quick Links