Eye Tracking XR Dataset

Dataset Description

Eye Tracking XR dataset is a set of eye images (left and right eyes) captured using a binocular eye tracking module manufactured by Pupil Labs inside a HTC Vive Virtual Reality headset (as seen in figure below). The Pupil Labs eye tracking module consists of two cameras running at 90 frames/sec with five light sources for eye illumination. The eye images (640x480) are labelled with information about reflections of light sources on the eye (known as Corneal Reflections). This information includes locations of corneal reflections in the image and its correspondence with an originating light source. Also, for each image, the information needed to fit an ellipse around the pupil is provided.

HTC Vive with Pupil Labs Module

For each of the eye image in the dataset, the following information is available:

• Corneal Reflection Locations and its correspondence with light source (list of x,y coordinates)
• Pupil Center Locations (x,y coordinate)
• Pupil Boundary Locations (list of x,y coordinates)
• Angle and Axis of the fitted pupil ellipse
• Validity of frame in the context of eye tracking (1 or 0)

The dataset is collected from 15 people. To increase the size of the dataset, we included images of another 10 people available by NVIDIA where a similar hardware compared to our system is used. The eye images and the labels (provided in a json file) can be downloaded by clicking here.

Please note that, due to license constraints, eye images from NVIDIA dataset can only be downloaded from their website. We have provided information for the labels of the NVIDIA dataset in our label file mentioned above.

Encoding of filenames in the Dataset

The naming convention for the filenames associated with labelled images is 'person_imageNumber_ValidityOfImage.'

'person' tag represents Pn's eye images where n ranges from 1-15 representing 15 different people recorded by us. For images from the NVIDIA dataset, we use 'Nvidia-Pn' where n ranges from 1-10 representing ten different people.

'imageNumber' represents the sequence number in the eye video recordings. For the NVIDIA dataset, the imageNumber will help identify the subset of images used by us.

'ValidityOfImage' represents an image as positive ('pos') or negative ('neg'). An image is said to be 'neg' if more than half of the pupil is occluded. If less than two corneal reflections are present in the image, then we label it as 'neg'. The labels (corneal reflections and pupil) for the 'neg' image is set to -1.

Corneal Reflection Correspondance Information

Our dataset provides information regarding corneal reflections correspondence with its light source for 'pos' eye images. Corneal reflections in an image are labelled according to light sources in the Pupil Labs module and saved in the form of a list. The first element in the list corresponds to the corneal reflection due to the topmost light source (as seen in the figure above). The other four elements in the list correspond to the four light source's corneal reflections in an anticlockwise direction, starting from the top light source. In the case of missing corneal reflection, we denote its corresponding value by -1.

For any questions, related to this dataset, please email to this e-mail address.

Examples of Labelled Images

All corneal reflections are present

Only four corneal reflections are present

Only three corneal reflections are present

Only two corneal reflections are present