Eye, face and hand tracking for VR/AR and smart glasses

Compact, low power components enabling advanced gaze and facial tracking 

Powering next generation eye, face and hand tracking 

Eye and face tracking enable natural interaction in virtual and augmented reality systems as well as in smart glasses. Eye gaze becomes a user interface and powers photorealistic avatars, while foveated rendering reduces compute and system power. Face tracking transfers facial expressions for social and professional presence. Critical requirements include high accuracy, fast update rates, minimal power consumption, and tiny form factor. 

How eye tracking works (camera-based) 

Camera-based eye tracking combines infrared (IR) illumination (LED or VCSEL) with a CMOS globalshutter image sensor. Illumination operates in the invisible IR spectrum; reflected light from the eye is captured and processed via pupil/glint detection. Globalshutter sensors avoid rollingshutter artifacts and support ultrashort exposure times for high precision at low motion blur. 

Component examples (camera-based) 

  • IR LEDs for illumination, e.g., SFH4060 Firefly®. 
     
  • AS1181 multichannel LED/VCSEL driver enabling ultrashort pulses for globalshutter sensors and singlefaulttolerant safety monitoring for eye safety under all operating conditions. 
×

How eye tracking works (photosensor-based)

For ultra-low power, especially in all-day smart glasses, photodiode-based approaches offer advantages. IR LEDs emit short pulses, and photodiodes detect reflections from cornea/sclera. Optical reflection sensors integrate LED, photodiode, LED driver, and analog front end with digital control and readout. 

  • TMD2636 proximity/optical reflection sensing module. 

Application block diagram 

The following application block diagram illustrates potential hardware architectures for eye and face tracking in XR devices. Camerabased systems combine infrared illumination (LEDs or VCSELs) with a CMOS globalshutter image sensor to capture detailed eye and facial features. Photosensor-based systems, in contrast, use highly integrated optical reflection (proximity) sensors to enable ultra-low-power gaze detection in compact wearable designs. 

FAQ

What is eye tracking in VR/AR?

Capturing eye gaze vectors using IR illumination and sensors to enable user interfaces, reduce compute via foveated rendering, and support emerging biosignal applications. 

Why globalshutter sensors for eye tracking?

They eliminate rollingshutter artifacts, capture fast eye movements precisely, and support ultrashort exposures synchronized with IR pulse illumination. 

Camera vs. photosensor-based—what’s the difference?

Camerabased delivers maximum accuracy and full image context at higher power; photosensor-based minimizes power and size with adequate tracking performance for many glasses designs. 

How is eye safety ensured?

Multichannel drivers with singlefaulttolerant monitoring limit IR output within regulatory thresholds, maintaining compliance even during faults. 

Which components are suitable?

MIRA globalshutter sensors, SFH4060 Firefly® IR LEDs, AS1181 LED/VCSEL driver, and TMD2636 optical reflection sensor. 

Featured Products

ams Mira016 CMOS Image Sensor

Mira016 is a compact 0.16MP Near IR enhanced global shutter image sensor designed for 2D and 3D consumer and industrial machine vision applications. The sensor has a small 2.79 µm pixel size with high sensitivity made possible by a state of the art BSI technology. The sensor has a MIPI CSI-2 interface to allow easy interfacing with a plethora of processors and FPGAs. With its small 1.8mm x 1.8mm size, configurability and high sensitivity both in visual as well as NIR wavelengths, the Mira016 is well suited for many 2D and 3D applications, including eye tracking in ARVR. Low active power and high sensitivity in NIR enables increased measurement range as well as overall system power consumption optimization which is key for battery powered consumer and industrial applications.
Details and Datasheet

ams Mira050 CMOS Image Sensor

Mira050 is a compact 0.5 MP Near IR enhanced global shutter image sensor designed for 2D and 3D consumer and industrial machine vision applications. The sensor has a small 2.79 µm pixel size with high sensitivity made possible by a state of the art BSI technology. The sensor has a MIPI CSI-2 interface to allow easy interfacing with a plethora of processors and FPGAs. Due to its small size, configurability and high sensitivity both in visual as well as NIR, the Mira050 is well suited for 2D and 3D applications, which include Active Stereo Vision, Structured Light Vision and AR/VR. High sensitivity in NIR enables increased measurement range and allows overall system power consumption optimization which is key for battery powered consumer and industrial applications.
Details and Datasheet

ams Mira220

2.2 MP NIR enhanced global shutter image sensor designed for 2D and 3D consumer and industrial machine vision applications. Due to its small size, configurability and high sensitivity both in visual as well as NIR, the Mira220 is well suited for 2D and 3D applications, which include Active Stereo Vision, Structured Light Vision for Robotics and AR/VR.
Details and Datasheet

OSRAM FIREFLY™, SFH 4060

Infrared Emitter (850 nm)
Details and Datasheet

ams AS1181 8-channel LED and VCSEL driver IC

AS1181 is an 8-channel highly integrated self-contained LED/VCSEL driver for near to eye applications such as Augmented Reality (AR) and Virtual Reality (VR) glasses. This eye-safety LED driver integrates extended safety monitoring functions to ensure eye-safe operation for smart glasses and other AR/VR applications. These safety monitors include LED short detection, LED open detection, LED overcurrent protection, LED on time monitor as well as temperature shutdown and a built-in-self test (BIST). The device is configurable via a serial wire interface (I²C or I3C) with interrupt and provides two Strobe/PWM inputs to synchronize illumination with up to two external cameras for binocular eye tracking systems. The LED current is individually programmable per channel, and it can drive up to 2 IR LEDs per current sink with a minimum ON time of 10μs and a maximum current of 66mA per channel. General purpose LED driving applications can be supported via a direct PWM input applied at the Strobe pin and it can drive RGB or white LEDs considering a maximum forward voltage of 5V. The device comes in a tiny wafer-level-chip-scale package (WLCSP) with 0.4mm pitch and dimensions of 2.9mm x 1 .75mm x 0.5mm (L x W x H).
Details and Datasheet

ams TMD2636 IR Proximity Detection Module

The TMD2636 is an advanced IR proximity sensor module suited for short range proximity measurement detection and offered in the industry's smallest size (1.0mm x 2.0mm) and thin (0.35mm) optical land grid array module that incorporates a 940nm IR VCSEL and is factory calibrated for IR proximity response. The proximity detection feature provides nearby object detection by photodiode detection of reflected IR energy sourced by the integrated VCSEL emitter. Sensitivity, power consumption, and noise can be optimized with adjustable timing and power. Detect/release events can be interrupt driven, and occur when proximity result crosses upper and/or lower threshold settings. The proximity engine features advanced crosstalk noise cancellation through a wide range offset adjustments to compensate for unwanted IR energy reflection at the sensor. Proximity results are further improved by automatic ambient light subtraction.
Details and Datasheet
\n