Large Virtual Sensor
For mass market computer vision, you generally need a lens and a sensor with the highest resolution at the lowest cost. While cost usually increases with resolution, for some special applications, it’s possible to use tricks to obtain higher resolutions while lowering the cost.
In the case of gaze-tracking for example, measuring precisely which point the user is looking at from a device at arm’s length requires quite a high resolution. Very high resolution sensors are accordingly expensive, but here we only need to capture a small portion of the user’s face: the eye. If we limit the captured image to that area only, the total resolution can be kept reasonably low.
One solution is to dynamically shift the sensor (or the lens) laterally to track the position of the user’s eye. The problem is that moving parts are usually costly and fragile.
There is also a way of solving this problem entirely optically, without relying on any moving parts.
We want to obtain a high resolution for a large field of view, but not all at once, only small portions at a time. The idea is then to divide the field of view into an array of areas and activate only one at a time. This can be achieved using an array of prismatic lenses, each dedicated to a single portion of the field of view, combined with an array of LCD light barriers that only let light through to a single lens at any given time.
The array of prismatic lenses can be produced with a single mold, and the LCD barrier can be produced at a very low cost.
General optical setup:
The light barrier is dynamically activated in a way that only ever enables one light path at a time:
Of course, several implementation details need to be handled, like overlap between regions to capture eyes located at zone boundaries, or correcting color fringing produced by the prism by using Fresnel lens shapes.
The information in our tech blogs is provided as a way to share knowledge for educational and research purposes. However, aspects of the described technology may be subject to Nintendo patents and/or pending patent applications