It is not very comfortable to type on small virtual keyboards available on smartphones. Although, the smart assistants are getting better at recognizing voice, but it is not possible to use them everywhere. They do not work well in noisy environments. Further, they do not protect your privacy.

A recent patent publication from Microsoft reveals a revolutionary eye-typing technology. Your eye-gaze is monitored to find out the gaze location on a virtual keyboard displayed on your smartphone. Then, advanced machine learning algorithms are used to predict responses, such as words that you wish to type. The words will be predicted based on statistical models ranking common words associated with certain letters and contexts. For example, if you are updating your sports blog, you are statistically more likely to use sports related terminology like “homerun”, “touchdown”, and “line-backer”.  The beginning and end of words will be predicted by your gaze location shifting away from the virtual keyboard, or towards specific keys like space, comma, or period. The predictions will then be shown for your approval, and implemented accordingly.

In this patent, Microsoft addresses deficiencies in the current eye-tracking and gaze-input systems by providing an intelligent, dwell-free eye gaze input system. Dwell time is a time in which your gaze remains fixed on the same letter for a set time period to register the letter in the eye typing system. Longer dwell time not only leads to unacceptable typing speeds, but also increases fatigue, as well as frustration.

Microsoft technology does not require you to fixate for extended amounts of time, but rather can quickly glance at letters that form a desired word. In some example aspects, a visual trace may be drawn on the screen that follows a path of eye movements to provide feedback to the user, as shown in the figure below.

In addition to typing, the technology also allows you to control your computer, tablet, or smartphone through your eyes by interacting with UI elements. Interacting with UI elements is comparatively easier than typing due to lesser amount of prediction required.

US20190034038