This article originally appeared in Issue 9 of IT Pro 20/20, available here. To sign up to receive each new issue in your inbox, click here.
Touchscreens have become ubiquitous in everyday life. Not only are they the gateway to our much loved mobile phones but, until recently, we have been happy to regularly interact with them at everything from ATMs and ticket machines to interactive kiosks and supermarket checkouts.
And then COVID-19 struck.
The pandemic has undoubtedly changed the ease with which we touch shared screens – especially once scientists found that SARS-CoV-2, the virus that causes COVID-19, can last on surfaces for up to three days. In fact, an UltraLeap survey of the UK and US public in May this year found that 80% of consumers think touchscreens are unhygienic and 50% would be unlikely to use them in the future.
“Consumers around the world are worried about the safety and cleanliness of public touchscreens, such as self-service kiosks, and would be very open to trying new ways of interacting if they are available,” explains Catherine Morgan, director of Ocean Labs, a technology division of digital out of home firm Ocean Outdoor. “The rapid adoption of contactless payments is one proof point.”
With this in mind, it should come as no surprise that interest is growing in touchless interfaces.
“COVID-19 has placed the topic of health and wellbeing under a global spotlight,” says Lee Skrypchuk, human machine interface technical specialist at Jaguar Land Rover. “Prior to this, technologies that could help to mitigate the spread of bacteria and viruses were a developing trend, but now it seems to be in sharp focus to the point where the perception of shared touch devices may change forever. Customers will consider these types of risks more intently in the future and are likely to expect devices, which are of a shared use nature, to be designed to mitigate any risks that might be created by touch surfaces which are potentially shared by many different people.”
Changing the game
Until recently, interest in touchless technology had been low as what’s on offer has generally been expensive, power hungry, bulky, and not very intuitive.
“Most touchless solutions capitalise on recent advances in sensing and data processing technologies,” explains Bashar Ahmad, a senior research associate in the department of engineering at University of Cambridge. “One example is gesture-recognition-based systems, where the user undertakes a pre-learnt mid-air symbolic gesture such as swipe or wave and the interface produces a particular response linked to this gesture. These types of systems have generally been quite clunky.”
New developments in touchless solutions promise something better, however. Ocean Outdoor, for example, has integrated touchless technologies across its portfolio in partnership with US haptics firm UltraLeap.
“UltraLeap has developed incredibly accurate hand tracking, as well as the world’s only mid-air haptics, which creates the sense of touch in mid-air using ultrasound,” explains Morgan. “The hand tracking technology uses an inexpensive infrared stereo camera, which means we can create a very cheap and effective touchless solution that can replace touchscreens.”
Meanwhile, Skrypchuk and Ahmad, along with their colleagues at the University of Cambridge and Jaguar Land Rover, have developed a no-touch touchscreen called ‘predictive touch’, which uses a combination of artificial intelligence (AI) and widely available sensor technology (such as hand/finger tracking) to predict a user’s intended target on touchscreens and other interactive displays or control panels, selecting the correct item before the user’s hand reaches the display.
“Our innovation is completely unique because it has the ability to predict the interface item the user intends to select from their freehand pointing movement, eye-gaze behaviour and other contextual information such as environment, user profile, interface design and historical data such as frequency of use,” says Ahmad. “It still uses the intuitive point-select hand/finger motion everyone is familiar with, except that the user needs not touch the display.”
Skrypchuk explains further: “Many similar technologies track the hand like a mouse cursor, mapping the movements to tile locations on a screen. With predictive touch, we are inferring intent through the path, speed and other motion characteristics of the pointing finger. The idea being that as your finger approaches the screen, we can make an increasingly accurate prediction the closer the finger gets to the screen, based on its trajectory. As we track finger or hand position, it gets to the point where the prediction is high enough that we can be certain of the selection, dependent upon the number of items of the screen and the distance of the finger from the screen surface. Therefore, you can continue to use the touchscreen as you would do ordinarily, we just make the selection sooner.”
COVID-proof contact
This is just the start of what the technology can achieve. As lockdown restrictions around the world continue to ease, Skrypchuk and Ahmad believe the technology will be incredibly useful in a post-COVID world. “The system doesn't have to be screen based either, it can work on any given surface or target and we could convert any screen (including non-touch enabled ones) into a touchscreen,” Skrypchuk says. “Imagine how many non-touch computer monitors there are in the world – we could potentially convert these to touchless operation just by adding a single camera.
Looking ahead, Ahmad expects touchless interaction technology to gradually replace touch-screens. “This will be accelerated by the proliferation of emerging new display technologies for 3D interfaces or holograms,” he explains.
Morgan agrees: “We expect to see more powerful integration of new technology into next-generation systems, making them future-proof against COVID-19 and any future pandemics,” she says. “The type of touchless system used will vary, depending on the scenario – but what’s clear to me is that the future of touchscreens is most definitely touchless.”
