At the end of last year EPDT magazine interviewed our CEO Nigel Clarke to discuss the RealSim tech, behind the haptic feedback glove we’re developing – which brings human-like touch to VR.
The following article originally appeared in the online edition of Electronic Product Design & Test.
What is haptic feedback and what is it good for?
Far from simply bringing a revolution to video games, virtual reality (VR) continues to prove a vital part of simulated learning, and offers training and rehabilitation applications – meaning the technology is an ideal fit for the military field.
RealSim mimics our very sense of touch, and can determine whether a virtual environment feels hot or cold, smooth or rough, wet or dry.
It does so through a thimble device that houses many actuators, which deliver variable frequencies, patterns and intensities to reflect the stimuli in question. The glove’s design is also now evolving to incorporate the power and VR as never before.
Haptic or tactile feedback in military training.
RealSim uses a number of actuators to apply pressure, vibration and different temperatures to specific receptors on the hand, particularly the fingertips and palm. By being able to program these actuators to deliver variable frequencies, patterns and intensities, RealSim is able to expand the horizons of existing haptic feedback devices – to mimic the feel of virtually any object, surface, substance or environment.
The device therefore mimics our very sense of touch, and can determine whether a virtual environment feels hot or cold, smooth or rough, wet or dry. It does so through a thimble device that houses many actuators, which deliver variable frequencies, patterns and intensities to reflect the stimuli in question (see diagram). The glove’s design is also now evolving to incorporate the power and pneumatics it needs for further enhancements, along with the development of robust infrastructure around the hand.
The RealSim haptic feedback glove is shaping the future of VR
RealSim was created and patented by MIAT, and the technology has already been proven to help rehabilitate stroke survivors by replicating their sense of touch. This was, in fact, how the device was first conceived: MIAT were in a collaborative project with the University of Southampton’s Health Sciences faculty, to replicate humans’ sense of touch – namely by focusing on neuroplasticity and retraining the brain’s neural pathways. A working prototype was developed with funding from Innovate UK, and the resulting product has been successfully tested by a leading stroke rehabilitation clinic.
MIAT are now repurposing the technology to make VR/AR (virtual reality/augmented reality) even more powerful, and in doing so, offer the potential to revolutionise an already ~$13.3 billion industry (expected to grow to $17.7 billion by 2026): military simulation.
Haptic feedback and VR
To meet the rigorous demands of the field, we must acknowledge that current technology in the VR/AR industry carries limitations: while it is already able to deliver an immersive audio-visual experience, it is notably lacking when it comes to the sense of touch. The solutions offered by traditional simulation technology have so far been basic haptic feedback and motion sensing, and although this has helped in the development of interfaces and controllers, it has not offered realistic environment sensing – a vital quality in the simulation of often life-or-death military scenarios.
True realism, in fact, is what MIAT believes to be the key missing attribute in the VR/AR industry, and by offering such a solution to the military, RealSim aims to address a range of defence and combat applications, including:
– Trainee bomb disposal engineers: able to feel the best way to deactivate an IED without it detonating, feeling the presence of wires and the pressure required to cut them – all while working safely in a classroom, touching the device only in virtual reality.
– Royal Military Police trainees: investigating a simulated crime scene for evidence, even being able to feel the difference between carpet fibres and hair.
– Weapons handling trainees: learning – from the shock of a poorly-controlled firearm – how to correctly hold and use their weapon.
– Trainee battlefield medics: able to quickly assess the condition of a wounded soldier, check vital signs and feel the presence of blood or other fluids.
– Army Medical Corps: virtually experiencing how to administer an epidural into the spine – while feeling the difference in resistance of incorrect versus ideal positioning. Army and Royal Navy fire trainees: immersed in a VR hostile environment with smoke, heat and flames – quickly learning to work together as a team. As they wear the haptic glove, they can lift a fire hose and feel its cold flow of pumping water. When they touch a door, they can get the sensation of heat, so they can apply the correct procedures for entering that room. In fact, the technology even facilitates training for ‘flashover’, wherein everything in a room ignites after it reaches a trigger temperature.
Currently, RealSim’s creators have a teaming agreement with Turbulent Design (TD), specialists in designing and developing new products for the military, and whose clients include Lockheed Martin, QinetiQ and the US Marine Corps. Aside from being in touch with other potential partners, MIAT is integrating its haptic capability with TD’s technology to bring a new dimension to simulation.
As Trevor Linn, CEO of Turbulent Design, explains: RealSim “opens the door to fully immersive training…
Your body and brain tell you you’re in that live situation where you have to make decisions that may be life changing.”
The VR/AR industry is already offering significant applications to the simulated training industry, but the technology as we understand it today is limited to being an immersive audio-visual world first – and a somaesthetic (relating to bodily sensations) experience second.
RealSim may be the start of fully multisensory simulation technology, and the applications for military training are considerable. If such technology enters the market, the future could see a world where soldiers are trained with real-world validity – and under pressures that reflect the potentially life-or-death scenarios that their role presents.
The possibilities ahead certainly look promising: “We are building a realistic, effective product that meets the military’s cutting-edge demands, and we have forged good relationships with partners.”