After developing a physical therapy device for the rehabilitation of stroke victims, Senseglove augmented its scope. Now, the Yes!Delft start-up is using simulators to give trainees at Volkswagen, Google Deepmind and other customers a virtual ‘feel’ for the job.
Back in 2015, Senseglove co-founders Gijs den Butter and Johannes Luijten were graduate students at Delft University of Technology. As one of their student projects, the duo had the idea to develop a robotic hand that could help stroke victims through rehabilitation while at home. “When you’re paralyzed by a stroke, you need to do a lot of exercises. Most of the time it’s just a physical therapist opening and closing your hand,” comments CEO Den Butter. “We thought it would be way more efficient if the patient could do it in their own time with the help from a robotic hand, because the more training you have, the better you recover.”
Armed with a prototype they developed as students, the pair went to the Yes!Delft high tech start-up incubator to pitch their idea, and in 2017, Senseglove was created. But just three months after launching the business and trying to validate its business model, it became very clear that getting into the therapeutics market, especially for a starting business, was going to be next to impossible.
Pivot
Just before this, virtual reality specialist Oculus was in the process of a buyout by Facebook. Even though the VR company had only released a development prototype, the social media tech conglomerate coughed up more than 2 billion dollars to add Oculus to its long list of acquisitions. For Den Butter, this was a definite sign. “It was pretty clear to me the virtual world was going to be the future of computing. It still needed several years to develop, but this 3D computing space and its developing market were going to be the next big thing.”
At this time, the state of the art in virtual reality consisted of a headset with a controller. If you wanted to interact with an object, for instance to pick something up, you would simply use the controller to click on it and the object would float in front of you until you clicked to put it down again. “I’m an interaction designer,” explains Den Butter, “this was not, in any way, a natural interaction.” Senseglove’s idea: if its system of robotics and motors could be used to open hands for therapy patients, the same method could be flipped around. The motors could be utilized to resist a closing hand and with the right tension, it could simulate the action and actually be felt.
Device lifecycle management for fleets of IoT devices
Microchip gives insight on device management, what exactly is it, how to implement it and how to roll over the device management during the roll out phase when the products are in the field. Read more. .
With its prototype in hand, still intended for medical purposes, the start-up headed to the Cebit Expo, where it attracted the attention of an unlikely client: Volkswagen. The automotive giant had a keen interest in the Senseglove system as it solved a growing problem in its industry. Namely, the car maker’s training centers were shifting toward the virtual world, but with the current technology, the training lacked the ability for any realistic touch feature. “So, what do you do when Volkswagen says they want to buy your product?” poses Den Butter. “You pivot. You switch your market. What’s better than having Volkswagen as the launching customer?”
The Senseglove team immediately got to work looking for ways to implement its product as a training device. But after about a year of development, it ran into a brick wall. The problem: the motor technology was simply way too complex to integrate into wearables. It was too bulky and heavy. The plan had to be scrapped and the company was once again sent back to the drawing board.
Dynamic gestures
This time, the Delft-based start-up came up with an innovation for a passive system of force feedback – what Senseglove considers to be its core IP. It consists of a glove-like exoskeleton that has a thin wire and strategically placed, patent-pending motor blocks embedded. In addition, there’s a system of internal pullies and springs that pull back on the wire to simulate force. The system can create a variable stoppage force of up to 40 newtons, or 4 kg. Den Butter: “That’s quite intense for force feedback on the fingers. With that sort of feedback, you’re able to actually feel the virtual objects.”
In addition to the developments in hardware, this new direction required some hefty adjustments to the software, all of which are done in house. As the system is no longer used only for open and close functions, it needs to be designed to take into account natural interaction and physics-based gripping. This means that it’s not just the ‘contact’ with objects that needs to be felt, there needs to be a recognition of dynamic gestures like squeezing, rotating or turning the interactive item – all of which are software-heavy features.
As there’s very little processing power in the gloves themselves, nearly all the interactions are calculated externally. Input from embedded sensors is sent to a computer where the information is processed and then relayed back to the gloves. This communication can be streamed wirelessly, via Bluetooth, or wired with a USB cable. “The actual intelligence is always in a computer or server,” clarifies Den Butter. “But there’s a microprocessor in the glove that can process those commands. This dictates how to interpret signals and activates vibrotactile motors inside the palm and fingertips.”
Obstacles
Though the Senseglove system is still in development, the start-up went to market with its first development kit in September of 2018. Having already produced hundreds of units, its customer list consists of several automotive, research and robotics companies from around the world. These include VW, Airbus, Google Deepmind and Honda. More importantly, however, is that these aren’t just customers, but they’re providing feedback to the young company to assist with further tweaks and development of the product. “They’re giving us constant feedback on how to improve both our software and hardware to better meet their needs,” says Den Butter.
Despite all the progress made, there’s still a laundry list of obstacles Senseglove needs to overcome. First, because all the computations take place externally, issues of lag have exposed themselves. With the wireless transmission of the data, via Bluetooth, it takes about 20 milliseconds to receive the force feedback, which is noticeable and less than ideal. To remedy this, the best option is to connect via USB cable, which reduces the latency to around 10 milliseconds and is considered to be good. The problem: now you’re tied to a computer with limited movement – which also isn’t optimal.
Another issue with the device, at least in its current form, is the sheer size. The exoskeleton is big and bulky. Yes, it’s adjustable, making it a one-size-fits-all solution and that helps to keep the cost down, but the reality is that the ‘glove’ is a rather large structure and it fits somewhat awkwardly.
To solve this issue, Senseglove is already working on its next-gen glove, which is designed to fit more like a glove and made from fabric, rather than the hard exoskeleton. “The technology is there to actually build this into a glove-ish type of product,” claims Den Butter. “It will contain small pieces of plastic to offer some form of stiffness and rigidity as this is needed to be able to transmit the force to the end of the fingertips. That’s where you feel most of the forces you encounter.”