Engineers at Newcastle University are using computer vision to develop a new generation of prosthetic limbs that enable wearers to reach for objects automatically, just like a hand. The bionic hand is fitted with a camera that instantaneously takes a picture of the object in front of it, assesses its shape and size and triggers a series of movements in the hand. The work is funded by the Engineering and Physical Sciences Research Council (EPSRC).
“Responsiveness has been one of the main barriers to artificial limbs,” explains Dr Kianoush Nazarpour. “For many amputees the reference point is their healthy arm or leg so prosthetics seem slow and cumbersome by comparison.” Current prosthetic hands are controlled via myoelectric signals – that is electrical activity of the muscles recorded from the skin surface of the stump. Controlling them takes practice, concentration and, crucially, time.
Using neural networks – the basis for Artificial Intelligence – the computer was shown numerous images of objects and taught to recognise the ‘grip’ needed for different objects. Using a basic webcam fitted to the prosthesis, the hand ‘sees’ an object, picks the most appropriate grasp and sends a signal to the hand – all within a matter of milliseconds and ten times faster than any other limb currently on the market.
Grouping objects by size, shape and orientation, according to the type of grasp that would be needed to pick them up, the team programmed the hand to perform four different ‘grasps’: palm wrist neutral (such as when you pick up a cup); palm wrist pronated (such as picking up the TV remote); tripod (thumb and two fingers) and pinch (thumb and first finger).
“Using computer vision, we have developed a bionic hand which can respond automatically – in fact, just like a real hand, the user can reach out and pick up a cup or a biscuit with nothing more than a quick glance in the right direction,” says Dr Nazarpour.
A small number of amputees have already trialled the new technology and now the team are working with experts at Newcastle upon Tyne Hospitals NHS Foundation Trust to offer the ‘hands with eyes’ to patients at Newcastle’s Freeman Hospital.
“It’s cheap and it can be implemented soon because it doesn’t require new prosthetics, as we can just adapt the ones we have,” Dr Nazarpour says. “But this is a stepping stone towards our ultimate goal.” In February 2018, the project won a Netexplo UNESCO Award. The team’s work is part of a larger research project to develop a bionic hand that can sense pressure and temperature and transmit the information back to the brain.