Flexor  - VR Carpal Tunnel Physiotherapy

About

LeapMotion - VR - Medical - Demo

Flexor is an interactive VR application that utilizes Leap-Motion to capture movement of both hands of the patient in order to facilitate stretching and yoga based exercises for the prevention and the practice physiotherapy for patients suffering from Carpal Tunnel Syndrome. Flexor aims to make this experience more interactive and enjoyable for patients and users by allowing them to experience small and interactive games while stretching and working out.

Flexor was made at HackTheNorth 2019. HackTheNorth is the biggest Canadian hackathon with 1500 student participants.

My Roles

  • Interaction Designer

  • VR/LeapMotion Prototyper

Team Size

  • 4 People

Timeline

  • 36 hours

Tools

  • LeapMotion

  • Oculus Rift S 

  • Unity

Demo

What is Carpal Tunnel Syndrome?

The syndrome can be caused and can be accelerated due to repetitive hand use or non-ergonomic hand and wrist position. 

"Carpal tunnel syndrome is a common condition that causes pain, numbness, and tingling in the hand and arm. The condition occurs when one of the major nerves to the hand — the median nerve — is squeezed or compressed as it travels through the wrist."

Source - OrthoInfo.com, September 2019

Area of hand affected by Carpal Tunnel Syndrome.

Why make this?

Carpal Tunnel Syndrome is a prevalent health issue in the tech world.

"Half of people who work with computers suffer from Repetitive Strain Injuries (RSI) symptoms: pain in the wrist, hand, neck and shoulders."

"Treatment and surgery is expensive. U.S. employees spend more than $7.4 billion in workers' compensation costs every year. For every $1 spent on RSI prevention, companies get back $17.80 in return."

By planning on a long career in the tech industry, I am at a high risk of suffering from Repetitive Strain Injury symptoms in the future. I want to be more aware of the issue and promote solutions as an investment in my own and my peers' health.

The audience

 Who is at high risk of Carpal Tunnel Syndrome?

Future Risk of Syndrome

  • Not reached the need of an operation yet.

  • Used as a preventative measure.

Early Stage of Syndrome

  • Has had an operation to release the pressure in the wrist nerves.

  • Used as a rehabilitation method.

Post-op and Rehab

  • Has had an operation to release the pressure in the wrist nerves.

  • Used as a rehabilitation method.

How do people deal with carpal tunnel syndrome currently?

Preventative measures

  • Stretching exercises

  • Not overusing hands

  • Hand Splints

  • Ergonomic hand positions while working

Nonintrusive treatement

  • Non inflamatory drugs

  • Injections to relieve swelling

  • For mild to moderate symptoms for less than 10 months

Surgery

  • Wrist ligament is cut to free the Carpal Tunnel nerve

  • For severe symptoms

How can I help? 

Gamification to Promote Stretching and Taking Breaks

 

Hand exercises are a useful practice for all 3 types of users and should be popularized in the tech industry. While hand exercises are used as a prevention technique, they are also used at all stages of the syndrome since it helps with recovery and even with rehabilitation after a person went through a surgery.

Creating an experience to follow an exercise routine

A regular exercise routine follow a few repetitions of stretching actions and pauses.

The tech

Oculus Rift

  • VR headset

  • World tracking with external sensors

LeapMotion Controller

  • Hand detection using an IR camera and image recognition

  • Remaps data to virtual models

By combining VR and hand tracking, I can get accurate hand position and orientation in VR.

Assessing stretching exercises

Due to certain motions involved in some stretching exercises, they were not able to be properly tracked by the LeapMotion controller.​ This restricted the types of exercises we could implement in our experience. The biggest challenge was that the hand had to be kept in the vision range of the LeapMotion and clearly shown to recognize the hand shape.

Vision cone of LeapMotion when mounted on a VR headset.

Hand is only recognized when a distinct shape is seen against background.

Pressing the hands together in front of your chest is a common exercise. However, LeapMotion cannot track when hands are joined together.

Example of an exercise not being tracked by LeapMotion.

Finger touching and using your opposite hands to stretch your wrist were also common exercises but there were tracking issues with individual finger detection and hands overlapping

More stretching exercise instructions.

More tracking limitations with LeapMotion.

An exercise that was able to be tracked accurately was the "wrist extension exercise."

The motion of the wrist is tracked accurately. 

How can this interaction be made more interesting and intuitive?

By adding motorcycle handlebars, the action of grabbing in front of the person and pulling the wrist back can seem very realistic and functional.

The same motion is used to accelerate a motorcycle. 

Tracking the wrist stretching exercise to control the speed of the motorcycle.

Following the exercise routine

The experience was made to follow the exercise routine, while adding variety to avoid fatigue.

Why would this work?

Making the interactions immersive and easy to understand, we are making the process relaxing and enjoyable. ​By engaging the player in the process, we create a more positive and proactive experience of prevention and rehabilitation. This increases retention and chance that the user will adopt the practice long term.

Outcome

This was a great project to work on because I became more aware of this health issue and how it might affect me and my teammates in the future.

I gained experience gamifying a physical exercise, which is a common use of medical VR. As VR technologies become more common, they will continue to improve the medical fields and it's very educative to learn about it.

This project was pitched to industry CFO's at HackTheNorth 2019 for feedback.

My team and I were ecstatic after submitting our project. We were able to combine two types of technology as well as our love and passion for designing games into a single project. We are happy with the results we've achieved and hope that we can explore further design project that involves gamification and healthcare.

Projects

illusionary photos.png
flexor.png