Posts | Comments

Planet Arduino

Archive for the ‘Gesture Control’ Category

Smartphones have become a part of our day-to-day lives, but for those with visual impairments, accessing one can be a challenge. This can be especially difficult if one is using a cane that must be put aside in order to interact with a phone.

The GesturePod offers another interface alternative that actually attaches to the cane itself. This small unit is controlled by a MKR1000 and uses an IMU to sense hand gestures applied to the cane. 

If a user, for instance, taps twice on the ground, a corresponding request is sent to the phone over Bluetooth, causing it to output the time audibly. Five gestures are currently proposed, which could expanded upon or modified for different functionality as needed.

People using white canes for navigation find it challenging to concurrently access devices such as smartphones. Build­ ing on prior research on abandonment of specialized devices, we explore a new touch free mode of interaction wherein a person with visual impairment can perform gestures on their existing white cane to trigger tasks on their smartphone. We present GesturePod, an easy-to-integrate device that clips on to any white cane, and detects gestures performed with the cane. With GesturePod, a user can perform common tasks on their smartphone without touch or even removing the phone from their pocket or bag. We discuss the challenges in build­ ing the device and our design choices. We propose a novel, efficient machine learning pipeline to train and deploy the gesture recognition model. Our in-lab study shows that Ges­ turePod achieves 92% gesture recognition accuracy and can help perform common smartphone tasks faster. Our in-wild study suggests that GesturePod is a promising tool to im­ prove smartphone access for people with VI, especially in constrained outdoor scenarios.

[Dimitris Platis] wanted to add gesture control to his PC. You’d think that would be expensive, but by combining a diminutive Arduino, a breakout board with a gesture controller, and an interconnect PCB, he managed to pull it off for about $7. That doesn’t include the optional 3D-printed case and we think you could omit the interconnect board if you don’t mind some wires and further cut costs. [Dimitris] calls it Nevma, and you can see how the device works in the video below.

The heart of the project is a sensor that measures light and motion. The chip and the breakout board are just a couple of bucks if you order them from China. You can find them in the US if you don’t mind spending a little bit more. The device has an I2C interface, and [Dimitris] uses a tiny Mini SS Micro for the USB interface and the CPU.

The sensor chip is made for the mobile phone market and can also sense proximity. From its data sheet:

Gesture detection utilizes four directional photodiodes to sense reflected IR energy… The architecture of the the gesture engine features automatic activation (based on proximity engine results), ambient light subtraction, cross-talk cancellation, dual 8-bit data converters, power saving inter-conversion delay, 32-dataset FIFO, and interrupt-driven I2C communications.

That seems like a lot of power for a few bucks. Sparkfun has a library (and a matching board) and [Dimitris] uses it. The library is released as beerware. In particular, the documentation says: “The code is beerware; if you see me (or any other SparkFun employee) at the local, and you’ve found our code helpful, please buy us a round!”

We really like Nevma. You don’t have to hold any device in your hand. It also looks slicker than the solutions we’ve seen (and even created) using SONAR.


Filed under: Arduino Hacks

While keyboards are great, and custom shortcuts can make things even better, why not do away with buttons and knobs altogether, controlling your computer instead via simple gestures? Maker Ben James has done just this, creating a unique interface using a Skywriter device to pick up finger movements, along with an Arduino Leonardo to emulate a keyboard on his laptop.

Since the Skywriter can detect a number of gestures, James assigned various swipes, taps and circular motions to keyboard commands. As you can see in the video here, the results are pretty neat. 

More info on this project can be found on his blog post, and its code is available on GitHub.

While touchscreens are nice, wouldn’t it be even better if you could simply wave your hand to your computer to get it to do what you want? That’s the idea behind this Iron Man-inspired gesture control device by B. Aswinth Raj.

The DIY system uses an Arduino Nano mounted to a disposable glove, along with hall effect sensors, a magnet attached to the thumb, and a Bluetooth module. This smart glove uses the finger-mounted sensors as left and right mouse buttons, and has a blue circle in the middle of the palm that the computer can track via a webcam and a Processing sketch to generate a cursor position.

You can see it demonstrated in the video below, drawing a stick man literally by hand, and also controlling an LED on the Nano. Check out this write-up for code and more info on the build!

Using a Kinect sensor with MATLAB/Simulink and an Arduino, B.Avinash and J.Karthikeyan made a robotic arm to mimic their every move.

If you need a robotic arm to follow your movements, the Kinect sensor is a great place to start. On the other hand, it’s a long leap programming-wise to go from sensor input to coordinated movement of servo motors. Through a toolchain stretching from the sensor itself, to a computer, and finally to an Arduino Mega controlling the servos directly, Avinash and Karthikeyan did just that.

For their process, the computer takes data from the Kinect sensor, then translates it into servo angles using the MATLAB and Simulink computer programs. Resulting data is then fed into the Arduino via a serial connection, which controls the robot’s movements appropriately with a slight delay.

Be sure to check out the project’s Instructables page to learn more about this awesome build!



  • Newsletter

    Sign up for the PlanetArduino Newsletter, which delivers the most popular articles via e-mail to your inbox every week. Just fill in the information below and submit.

  • Like Us on Facebook