Posts | Comments

Planet Arduino

Archive for the ‘usb hid’ Category

Controlling your computer with a wave of the hand seems like something from science fiction, and for good reason. From Minority Report to Iron Man, we’ve seen plenty of famous actors controlling their high-tech computer systems by wildly gesticulating in the air. Meanwhile, we’re all stuck using keyboards and mice like a bunch of chumps.

But it doesn’t have to be that way. As [Norbert Zare] demonstrates in his latest project, you can actually achieve some fairly impressive gesture control on your computer using a $10 USD PAJ7620U2 sensor. Well not just the sensor, of course. You need some way to convert the output from the I2C-enabled sensor into something your computer will understand, which is where the microcontroller comes in.

Looking through the provided source code, you can see just how easy it is to talk to the PAJ7620U2. With nothing more exotic than a switch case statement, [Norbert] is able to pick up on the gesture flags coming from the sensor. From there, it’s just a matter of using the Arduino Keyboard library to fire off the appropriate keycodes. If you’re looking to recreate this we’d go with a microcontroller that supports native USB, but technically this could be done on pretty much any Arduino. In fact, in this case he’s actually using the ATtiny85-based Digispark.

This actually isn’t the first time we’ve seen somebody use a similar sensor to pull off low-cost gesture control, but so far, none of these projects have really taken off. It seems like it works well enough in the video after the break, but looks can be deceiving. Have any Hackaday readers actually tried to use one of these modules for their day-to-day futuristic computing?

When building your own homebrew computer, everything is a challenge. Ultimately, that’s kind of the point. If you didn’t want to really get your hands dirty with the nuts and bolts of the thing, you wouldn’t have built it in the first place. For example, take the lengths to which [rehsd] was willing to go in order to support standard USB mice on their 6502 machine.

Code for mapping mouse movement to digital output.

The idea early on was to leverage existing Arduino libraries to connect with a standard USB mouse, specifically, the hardware would take the form of an Arduino Mega 2560 with a USB Host Shield. There was plenty of code and examples that showed how you could read the mouse position and clicks from the Arduino, but [rehsd] still had to figure out a way to get that information into the 6502.

In the end, [rehsd] connected one of the digital pins from the Arduino to an interrupt pin on the computer’s W65C22 versatile interface adapter (VIA). Then eleven more digital pins were connected to the computer, each one representing a state for the mouse and buttons, such as MOUSE_CLICK_RIGHT and MOUSE_LEFT_DOWN.

Admittedly, [rehsd] says the mouse action is far from perfect. But as you can see in the video after the break, it’s at least functional. While the code could likely be tightened up, there’s obviously some improvements to be made in terms of the electrical interface. The use of shift registers could reduce the number of wires between the Arduino and VIA, which would be a start. It’s also possible a chip like the CH375 could be used, taking the microcontroller out of the equation entirely.

From classic breadboard builds to some impressively practical portable machines, we’ve seen our fair share of 6502 computers over the years. Despite the incredible variation to be found in these homebrew systems, one thing is always the same: they’re built by some of the most passionate folks out there.

[Thanks to Jim for the tip.]

There was a time when building realistic simulations of vehicles was the stuff of NASA and big corporations. Today, many people have sophisticated virtual cockpits or race cars that they use with high-resolution screens or even virtual reality gear. If you think about it, a virtual car isn’t that hard to pull off. All you really need is a steering wheel, a few pedals, and a gear shifter. Sure, you can build fans to simulate the wind and put haptics in your seat, but really the input devices alone get you most of the way there. [Oli] decided he wanted a quick and easy USB gear shifter so he took a trip to the hardware store, picked up an arcade joystick, and tied it all together with an Arduino Leonardo. The finished product that you can see in the video below cost about $30 and took less than six hours to build.

The Leonardo, of course, has the ability to act like a USB human interface device (HID) so it can emulate a mouse or a keyboard or a joystick. That comes in handy for this project, as you would expect. The computer simply has to read the four joystick buttons and then decide which gear matches which buttons. For example up and to the left is first gear, while 4th gear is only the down button depressed. A custom-cut wooden shifter plate gives you the typical H pattern you expect from a stick shift.

Of course, the joystick doesn’t have a long handle like a true stick shift, so [Oli] added some extensions. In addition, a real shifter doesn’t require you to hold it in position as a joystick would. To rectify this, the shifter plate has magnets that grab the stick and hold it. They aren’t strong enough that you can’t move the stick, but they are strong enough to keep it from moving on its own.

We noticed that the design doesn’t allow for a clutch, so it isn’t quite the same as driving a real stick. However, [Oli] mentions several upgrades he has in mind and a clutch is one of them. Some haptics would be a cool addition so could feel the gears grind if you didn’t do the shift correctly.

The last shifter we saw like this was 3D printed. It is getting harder to find a car in the US with a manual transmission, but [Kristina Panos] is definitely a fan.

You’ve (probably) got four limbs, so why are you only using half of them when you’re working on the computer? Just because your toes don’t have the dexterity to type (again, probably) doesn’t mean your feet should get to just sit there doing nothing all day. In a recent project, [MacCraiger] shows you just how easy it can be to put some functionality under foot by building a pair of media control stomp switches.

Crimp pin connectors grant +50 professionalism.

If the devices pictured above look a lot like guitar effects, that’s because they share a lot of parts. [MacCraiger] used the same sort of switch and aluminum case that you might see on a pedal board, as he figured they’d be better suited to a lifetime of being stepped on than something he 3D printed.

Up on the desk, and this time in a printed case, is the Arduino Leonardo that they connect to. The wiring for this project is very straightforward, with the switches connected directly to the GPIO pins. From there, the Arduino firmware emulates a USB Human Interface Device and fires off the appropriate media control keystrokes to skip to the next track or pause playback depending on which switch has been engaged.

This hardware isn’t exactly breaking any new ground here, but we did like how [MacCraiger] used standard 3.5 mm audio cable and the associated jacks to connect everything up. It’s obviously on-theme for what’s essentially a music project, but more importantly, gives the whole thing a very professional look. Definitely a tip to mentally file away for the future.

For the more accomplished toe-tapper, our very own [Kristina Panos] recently recently took us through the construction of her macro slinging footstool. Between these two examples of bespoke peripherals, you should have everything you need to create your own custom input devices. We suppose you could even make one that’s hand operated if you’re into that sort of thing.

The modern keyboard enthusiast is blessed with innumerable choices when it comes to typing hardware. There are keyboards designed specifically for gaming, fast typing, ergonomics, and all manner of other criteria. [iot4c] undertook their own build for no other reason than nostalgia – which sounds plenty fun to us.

An Arduino Leonardo is pressed into service for this hack. With its USB HID capabilities, it’s perfectly suited for custom keyboard builds. It’s built into a working Atari 65XE computer, and connected to the keyboard matrix. The Keypad and Keyboard libraries are pressed into service to turn keypresses on the 80s keyboard into easily digseted USB data.

There’s plenty of room inside the computer for the added hardware, with the USB cable neatly sneaked out the rear. [iot4c] notes that everything still works and the added hardware does not cause any problems, as long as it’s not used as a computer and a keyboard at the same time.

It’s possible to do a similar hack on the Commodore 64, too. If you’re doing tricky keyboard builds yourself, you know where to send ’em.

In the era of touch screens and capacitive buttons, we’d be lying if we said we didn’t have the occasional pang of nostalgia for the good old days when interfacing with devices had a bit more heft to it. The physical clunk and snap of switches never seems to get old, and while you can always pick up a mechanical keyboard for your computer if you want to hear that beautiful staccato sound while firing off your angry Tweets, there’s a definite dearth of mechanical interface devices otherwise.

[Jeremy Cook] decided to take matters into his own hands (literally and figuratively) by designing his own multipurpose USB rotary input device. It’s not a replacement for the mouse or keyboard, but a third pillar of the desktop which offers a unique way of controlling software. It’s naturally suited to controlling things like volume or any other variable which would benefit from some fine tuning, but as demonstrated in the video after the break even has some gaming applications. No doubt the good readers of Hackaday could think of even more potential applications for a gadget like this.

The device is built around the diminutive Arduino-compatible PICO board by MellBell, which features a ATmega32u4 and native USB. This allowed him to very rapidly spin up a USB Human Interface Device (HID) with minimal headaches, all he had to do was hang his buttons and rotary encoder on the PICO’s digital pins. To that end, he [Jeremy] used the fantastic I2C rotary encoder designed by [fattore.saimon], which readers may remember as a finalist in the Open Hardware Design Challenge phase of the 2018 Hackaday Prize. He also added a NeoPixel ring around the encoder to use for some visual feedback and because, well, it just looks cool.

Since all of the core components are digital, there’s not a whole lot required in the way of wiring or passive components. This let [Jeremy] put the whole thing together on a piece of perfboard, freeing him up to spend time designing the 3D printed enclosure complete with translucent lid so he can see the NeoPixel blinkenlights. He got the tolerances tight enough that the whole device can be neatly press-fit together, and even thought to add holes in the bottom of the case so he could push the perfboard back out if he needed to down the line.

[Jeremy] spends a good chunk of the video going over the software setup and development of the firmware, and details some of the nuances he had to wrap his head around when working with the I2C encoder. He also explains the math involved in getting his encoder to emulate a mouse cursor moving in a circle, which he thinks could be useful when emulating games that originally used an encoder such as Tempest or Pong.

We’ve seen similar USB “knobs” in the past for controlling volume, but the additional inputs that [Jeremy] built into his version definitely makes it a bit more practical. Of course we’re suckers for interesting USB input devices to begin with.

Motion control is a Holy Grail of input technology. Who doesn’t want an interface that they can control with simple and natural movements? But making this feel intuitive to the user, and making it work robustly are huge hills to climb. Leap Motion has done an excellent job creating just such a sensor, but what about bootstrapping your own? It’s a fun hack, and it will give you much greater appreciation for the currently available hardware.

Let’s get one thing straight: This device isn’t going to perform like a Leap controller. Sure the idea is the same. Wave your hands and control your PC. However, the Leap is a pretty sophisticated device and we are going to use a SONAR (or is it really SODAR?) device that costs a couple of bucks. On the plus side, it is very customizable, requires absolutely no software on the computer side, and is a good example of using SONAR and sending keyboard commands from an Arduino Leonardo to a PC. Along the way, I had to deal with the low quality of the sensor data and figure out how to extend the Arduino to send keys it doesn’t know about by default.

The Plan

The plan is to take an inexpensive SONAR module (the HC-SR04) and an Arduino Leonardo and use it to perform some simple tasks by mimicking keyboard input from the user. The Leonardo is a key element because it is one of the Arduinos that can impersonate a USB keyboard (or mouse) easily. The Due, Zero, and Micro can also do the trick using the Arduino library.

I wanted to determine how many gestures I could really determine from the HC-SR04 and then do different things depending on the gesture. My first attempt was just to have the Arduino detect a few fingers or a hand over the sensor and adjust the volume based on moving your hand up or down. What I didn’t know is that the default Arduino library doesn’t send multimedia keys! More on that later.

How the SONAR Works

The SONAR boards come in several flavors, but the one I used takes 4 pins. Power and ground, of course, are half of the pins. In fact, my early tests didn’t work and I finally realized the module requires more power than I could draw from the Arduino. I had to add a bench supply to power the module (and, of course, I could have powered the module and the Arduino from the same supply).

The other two pins are logic signals. One is an input and a high-going pulse causes the module to ping (8 cycles at 40kHz). There is a delay and then the other pin (an output) will go high and return low when the module detects the return ping. By measuring the time between your signal to ping and the return, you can judge the distance. In my case, I didn’t care about the actual distance (although that’s easy to compute). I just wanted to know if something was farther away or closer.

pingThe scope trace to the right shows the sensor pointing at something relatively near. The top trace is the start pulse and the bottom trace is the input to the Arduino. The center trace is the output of the SONAR transducer. All the signal conditioning is inside the sensor, so you don’t need to worry about the actual signal processing to generate and recover the audio. You only need to measure the width of that bottom pulse.

The scope has persistence and you can see that the bottom trace does not always come out right at the same time (look at falling edge and you can see “ghosts” for previous samples. It shouldn’t come as a surprise that it may take a little effort to reduce the variations of the signal coming back from the SONAR.

Noise Reduction and Actions

Averaging

I wound up trying several different things to attempt to stabilize the input readings. The most obvious was to average more than one sample. The idea is that one or two samples that are way off will get wiped out by the majority of samples that are hovering around some center value. I also found that sometimes you just miss–especially when looking for fingers–and you get a very large number back. I elected to throw out any data that seemed way off when compared to the majority of received data.

Verifying

One other tactic I used was to verify certain elements with a second reading. For example, the start event occurs when the SONAR reports a value under the idle limit. The idle limit is a number less than the reading you get when the SONAR is pointed at the ceiling (or wherever it is pointing) and you don’t have anything blocking it. To recognize a valid start, the code reads twice to make sure the value is under the limit.

The code inside the Arduino loop is essentially a state machine. In the IDLE state, it looks for a reading that is below the idle limit. When found, that causes a transition to the sampling state. When the reading goes up or down more than some preset value, the code in the sample state sends a volume up or down key via the keyboard interface. If the sample goes back over the idle limit, the state machine returns to IDLE.

I got pretty good results with this data reduction,  but I also found the NewPing library and installed it. Even though it isn’t hard to write out a pulse and then read the input pulse, the NewPing library makes it even easier (and the code shorter). It also has a method, ping_median, that does some sort of data filtering and reduction, as well.

You can select either method by changing the USE_NEW_PING #define at the top of the file. Each method has different configuration parameters since the return values are slightly different between the two methods.

I said earlier that the code sends volume up and down commands when it detects action. Actually, the main code doesn’t do that. It calls an action subroutine and that subroutine is what sends the keys. It would be easy to make the program do other things, as well. In this case, it simply prints some debugging information and sends the keys (see below). I didn’t react to the actual position, although since the action routine gets that as a parameter, you could act on it. For example, you could make extreme positions move the volume up two or three steps at a time.

Sending Keyboard Commands

I wanted to send standard multimedia keys to the PC for volume up and down. Many keyboards have these already and usually your software will understand them with no effort on  your part. The problem, though, is that the default Arduino library doesn’t know how to send them.

Fortunately, I found an article about modifying the Arduino’s library to provide a Remote object that wasn’t exactly what I had in mind, but would work. Instead of sending keys, you have methods on a global Remote object that you can call to do things like change or mute the volume. The article was for an older version of the Arduino IDE, but it wasn’t hard to adapt it to the version I was using (version 2.1.0.5).

The action routine really only needs the UP_IN and DN_IN cases for this example. However, I put in all four branches for future expansion. Here’s the action subroutine:

void action(int why, unsigned value=0)
{
 Serial.print(value);
 switch (why)
 {
 case START_IN:
 Serial.println(" Start");
 break;
 case STOP_IN:
 Serial.println(" Stop");
 break;
 case UP_IN:
 Serial.println(" Up");
 Remote.increase();
 break;
 case DN_IN:
 Serial.println(" Down");
 Remote.decrease();
 break;
 }
}

The Final Result

sonarThe final result works pretty well, although the averaging makes it less responsive than you might wish. You can turn down the number of samples to make it faster, but then it becomes unreliable. You can download the complete code from Github. The first thing you’ll want to do is check the top of the file to make sure your module is wired the same (pin 3 is the trigger pin and pin 8 is the echo return pin). You’ll also want to select if you are going to use the NewPing library or not. If you choose to use it, you’ll need to install it. I flipped my Leonardo upside down and mounted it on a breadboard with some adapters (see picture to right). It really needs a more permanent enclosure to be useful. Don’t forget to give the SONAR module its own 5V power supply.

If you look near the top of the loop function there is an #if statement blocking out 3 lines of code. Change the 0 to a 1 and you’ll be able to just get averaged data from the sensor. Put the module where you want it and see what kind of numbers you get. Depending on the method I used I was getting between 4000 and 9000 pointed up to the ceiling. Deduct a bit off of that for margin and change IDLETHRESHOLD (near the top of the file) to that number.

The DELTATHRESHOLD is adjustable too. The code sees any change that isn’t larger than that parameter as no change. You might make that bigger if you have shaky hands or smaller if you want to recognize more “zones”. However, the smaller the threshold, the more susceptible the system will be to noise. The display of samples is helpful because you can get an idea how much the readings vary when your hand is at a certain spot over the sensor. You can try using one or two fingers, but the readings are more reliable when the sound is bouncing off the fleshy part of your palm.

If you want to add some more gestures, you may have to track time a bit better. For example, holding a (relatively) stationary position for a certain amount of time could be a gesture. To get really sophisticated gestures, you may have to do some more sophisticated filtering of the input data than a simple average. A Kalman filter might be overkill, but would probably work well.

If you look around, many robots use these sensors to detect obstacles. Makes sense, they’re cheap and work reasonably well. There are also many projects that use these to show an estimate of distance (like an electronic tape measure). However, you can use them for many other things. I’ve even used a similar set up to measure the level of liquid in a tank and earlier this week we saw ultrasonic sensors used to monitor rice paddies.

If you really want to get serious, [uglyduck] has some analysis of what makes spurious readings on this device. He’s also redesigning them to use a different processor so he can do a better job. That might be a little further than I’m willing to go, although I was impressed with the 3D sonic touchscreen which also modified the SONAR units.


Filed under: Arduino Hacks, Featured, peripherals hacks


  • Newsletter

    Sign up for the PlanetArduino Newsletter, which delivers the most popular articles via e-mail to your inbox every week. Just fill in the information below and submit.

  • Like Us on Facebook