Posts | Comments

Planet Arduino

Archive for the ‘ESP8266’ Category

Potentially, one of the great things about having a device connected to the network is that you can update it remotely. However, how do you make that happen? If you use the Arduino setup for the ESP8266 or ESP32, you might try [scottchiefbaker’s] library which promises to make the process easy.

Adding it looks to be simple. You’ll need an include, of course. If you don’t mind using port 8080 and the path /webota, you only need to call handle_webota() from your main loop. If you want to change the defaults, you’ll need to add an extra call in your setup. You also need to set up a few global variables to specify your network parameters.

The only caveat is that long delay statements in your loop can block things from working and aren’t a great idea anyway. If you have them, you can replace all your delay calls with webota_delay which will stop the system from ignoring update requests.

The code started from a different online tutorial but packaged the code up nicely for reuse. To do an update, simply navigate to the device with a web browser and use the correct port number and path. From there you can upload a new binary image taken from the Arduino IDE with the export compiled binary command.

The only concern we saw was the code didn’t appear to authenticate you at all. That means anyone could load code into your ESP. That might be ok on a private network, but on the public Internet it is surely asking for trouble. The original tutorial code did have a hardcoded user and password, but it didn’t look very useful as the password was in the clear and didn’t stop you from uploading if you knew the right URL. Dropping it from the library probably makes sense, but we would want to build some kind of meaningful security into anything we deployed.

If you have a network connection, we’ve seen the same trick done with a normal Arduino with a wireless chip. You can even do it over WiFi but using an ESP8266 which you’ll then want to be able to update, too.

It used to be that Web browsing was simple. You asked a server for some text, which was duly sent, and then formatted by your browser. Now a web page is as likely to be a full-blown application that is reading mail, editing text, or lots of other things and may use WebSockets to create a back channel to the server. Thanks to affordable hardware like the ESP8266 one of those things a modern web browser can do is sense and control the real world. [Acrobotic] has an interesting video about using WebSockets to allow a browser to talk to an ESP8266 web server in real time. You can see his simple demo in the video below.

Of course, you’ll use the usual language you use on the ESP8266 — [Acrobotic] uses C++ in the Arduino IDE. On the browser side you’ll use JavaScript, although that will be embedded in your C++ program which acts as a web server.

It’s as well to remember that there are several other ways you could do this. You could, for example ask for a different URL, or pass data in a query string. The problem here is that the performance would suffer as you have to establish a new connection every time. you want to transact with the server. You could also use AJAX methods but they are not as efficient either since they are primarily aimed at updating a portion of a web page dynamically. The web socket is simple enough and as you can see in the video, the performance is quite good. It also facilitates non-browser based clients that use the same service.

We’ve seen this technique used to fly a quadcopter. WebSockets have been around for a while, so your browser should support them. If it doesn’t, though, you can always use this hack — at least in one direction.

Connecting your shiny new ESP8266 to WiFi can be as simple or as complicated as you please. Most people decide to manually add it. Some people find clever ways to make the bloody thing connect itself. [Eduardo Zola] transfers his WiFi password using the flashing light of a smartphone screen.

A simple photo-resistor and a bit of tinkering allows him to easily send credentials — or any data really — to his ESP8266, through the power of LiFi. Short for Light Fidelity, LiFi transmits data using light with on and off states representing digital values. It can use visible light, or reach into either the ultraviolet or infra-red radiation if need be. For the nitty-gritty details on the subject, check out our primer on LiFi.

 A flashing LCD screen and a photo-resistor barely make the cut for a one-way LiFi system, but [Eduardo Zola] makes it work. The approach is to build a resitor divider and watch an input pin on the ESP for changes.

The trick is to keep ambient light out of the mix. The test sensor shown here places the LDR in a black cap, but [Eduardo] 3D-Printed a slick little enclosure for his reverse flashlight so it fits flush with the phone screen. One click and about half a minute of a flashing screen later, and the Wi-Fi credentials are transferred. This circuit could really be added onto any project, for short data transfers. With a bit more work on the sensor circuit, speed could be improved with the limiting factor being the timing on the phone screen itself.

Since the ESP8266 has its own WiFi connection, it’s likely you’ll use that for data transfer once the LiFi gets it onto the network. But any situation where you don’t have a full user input or a network connection could benefit from this. Pull out that old scrolling LED matrix project and add this as a way to push new messages to the device!

When [Im-pro] wants a display, he wants it to spin.  So he built a persistence of vision (POV) display capable of showing a 12-bit color image of 131 x 131 pixels at 16 frames per second. You can see a video about the project below, but don’t worry, you can view it on your normal monitor.

The project starts with a Java-based screen capture on a PC. Data goes to the display wirelessly to an ESP8266. However, the actual display drive is done by an FPGA that drives the motor, reads a hall effect index sensor, and lights the LEDs.

Perhaps the most interesting part of the project is the FPGA-based mapping of the rectangular coordinates of the incoming video to the polar coordinates required by the display. There are 4 arms of LEDs or “wings” and a 3D printed structure that is all included in the post.

The FPGA is a Cmod S6 which is a breakout board for a Xilinx Spartan 6 with more than enough horsepower to handle the workload. There are also custom PCBs involved, so when you think about it, it is a fairly wide-ranging project. Java software, ESP8266 software, FPGA configurations, a 3D-printed design, and PCB layouts. If you want something simple to tackle that has a bit of everything in it, this might be your next project.

Most of the POV displays we see don’t have this kind of color-depth and resolution. We’ve seen displays built around fans. Our favorite, though, is the dog speedometer.

Pools have come a long way. It used to be you had a pump and if you were lucky it had a mechanical timer switch on it. That was it. Now you have digital controllers and spa jets and heaters. You can even get them that connect to your home automation system. If your pool isn’t new enough to do that already, you can get a range of add-on accessories. For a price. [Rob] paid $500 to get a remote for his pool. It wasn’t even WiFi, just a simple RF remote. In 3 years, the transmitter had burned out ($300 to replace) and he decided he had enough. For $20, [Rob] added MQTT control and monitoring to his pool using an ESP8266. You can see the video description of the project below.

Naturally, the instructions are a bit specific to the Pentair system he has. However, it isn’t as specialized as you might think. The project relies on the connection for a wired “spa-side remote” that most modern pool systems support. The electrical connections for these aren’t quite standard, but they are all very similar, so you have a good chance of reproducing this for your setup assuming you have a connection for one of these wired remotes.

The remote has a few buttons and LED for status. The LED reacts differently depending on the pool’s current mode, so connecting there not only gives you control but also allows you to provide some limited status. It isn’t going to let you monitor pump currents or anything exotic, but it is a simple place to gain access. Using the Arduino pulse input function makes it easy to sense if the LED is on, off, or blinking. Another sensor reads the water temperature. The controller makes it available, but it isn’t simple to read, so the project just reads the raw sensor voltage from the existing thermistor and computes the temperature.

[Ron] does a nice job of explaining some basic concepts like using opto-isolators. However, the real value to the video is the easy way to interface to the existing controller. A little configuration into Home Automation rounds out the project.

If you have an older system, you might like to see more of a pool system rebuild. If you are interested in controlling the pool chemistry, we’ve seen that before, too.

We admit, we see a lot of weather stations. What makes [Mike Diamond’s] take on this old favorite interesting is that it is tiny enough to carry with you, and uses your cell phone as a hotspot to deliver its data. Of course, that assumes you have a phone that can act as a hotspot.

The parts are straightforward, a power supply, an ESP8266, and a weather sensor board. It looks as though you could easily slip the whole affair into a tube or maybe a 3D printed enclosure. We were a little concerned about the bare wire used, but as [Mike] points out you can use insulated wire if you like, and we’d encourage you to do so.

There are some modifications required including removing the pin headers. [Mike] uses the old trick of smacking your hand on the table after melting the pins. You can also heat up each pin and pull it out with pliers. Or, if you have a hot air gun, get all the pins molten at once — just don’t heat up more of the board than you need.

On the data end, the ESP8266 uses Cayenne to transmit data which is the same kind of IoT backend we see a lot lately. On the one hand, this allows you to distribute it without developing a phone application. On the other hand, we would have been tempted to just make the ESP8266 a web server and populate a simple web page. Of course, you could still do that if you wanted to.

We’ve seen plenty of weather stations, but a lot of them are not nearly as compact. If you want to go old school, there’s always the TI 99/4A weather station.

We use the Internet to do everything from filing our taxes to finding good pizza, but most critically it fulfills nearly all of our communication needs. Unfortunately, this reliance can be exploited by those pulling the strings; if your government is trying to do something shady, the first step is likely to be effecting how you can communicate with the outside world. The Internet is heavily censored and monitored in China, and in North Korea the entire country is effectively running on an intranet that’s cutoff from the wider Internet. The need for decentralized information services and communication is very real.

While it might not solve all the world’s communication problems, [::vtol::] writes in to tell us about a very interesting communication device he’s been working on that he calls “Hot Ninja”. Operating on the principle that users might be searching for accessible Wi-Fi networks in a situation where the Internet has been taken down, Hot Ninja allows the user to send simple messages through Wi-Fi SSIDs.

We’ve all seen creatively named Wi-Fi networks before, and the idea here is very much the same. Hot Ninja creates a Wi-Fi network with the user’s message as the SSID in hopes that somebody on a mobile device will see it. The SSID alone could be enough depending on the situation, but Hot Ninja is also able to serve up a basic web page to devices which actually connect. In the video after the break, [::vtol::] even demonstrates some rudimentary BBS-style functionality by presenting the client devices with a text field, the contents of which are saved to a log file.

In terms of hardware, Hot Ninja is made up of an Arduino Mega coupled to three ESP8266 boards, and a battery to keep it all running for up to eight hours so you can subvert a dictatorship while on the move. The user interface is provided by a small OLED screen and a keyboard made entirely of through-hole tactile switches, further reinforcing the trope that touch-typing will be a must have skill in the dystopian future. It might not be the most ergonomic device we’ve ever seen, but the fact it looks like something out of a Neal Stephenson novel more than makes up for it in our book.

This is not the first time we’ve seen Wi-Fi SSIDs used as a method of communication, thanks largely to how easy the ESP8266 makes it. For his part, [::vtol::] has previously experimented with using them to culturally enrich the masses.

If your Arduino runs out of I/O lines, you can always add one of the several I/O expander chips that takes a serial interface to set its several pins. Or perhaps you could buy something like an Arduino Mega, with its extra sockets to fulfil your needs. But what would you do if you really needed more pins, say a thousand of them? Perhaps [Brian Lough] has the answer. OK, full disclosure: If you really need a thousand, the video isn’t exactly for you, as he shows you how to add up to 992 PWM outputs. The chip he uses works with any microcontroller (the video shows an ESP8266), and we suppose you could use two daisy chains of them and break the 1,000 barrier handily.

We like how short the video is (just two minutes; see below) as it gets right to the point. The PCA9685 chip gives you 16 12-bit PWM channels via an I2C interface. You can daisy chain up to 62 of the boards to get the 992 outputs promised.

[Brian] uses a cheap $2 breakout board that lets you set a 6-bit address, has a nice power connector and makes it easy to use the little surface mount device. Each of the 16 outputs on the board can have an independent duty cycle, but they do share a single output frequency. That means if you want to use some channels for low-frequency devices like motors and some for high-frequency devices like LEDs, you might have to spring $4 for two boards.

Over on Hackaday.io, we’ve seen these devices driving 128 vibration motors. The PCA9685 made us think of the time we rolled our own serial to PWM devices using an FPGA.

There’s something about impressing strangers on the Internet that brings out the best in us. Honestly, we wouldn’t be able to run this site otherwise. A perfect example of this phenomenon is the annual Reddit Secret Santa, where users are challenged to come up with thoughtful gifts for somebody they’ve never even met before.

For his entry into this yearly demonstration of creativity, [Harrison Pace] wanted to do something that showcased his improving electronic skills while also providing something entertaining to the recipient. So he came up with a box of goodies which is unlocked by the successful completion of a built-in trivia game tailored around their interests. If this is how he treats strangers, we can’t wait to see what he does for his friends.

Hardware packed into the lid so the box itself remains empty.

There’s quite a bit of hardware hidden under the hood of this bedazzled gift box. The primary functions of the box are handled by an Arduino Nano; which runs the trivia game and provides user interaction via a 16×2 LCD, three push buttons, and a buzzer. Once the trivia game is complete, a servo is used to unlock the box and allow the recipient access to the physical gifts.

But that’s not the only trick this box has hidden inside. Once the main trivia game is complete, a ESP8266 kicks into action and advertises an access point the user can connect to. This starts the second level of challenges and gifts, which includes a code breaking challenge and gifted software licenses.

The project wasn’t all smooth sailing though. [Harrison] admits that his skills are still developing, and there were a few lessons learned during this project he is unlikely to forget in the future. Some Magic Smoke managed to escape when he connected his 5V Arduino directly to the 3.3V ESP8266, but at least it was a fairly cheap mistake and he had spares on hand to get the project completed anyway.

This project is reminiscent of reverse geocache boxes which only open when moved to a certain location, but the trivia aspect makes it perfect even for those of us who don’t want to put pants on just to receive our Internet gifts.


Filed under: Arduino Hacks, Holiday Hacks

One of the tasks I dread is configuring a web server to send email correctly via Gmail. The simplest way of sending emails is SMTP, and there are a number of scripts out there that provide a simple method to send mail that way with a minimum of configuration. There’s even PHP mail(), although it’s less than reliable.

Out of the box, Gmail requires OAUTH2 for authentication and to share user data, which has the major advantage of not requiring that you store your username and password in the application that requires access to your account. While they have an ‘allow less secure apps’ option that allows SMTP access for legacy products like Microsoft Outlook, it just doesn’t seem like the right way forward. Google documents how to interact with their API with OAUTH2, so why not just use that instead of putting my username and password in plaintext in a bunch of prototypes and test scripts?

Those are the thoughts that run through my head every time this comes up for a project, and each time I’ve somehow forgotten the steps to do it, also forgotten to write it down, and end up wasting quite a bit of time due to my own foolishness. As penance, I’ve decided to document the process and share it with all of you, and then also make it work on an ESP8266 board running the Arduino development environment.

Before we continue, now would be a good time for a non-technical refresher on how OAUTH works. The main differences between OAUTH and OAUTH2 are that the latter requires HTTPS, and the access tokens that allow an application to use specific services in a user account have an expiry.

To use Gmail with OAUTH2, we will need to start with five things: An application registered in the Google APIs, its client ID and client secret, a computer running LAMP (a by-the-hour VPS works just fine here), and a domain name that points to it.

Registering an application with Google API is easy. Go to the Google API console, log in, create a new project, and enter it. Enable the Gmail API; it should be suggested on the front page.

With the project created and the Gmail API enabled, the dashboard should look something like this

Then click on ‘credentials’ on the sidebar, create credentials, and finally ‘create OAUTH Client ID’. Before you can continue, you need to create a consent screen. The only entry you really need to fill out at this time is ‘Product Name Shown to Users’.

After saving that form, select ‘Web Application’ as your application type. Note the field called ‘Authorized redirect URIs’, we’ll return to it later. It’s important that it be correctly set for us to be able to receive a refresh token later on in this process.

For now, just press ‘Create’. A pop-up will display containing your Client ID and Client secret. You’ll need them soon, so best to copy/paste them into a local file on your computer for now.

Next, we will use those two pieces of data to request an access token and refresh token. We may as well accomplish two things at the same time here by installing the popular PHP email sender called PHPMailer on our web server. It includes a tool to request an OAUTH2 access/refresh token as well as being easily capable of sending a quick test email. To install it, we’ll use the Composer PHP dependency management tool:

$sudo apt-get install composer

Then we should navigate to our web-accessible directory, in my case /var/www/html, and install a few PHP scripts. Note that this should not be done as root, so create another user if needed and give them access to the directory:

$composer require phpmailer/phpmailer
$composer require league/oauth2-client
$composer require league/oauth2-google

Now enter the directory vendor/phpmailer/phpmailer. There will be a script called get_oauth_token.php. Move this script up three directories into the directory you just ran the ‘composer’ commands from. The location of this script as seen from the web needs to be entered into the ‘Authorized redirect URIs’ field of the Google API that we saw earlier. In this case it would have been https://mydomain.com/get_oauth_token.php. Public IP addresses will not work, this is why a domain name pointed to your web server is a requirement.

Now, open get_oauth_token.php in a text editor and paste in your Client ID and Client Secret where needed. Don’t try to run the script locally, it will fail. Open up a web browser on any computer, and navigate to the URL you entered as the ‘Authorized redirect URI’. Then select Google from the list of email services – at this point if it worked you will be asked to log in and then authorize the unverified application, under ‘Advanced’ under the warning prompt, at which point you will finally receive a refresh token. If you only want an access token for some reason you’ll have to edit the script to echo it back.

If that didn’t work, there are two common reasons: a wrong redirect URI or the script cannot find its dependencies. In the former case, the error message from Google will tell you the script URL as it sees it, and you can use that information to update the redirect URI in the Google API Console to fix the issue. For the latter, check your apache error log, probably located in /var/log/apache2/error.log, to see what dependency is not being found. You might see something like this:

PHP Warning: require(vendor/autoload.php): failed to open stream: No such file or directory in /var/www/html/mydomain/get_oauth_token.php on line 59, referer: http://mydomain.com/get_oauth_token.php

If you have received your refresh token, congratulations: the painful part is over. You can just go to the PHPMailer Github page and fill out their OAUTH2 example (gmail_xoauth.phps), and it ought to just work. If all you needed to do is send mail from a project on your VPS, you’re more or less ready to move on to more interesting parts of your project:

$email = 'someone@gmail.com';
$clientId = 'RANDOMCHARS-----duv1n2.apps.googleusercontent.com';
$clientSecret = 'RANDOMCHARS-----lGyjPcRtvP';
//Obtained by configuring and running get_oauth_token.php
//after setting up an app in Google Developer Console.
$refreshToken = 'RANDOMCHARS-----DWxgOvPT003r-yFUV49TQYag7_Aod7y0';

Remember to clean up any unnecessary scripts that contain your refresh token and other sensitive data before continuing.

ESP8266: We Don’t Need No Stinking Servers

Now what if we wanted to use these tokens to send email directly from project on a Raspberry Pi without needing a server in the middle? It turns out that once we have the client ID, client secret, and refresh token, we no longer require the server and domain name we’ve been using so far, and a mail-sending application, e.g. PHPMailer, can be installed on a computer anywhere with Internet access as long as it is configured with those values.

Things get a little more complicated when we try to do this on an ESP8266. OAUTH2 requires that we use SSL, and access tokens regularly expire and need to be refreshed. Thankfully, [jalmeroth] generously wrote a proof-of-concept and published it on GitHub. If provided with an access token, it can access your Gmail account and use it to send an email. It can also directly update/get data from Google Sheets, but I didn’t test this. However, if the access token was expired, it couldn’t detect that, although it did include working code to actually request a new token, but not parse it out and use it.

In an attempt to add to the functionality of that proof of concept, I forked the project and made a few changes. First, I changed to order of operations in the code to make it check if the current access token was valid before doing anything else. Second, Google API was responding ‘400 Bad Request’ if the access token was invalid, and everything but ‘200 OK’ responses were being filtered out by the code. Finally, I wrote a couple of JSON parsers that check the reason for the ‘400 Bad Request’ and extract and use the access token returned by Google API when a new one is requested.

It works, but it’s hardly reliable – not surprising considering I’ve never really used the Arduino platform before. Notably, the SHA1 fingerprint for Google API fails often. Checking from my local machine, the SHA1 fingerprint varies between two signatures there too. It would be fairly easy to check for either of them, or just keep trying, but I’d rather understand what’s going on first. (Is it just a CDN or something else?) Or perhaps I should rewrite the whole application in Lua where I’m more competent.

A fun little application built on the above was to place a button on my office that sends an email to my phone. I don’t want people to contact me at that email address frivolously, but do want to know immediately if someone is waiting outside my office. The big red button is for normal requests, but urgent requests require lockpicking. If it’s urgent it better also be interesting.

Finally, did you know that Hackaday provides an API for accessing hackaday.io? It uses the simpler OAUTH (not OAUTH2) authentication, so should be more straightforward than the above to implement on the ESP8266. Have any of you used it?


Filed under: Arduino Hacks, google hacks, how-to, Original Art


  • Newsletter

    Sign up for the PlanetArduino Newsletter, which delivers the most popular articles via e-mail to your inbox every week. Just fill in the information below and submit.

  • Like Us on Facebook