Posts | Comments

Planet Arduino

Archive for the ‘Edge Impulse’ Category

As announced at CES 2023 in Las Vegas, our tiny form factor family keeps growing: the 22.86 x 22.86 mm Nicla range now includes Nicla Voice, allowing for easy implementation of always-on speech recognition on the edge.

How? Let’s break it down.

  1. The impressive sensor package. Nicla Voice comes with a full set of sensors: microphone, smart 6-axis motion sensor and magnetometer – so it can not only listen to you, your machines, the environment around it, but also recognize gestures, vibrations and other movements. 
  2. The high-performance AI brains. Nicla Voice runs audio inputs through the powerful Syntiant NDP120 Neural Decision processor, which mimics human neural pathways to run multiple AI algorithms and automate complex tasks. In other words, it hears different events and keywords simultaneously, and is capable of understanding and learning what sounds mean.
  3. The easy connectivity features. It connects to existing devices thanks to onboard Bluetooth® Low Energy connectivity. 
  4. The effortless integration with custom boards. Thanks to its headers and castellated pins, Nicla Voice is ready to go from prototype to industrial-scale production, fitting right into any custom carrier board you develop.
  5. The Edge Impulse compatibility. In line with our mission to make complex technologies accessible to all, Nicla Voice is compatible with Edge Impulse, the leading development platform for machine learning on edge devices.
  6. The minimal power needs. And last but absolutely not least, it is so ultra-low power it can be the brain of always-on – and even battery-operated – solutions. No need to run dedicated power lines, no switches or interfaces to activate the system. It’s ready to listen, 24/7, anywhere you want to install it.

Speechless? We’re sure you’ll find your voice soon. With Nicla Voice’s ready-to-use combination of sensors and processing power, you can prototype and develop new solutions that leverage voice detection and voice recognition, or interpret any other audio input – from machines that need maintenance to water dripping, and from glass breaking to alarms that must get through headphones’ noise-canceling features. We can’t wait to hear what you’ll create with it!

Need to hear a pin drop? Nicla Voice is all ears. 

To find out more, access our free online documentation or check out the technical details from the Arduino Store page.

The post Have you heard? Nicla Voice is out at CES 2023! appeared first on Arduino Blog.

Going for a hike outdoors is a great way to relieve stress, do some exercise, and get closer to nature, but tracking them can be a challenge. Our recent collaboration with K-Way led Zalmotek to develop a small wearable device that can be paired to a weather-resistant jacket to track walking speed, steps taken, and even the current atmospheric conditions.

At its core, the tracker can be split into having three main functions: weather prediction, step/climbing activity, and a way to gather and send raw data over Bluetooth® Low Energy to the Arduino IoT Cloud for additional processing and training machine learning models. Performing these tasks is a Nicla Sense ME board, which contains an advanced six-axis BHI260AP IMU, a three-axis magnetometer, a pressure sensor, and a BME688 four-in-one gas sensor with temperature and humidity capabilities.

Zalmotek first collected data samples using the Edge Impulse Studio from the barometer ranging from rising to falling air pressure, as they predict clear or stormy conditions, respectively. Once finished, a classification model was trained and deployed to the Nicla Sense, where the LEDs could indicate which weather pattern is more likely. The activity tracking model, however, was trained using data collected from the IMU and labeled with either walking, climbing, or staying. After integrating them both into a single sketch, Zalmotek created an Arduino IoT Cloud dashboard for displaying these values in real-time.

For a deeper dive into the device, read Edge Impulse’s blog post. You can also discover more about the Arduino x K-Way project here.

The post Turning a K-Way jacket into an intelligent hike tracker with the Nicla Sense ME appeared first on Arduino Blog.

Wevolver’s previous article about the Arduino Pro ecosystem outlined how embedded sensors play a key role in transforming machines and automation devices to Cyber Physical Production Systems (CPPS). Using CPPS systems, manufacturers and automation solution providers capture data from the shop floor and use it for optimizations in areas like production schedules, process control, and quality management. These optimizations leverage advanced data Internet of Things (IoT) analytics over manufacturing datasets, which is the reason why data are the new oil.

Deployment Options for IoT Analytics: From Cloud Analytics to TinyML

IoT analytics entail statistical data processing and employ Machine Learning (ML) functions, including Deep Learning (DL) techniques i.e., ML based on deep neural networks. Many manufacturing enterprises deploy IoT analytics in the cloud. Cloud IoT analytics use the vast amounts of cloud data to train accurate DL models. Accuracy is important for many industrial use cases like Remaining Useful Life calculation in predictive maintenance. Nevertheless, it is also possible to execute analytics at the edge of the network. Edge analytics are deployed within embedded devices or edge computing clusters at the factory’s Local Area Network (LAN). They are appropriate for real-time use cases that demand low latency such as real-time detection of defects. Edge analytics are more power-efficient than cloud analytics. Moreover, they offer increased data protection as data stays within the LAN.

During the last couple of years, industrial organizations use TinyML to execute ML models within CPU and memory-constrained devices. TinyML is faster, real-time, more power-efficient, and more privacy-friendly than any other form of edge analytics. Therefore, it provides benefits for many Industry 4.0 use cases.

TinyML is the faster, real-time, most power-efficient, and most privacy friendly form of edge analytics. Image credit: Carbon Robotics.

Building TinyML Applications

The process of developing and deploying TinyML applications entails:

  1. Getting or Producing a Dataset, which is used for training the TinyML model. In this direction, data from sensors or production logs can be used.
  2. Train an ML or DL Model, using standard tools and libraries like Jupyter Notebooks and Python packages like TensorFlow and NumPy. The work entails Exploratory Data Analysis steps towards understanding the data, identifying proper ML models, and preparing the data for training them.
  3. Evaluate the Model’s Performance, using the trained model predictions and calculating various error metrics Depending on the achieved performance, the TinyML engineer may have to improve the model and avoid overfitting on the data. Different models must be tested to find the best one.
  4. Make the Model Appropriate to Run on an Embedded Device, using tools like TensorFlow Lite which provides a “converter” library that turns a model into a space-efficient format. TensorFlow Lite provides also an “interpreter” library that runs the converted model using the most efficient operations for a given device. In this step, a C/C++ sketch is produced to enable on device deployment.
  5. On-device Inference and Binary Development, which involves the C/C++ and embedded systems development part and produces a binary application for on-device inference.
  6. Deploying the Binary to a Microcontroller, which makes the microcontroller able to analyse data and derive real-time insights.
Building a Google Assistant using tinyML. Image credit: Arduino.

Leveraging AutoML for Faster Development with Arduino Pro

Nowadays, Automatic Machine Learning (AutoML) tools are used to develop TinyML on various boards, including Arduino boards. Emerging platforms such as Edge Impulse, Qeexo and SensiML, among others, provide AutoML tools and developers’ resources for embedded ML development. Arduino is collaborating with such platforms as part of their strategy to make complex technologies open and simple to use by anyone.

Within these platforms, users collect real-world sensor data, train ML models on the cloud, and ultimately deploy the model back to an Arduino device. It is also possible to integrate ML models with Arduino sketches based on simple function calls. AutoML pipelines ease the tasks of (re)developing and (re)deploying models to meet complex requirements.

The collaboration between Arduino and ML platforms enables thousands of developers to build applications that embed intelligence in smart devices such as applications that recognize spoken keywords, gestures, and animals. Implementing applications that control IoT devices via natural language or gestures is relatively straightforward for developers who are familiar with Arduino boards.

Arduino has recently introduced its new Arduino Pro ecosystem of industrial-grade products and services, which support the full development, production and operation lifecycle from Hardware and Firmware to Low Code, Clouds, and Mobile Apps. The Pro ecosystem empowers thousands of developers to jump into Industry 4.0 development and to employ advanced edge analytics.

Big opportunity at every scale

The Arduino ecosystem provides excellent support for TinyML, including boards that ease TinyML development, as well as relevant tools and documentation. For instance, the Arduino Nano 33 BLE Sense board is one of the most popular boards for TinyML. It comes with a well-known form factor and various embedded sensors. The latter include a 9-axis inertial sensor that makes the board ideal for wearable devices, as well as for humidity and temperature sensors. As another example, Arduino’s Portenta H7 board includes two asymmetric cores, which enables simultaneously runs of high level code such as protocol stacks, machine learning or even interpreted languages (e.g., MicroPython or JavaScript). Furthermore, the Arduino IDE (Integrated Development Environment) provides the means for customizing embedded ML pipelines and deploying them in Arduino boards.

In a Nutshell

ML and AI models need not always to run over powerful clouds and related High Performance Computing services. It is also possible to execute neural networks over tiny memory-limited devices like microcontrollers, which opens unprecedented opportunities for pervasive intelligence. The Arduino ecosystem offers developers the resources they need to ride the wave of Industry 4.0 and TinyML. Arduino boards and the IDE lower the barriers for thousands of developers to engage with IoT analytics for industrial intelligence.

Read the full article on Wevolver.

The post From Embedded Sensors to Advanced Intelligence: Driving Industry 4.0 Innovation with TinyML appeared first on Arduino Blog.

After learning about the basics of embedded ML, industrial designer and educator Phil Caridi had the idea to build a metal detector, but rather than using a coil of wire to sense eddy currents, his device would use a microphone to determine if metal music is playing nearby. 

Caridi started out by collecting around two hours of music and then dividing the samples into two labels: “metal” and “non_metal” using Edge Impulse. After that, he began the process of training a neural network after passing each sample through an MFE filter. The end result was a model capable of detecting if a given piece of music is either metal or non-metal with around 88.2% accuracy. This model was then deployed onto a Nano 33 BLE Sense, which tells the program what kind of music is playing, but Caridi wasn’t done yet. He also 3D-printed a mount and gauge that turns a needle further to the right via a servo motor as the confidence of “metal music” increases.

As seen in his video, the device successfully shows the difference between the band Death’s “Story to Tell” track and the much tamer and non-metal song “Oops!… I Did It Again” by Britney Spears. For more details about this project, you can read Caridi’s blog post.

The post Instead of sensing the presence of metal, this tinyML device detects rock (music) appeared first on Arduino Blog.

Although smartphone users have had the ability to quickly translate spoken words into nearly any modern language for years now, this feat has been quite tough to accomplish on small, memory-constrained microcontrollers. In response to this challenge, Hackster.io user Enzo decided to create a proof-of-concept project that demonstrated how an embedded device can determine the language currently being spoken without the need for an Internet connection. 

This so-called “language detector” is based on an Arduino Nano 33 BLE Sense, which is connected to a common PCA9685 motor driver that is, in turn, attached to a set of three micro servo motors — all powered by a single 9V battery. Enzo created a dataset by recording three words: “oui” (French), “si” (Italian), and “yes” (English) for around 10 minutes each for a total of 30 minutes of sound files. He also added three minutes of random background noise to help distinguish between the target keywords and non-important words. 

Once a model had been trained using Edge Impulse, Enzo exported it back onto his Nano 33 BLE Sense and wrote a small bit of code that reads audio from the microphone, classifies it, and determines which word is being spoken. Based on the result, the corresponding nation’s flag is raised to indicate the language.

You can see the project in action below and read more about it here on Hackster.io.

The post This Arduino device can detect which language is being spoken using tinyML appeared first on Arduino Blog.

Machine learning is an incredible tool for conservation research, especially for scenarios like long term observation, and sifting through massive amounts of data. While the average Hackaday reader might not be able to take part in data gathering in an isolated wilderness somewhere, we are all surrounded by bird life. Using an Arduino Nano 33 BLE Sense and an online machine learning tool, [Errol Joshua] demonstrates how to set up an automated bird call classifier.

The Arduino Nano 33 BLE Sense  is a fully featured little dev board that features the very capable NRF52840 microcontroller with Bluetooth Low Energy, and a variety of onboard sensors, including a microphone. Training a machine learning model might seem daunting to many people, but online services like Edge Impulse makes the process very beginner-friendly. Once you start training your own models for specific applications, you quickly learn that building and maintaining a high quality dataset is often the most time-consuming part of machine learning. Fortunately for this use case, a massive online library of bird calls from all over the world is available on Xeno-Canto. This can be augmented with background noise from the area where the device will be deployed to reduce false-positives. Edge Impulse will train the model using the provided dataset, and generate a library that can be used on the Arduino with one of the provided sample sketches to log and send the collected data to a server. Then comes the never ending process of iteratively testing and improving the recognition model. Edge Impulse is also compatible with more powerful devices such as the Raspberry Pi and Jetson Nano if you want more intensive machine learning models.

We’ve also seen the exact same setup get used for smart baby monitor. If you want to learn more, be sure to watch at [Shawn Hymel]’s talk from the 2020 Remoticon about machine learning on microcontrollers.

Header photo by Joey Smith on Unsplash

In light of the ongoing COVID-19 pandemic, being able to quickly determine a person’s current health status is very important. This is why Manivannan S wanted to build his very own COVID Patient Health Assessment Device that could take several data points from various vitals and make a prediction about what they indicate. The pocket-sized system features a Nano 33 BLE Sense at its core, along with a Maxim Integrated MAX30102 pulse oximeter/heart-rate sensor to measure oxygen saturation and pulse. 

From this incoming health data, Manivannan developed a simple algorithm that generates a “Health Index” score by plugging in factors such as SpO2, respiration rate, heart rate, and temperature into a linear regression. Once some sample data was created, he sent it to Edge Impulse and trained a model that uses a series of health indices to come up with a plausible patient condition. 

After deploying the model to the Nano 33 BLE Sense, Manivannan put some test data on it to simulate a patient’s vital signs and see the resulting inferences. As expected, his model successfully identified each one and displayed it on an OLED screen. To read more about how this device works, plus a few potential upgrades, you can visit its write-up on Hackster.io here or check out the accompanying video below.

The post This pocket-sized uses tinyML to analyze a COVID-19 patient’s health conditions appeared first on Arduino Blog.

In light of the ongoing COVID-19 pandemic, being able to quickly determine a person’s current health status is very important. This is why Manivannan S wanted to build his very own COVID Patient Health Assessment Device that could take several data points from various vitals and make a prediction about what they indicate. The pocket-sized system features a Nano 33 BLE Sense at its core, along with a Maxim Integrated MAX30102 pulse oximeter/heart-rate sensor to measure oxygen saturation and pulse. 

From this incoming health data, Manivannan developed a simple algorithm that generates a “Health Index” score by plugging in factors such as SpO2, respiration rate, heart rate, and temperature into a linear regression. Once some sample data was created, he sent it to Edge Impulse and trained a model that uses a series of health indices to come up with a plausible patient condition. 

After deploying the model to the Nano 33 BLE Sense, Manivannan put some test data on it to simulate a patient’s vital signs and see the resulting inferences. As expected, his model successfully identified each one and displayed it on an OLED screen. To read more about how this device works, plus a few potential upgrades, you can visit its write-up on Hackster.io here or check out the accompanying video below.

The post This pocket-sized uses tinyML to analyze a COVID-19 patient’s health conditions appeared first on Arduino Blog.

Baby monitors are cool, but [Ish Ot Jr.] wanted his to only transmit sounds that required immediate attention and filter any non-emergency background noise. Posed with this problem, he made a baby monitor that would only send alerts when his baby was crying.

For his project, [Ish] used an Arduino Nano 33 BLE Sense due to its built-in microphone, sizeable RAM for storing large chunks of data, and it’s BLE capabilities for later connecting with an app. He began his project by collecting background noise using Edge Impulse Studio’s data acquisition functionality. [Ish] really emphasized that Edge Impulse was really doing all the work for him. He really just needed to collect some test data and that was mostly it on his part. The work needed to run and test the Neural Network was taken care of by Edge Impulse. Sounds handy, if you don’t mind offloading your data to the cloud.

[Ish] ended up with an 86.3% accurate classifier which he thought was good enough for a first pass at things. To make his prototype a bit more “finished”, he added some status LEDs, providing some immediate visual feedback of his classifier and to notify the caregiver. Eventually, he wants to add some BLE support and push notifications, alerting him whenever his baby needs attention.

We’ve seen a couple of baby monitor projects on Hackaday over the years. [Ish’s] project will most certainly be a nice addition to the list.

This post is written by Jan Jongboom and Dominic Pajak.

Running machine learning (ML) on microcontrollers is one of the most exciting developments of the past years, allowing small battery-powered devices to detect complex motions, recognize sounds, or find anomalies in sensor data. To make building and deploying these models accessible to every embedded developer we’re launching first-class support for the Arduino Nano 33 BLE Sense and other 32-bit Arduino boards in Edge Impulse.

The trend to run ML on microcontrollers is called Embedded ML or Tiny ML. It means devices can make smart decisions without needing to send data to the cloud – great from an efficiency and privacy perspective. Even powerful deep learning models (based on artificial neural networks) are now reaching microcontrollers. This past year great strides were made in making deep learning models smaller, faster and runnable on embedded hardware through projects like TensorFlow Lite Micro, uTensor and Arm’s CMSIS-NN; but building a quality dataset, extracting the right features, training and deploying these models is still complicated.

Using Edge Impulse you can now quickly collect real-world sensor data, train ML models on this data in the cloud, and then deploy the model back to your Arduino device. From there you can integrate the model into your Arduino sketches with a single function call. Your sensors are then a whole lot smarter, being able to make sense of complex events in the real world. The built-in examples allow you to collect data from the accelerometer and the microphone, but it’s easy to integrate other sensors with a few lines of code. 

Excited? This is how you build your first deep learning model with the Arduino Nano 33 BLE Sense (there’s also a video tutorial here: setting up the Arduino Nano 33 BLE Sense with Edge Impulse):

  • Download the Arduino Nano 33 BLE Sense firmware — this is a special firmware package (source code) that contains all code to quickly gather data from its sensors. Launch the flash script for your platform to flash the firmware.
  • Launch the Edge Impulse daemon to connect your board to Edge Impulse. Open a terminal or command prompt and run:
$ npm install edge-impulse-cli -g
$ edge-impulse-daemon
  • Your device now shows in the Edge Impulse studio on the Devices tab, ready for you to collect some data and build a model.
  • Once you’re done you can deploy your model back to the Arduino Nano 33 BLE Sense. Either as a binary which includes your full ML model, or as an Arduino library which you can integrate in any sketch.
Deploy to Arduino from Edge Impulse
Deploying to Arduino from Edge Impulse
  • Your machine learning model is now running on the Arduino board. Open the serial monitor and run `AT+RUNIMPULSE` to start classifying real world data!
Keyword spotting on the Arduino Nano 33 BLE Sense
Keyword spotting on the Arduino Nano 33 BLE Sense

Integrates with your favorite Arduino platform

We’ve launched with the Arduino Nano 33 BLE Sense, but you can also integrate Edge Impulse with your favourite Arduino platform. You can easily collect data from any sensor and development board using the Data forwarder. This is a small application that reads data over serial and sends it to Edge Impulse. All you need is a few lines of code in your sketch (here’s an example).

After you’ve built a model you can easily export your model as an Arduino library. This library will run on any Arm-based Arduino platform including the Arduino MKR family or Arduino Nano 33 IoT, providing it has enough RAM to run your model. You can now include your ML model in any Arduino sketch with just a few lines of code. After you’ve added the library to the Arduino IDE you can find an example on integrating the model under Files > Examples > Your project – Edge Impulse > static_buffer.

To run your models as fast and energy-efficiently as possible we automatically leverage the hardware capabilities of your Arduino board – for example the signal processing extensions available on the Arm Cortex-M4 based Arduino Nano BLE Sense or more powerful Arm Cortex-M7 based Arduino Portenta H7. We also leverage the optimized neural network kernels that Arm provides in CMSIS-NN.

A path to production

This release is the first step in a really exciting collaboration. We believe that many embedded applications can benefit from ML today, whether it’s for predictive maintenance (‘this machine is starting to behave abnormally’), to help with worker safety (‘fall detected’), or in health care (‘detected early signs of a potential infection’). Using Edge Impulse with the Arduino MKR family you can already quickly deploy simple ML based applications combined with LoRa, NB-IoT cellular, or WiFi connectivity. Over the next months we’ll also add integrations for the Arduino Portenta H7 on Edge Impulse, making higher performance industrial applications possible.

On a related note: if you have ideas on how TinyML can help to slow down or detect the COVID-19 virus, then join the UNDP COVID-19 Detect and Protect Challenge. For inspiration, see Kartik Thakore’s blog post on cough detection with the Arduino Nano 33 BLE Sense and Edge Impulse.

We can’t wait to see what you’ll build!

Jan Jongboom is the CTO and co-founder of Edge Impulse. He built his first IoT projects using the Arduino Starter Kit.

Dominic Pajak is VP Business Development at Arduino.



  • Newsletter

    Sign up for the PlanetArduino Newsletter, which delivers the most popular articles via e-mail to your inbox every week. Just fill in the information below and submit.

  • Like Us on Facebook