Due to an ever-warming planet thanks to climate change and greatly increasing wildfire chances because of prolonged droughts, being able to quickly detect when a fire has broken out is vital for responding while it’s still in a containable stage. But one major hurdle to collecting machine learning model datasets on these types of events is that they can be quite sporadic. In his proof of concept system, engineer Shakhizat Nurgaliyev shows how he leveraged NVIDIA Omniverse Replicator to create an entirely generated dataset and then deploy a model trained on that data to an Arduino Nicla Vision board.
The project started out as a simple fire animation inside of Omniverse which was soon followed by a Python script that produces a pair of virtual cameras and randomizes the ground plane before capturing images. Once enough had been created, Nurgaliyev utilized the zero-shot object detection application Grounding DINO to automatically draw bounding boxes around the virtual flames. Lastly, each image was brought into an Edge Impulse project and used to develop a FOMO-based object detection model.
By taking this approach, the model achieved an F1 score of nearly 87% while also only needing a max of 239KB of RAM and a mere 56KB of flash storage. Once deployed as an OpenMV library, Nurgaliyev shows in his video below how the MicroPython sketch running on a Nicla Vision within the OpenMV IDE detects and bounds flames. More information about this system can be found here on Hackster.io.
Based on the Renesas RA4M1 microcontroller, the new Arduino UNO R4 boasts 16x the RAM, 8x the flash, and a much faster CPU compared to the previous UNO R3. This means that unlike its predecessor, the R4 is capable of running machine learning at the edge to perform inferencing of incoming data. With this fact in mind, Roni Bandini wanted to leverage his UNO R4 Minima by training a model to predict the likelihood of a FIFA team winning their match.
Bandini began his project by first downloading a dataset containing historical FIFA matches, including the country, team, opposing team, ranking, and neutral location. Next, the data was added to Edge impulse as a time-series dataset which feeds into a Keras classifier ML block and produces “win” and “lose/draw” values. Once trained, the model achieved an accuracy of 69% with a loss value of 0.58.
Inputting the desired country and rank to make a prediction is done by making selections on a DFRobot LCD shield, and these values are then used to populate the input tensor for the model before it gets invoked and returns its classification results. Bandini’s device demonstrates how much more powerful the Arduino UNO R4 is over the R3, and additional information on the project can be found here in his post.
Shortly after setting the desired temperature of a room, a building’s HVAC system will engage and work to either raise or lower the ambient temperature to match. While this approach generally works well to control the local environment, the strategy also leads to tremendous wastes of energy since it is unable to easily adapt to changes in occupancy or activity. In contrast, Jallson Suryo’s smart HVAC project aims to tailor the amount of cooling to each zone individually by leveraging computer vision to track certain metrics.
Suryo developed his proof of concept as a 1:50 scale model of a plausible office space, complete with four separate rooms and a plethora of human figurines. Employing Edge Impulse and a smartphone, 79 images were captured and had bounding boxes drawn around each person for use in a FOMO-based object detection model. After training, Suryo deployed the OpenMV firmware onto an Arduino Nicla Vision board and was able to view detections in real-time.
The last step involved building an Arduino library containing the model and integrating it into a sketch that communicates with an Arduino Nano peripheral board over I2C by relaying the number of people per quadrant. Based on this data, the Nano dynamically adjusts one of four 5V DC fans to adjust the temperature while displaying relevant information on an OLED screen. To see how this POC works in more detail, you can visit Suryo’s write-up on the Edge Impulse docs page.
On June 26th-28th, the Arduino Pro team will be in Amsterdam for the tinyML EMEA Innovation Forum – one of the year’s major events for the world where AI models meet agile, low-power devices.
This is an exciting time for companies like Arduino and anyone interested in accelerating the adoption of tiny machine learning: technologies, products, and ideas are converging into a worldwide phenomenon with incredible potential – and countless applications already.
At the summit, our team will indeed present a selection of demos that leverage tinyML to create useful solutions in a variety of industries and contexts. For example, we will present:
A fan anomaly detection system based on the Nicla Sense ME. In this solution developed with SensiML, the Nicla module leverages its integrated accelerometer to constantly measure the vibrations generated by a computer fan. Thanks to a trained model, condition monitoring turns into anomaly detection – the system is able to determine whether the fan is on or off, notify users of any shocks, and even alert them if its super precise and efficient sensor detects sub-optimal airflow.
A vineyard pest monitoring system with the Nicla Vision and MKR WAN 1310. Machine vision works at the service of smart agriculture in this solution: even in the most remote field, a pheromone is used to attract insects inside a case lined with glue traps. The goal is not to capture all the insects, but to use a Nicla Vision module to take a snapshot of the captured bugs, recognize the ones that pose a real threat, and send updated data on how many specimens were found. New-generation farmers can thus schedule interventions against pests as soon as needed, before the insects get out of control and cause damage to the crops. Leveraging LoRa® connectivity, this application is both low-power and high-efficiency.
An energy monitoring-based anomaly detection solution for DC motors, with the Opta. This application developed with Edge Impulse leverages an Opta WiFi microPLC to easily implement industrial-level, real-time monitoring and fault detection – great to enable predictive maintenance, reducing downtime and overall costs. A Hall effect current sensor is attached in series with the supply line of the DC motor to acquire real-time data, which is then analyzed using ML algorithms to identify patterns and trends that might indicate faulty operation. The DC motor is expected to be in one of two statuses – ON or OFF – but different conditions can be simulated with the potentiometer. When unexpected electric consumption is shown, the Opta WiFi detects the anomaly and turns on a warning LED.
The Arduino Pro team is looking forward to meeting customers and partners in Amsterdam – championing open source, accessibility, and flexibility in industrial-grade solutions at the tinyML EMEA Innovation Forum!
On June 26th-28th, the Arduino Pro team will be in Amsterdam for the tinyML EMEA Innovation Forum – one of the year’s major events for the world where AI models meet agile, low-power devices.
This is an exciting time for companies like Arduino and anyone interested in accelerating the adoption of tiny machine learning: technologies, products, and ideas are converging into a worldwide phenomenon with incredible potential – and countless applications already.
At the summit, our team will indeed present a selection of demos that leverage tinyML to create useful solutions in a variety of industries and contexts. For example, we will present:
A fan anomaly detection system based on the Nicla Sense ME. In this solution developed with SensiML, the Nicla module leverages its integrated accelerometer to constantly measure the vibrations generated by a computer fan. Thanks to a trained model, condition monitoring turns into anomaly detection – the system is able to determine whether the fan is on or off, notify users of any shocks, and even alert them if its super precise and efficient sensor detects sub-optimal airflow.
A vineyard pest monitoring system with the Nicla Vision and MKR WAN 1310. Machine vision works at the service of smart agriculture in this solution: even in the most remote field, a pheromone is used to attract insects inside a case lined with glue traps. The goal is not to capture all the insects, but to use a Nicla Vision module to take a snapshot of the captured bugs, recognize the ones that pose a real threat, and send updated data on how many specimens were found. New-generation farmers can thus schedule interventions against pests as soon as needed, before the insects get out of control and cause damage to the crops. Leveraging LoRa® connectivity, this application is both low-power and high-efficiency.
An energy monitoring-based anomaly detection solution for DC motors, with the Opta. This application developed with Edge Impulse leverages an Opta WiFi microPLC to easily implement industrial-level, real-time monitoring and fault detection – great to enable predictive maintenance, reducing downtime and overall costs. A Hall effect current sensor is attached in series with the supply line of the DC motor to acquire real-time data, which is then analyzed using ML algorithms to identify patterns and trends that might indicate faulty operation. The DC motor is expected to be in one of two statuses – ON or OFF – but different conditions can be simulated with the potentiometer. When unexpected electric consumption is shown, the Opta WiFi detects the anomaly and turns on a warning LED.
The Arduino Pro team is looking forward to meeting customers and partners in Amsterdam – championing open source, accessibility, and flexibility in industrial-grade solutions at the tinyML EMEA Innovation Forum!
People with visual impairments also enjoy going out to a restaurant for a nice meal, which is why it is common for wait staff to place the salt and pepper shakes in a consistent fashion: salt on the right and pepper on the left. That helps visually impaired diners quickly find the spice they’re looking for and a similar arrangement works for utensils. But what about after the diner sets down a utensil in the middle of a meal? The ForkLocator is an AI system that can help them locate the utensil again.
This is a wearable device meant for people with visual impairments. It uses object recognition and haptic cues to help the user locate their fork. The current prototype, built by Revoxdyna, only works with forks. But it would be possible to expand the system to work with the full range of utensils. Haptic cues come from four servo motors, which prod the user’s arm to indicate the direction in which they should move their hand to find the fork.
The user’s smartphone performs the object recognition and should be worn or positioned in such a way that its camera faces the table. The smartphone app looks for the plate, the fork, and the user’s hand. It then calculates a vector from the hand to the fork and tells an Arduino board to actuate the servo motors corresponding to that direction. Those servos and the Arduino attach to a 3D-printed frame that straps to the user’s upper arm.
A lot more development is necessary before a system like the ForkLocator would be ready for the consumer market, but the accessibility benefits are something to applaud.
Pipelines are integral to our modern way of life, as they enable the fast transportation of water and energy between central providers and the eventual consumers of that resource. However, the presence of cracks from mechanical or corrosive stress can lead to leaks, and thus waste of product or even potentially dangerous situations. Although methods using thermal cameras or microphones exist, they’re hard to use interchangeably across different pipeline types, which is why Kutluhan Aktar instead went with a combination of mmWave radar and an ML model running on an Arduino Nicla Vision board to detect these issues.
The project was originally conceived as an arrangement of parts on a breadboard, including a Seeed Studio MR60BHA1 60GHz radar module, an ILI9341 TFT screen, an Arduino Nano for interfacing with the sensor and display, and a Nicla Vision board. From here, Kutluhan designed his own Dragonite-themed PCB, assembled the components, and began collecting training and testing data for a machine learning model by building a small PVC model, introducing various defects, and recording the differences in data from the mmWave sensor.
After configuring a time-series impulse, a classification model was trained with the help of Edge Impulse and then deployed to the Nicla Vision where it achieved an accuracy of 90% on real-world data. With the aid of the display, operators can tell the result of the classification immediately, as well as send the data to a custom web application.
With an array of onboard sensors, Bluetooth® Low Energy connectivity, and the ability to perform edge AI tasks thanks to its nRF52840 SoC, the Arduino Nano 33 BLE Sense is a great choice for a wide variety of embedded applications. Further demonstrating this point, a group of students from the Introduction to Embedded Deep Learning course at Carnegie Mellon University have published the culmination of their studies through 10 excellent projects that each use the Tiny Machine Learning Kit and Edge Impulse ML platform.
Wrist-based human activity recognition
Traditional human activity tracking has relied on the use of smartwatches and phones to recognize certain exercises based on IMU data. However, few have achieved both continuous and low-power operation, which is why Omkar Savkur, Nicholas Toldalagi, and Kevin Xie explored training an embedded model on combined accelerometer and microphone data to distinguish between handwashing, brushing one’s teeth, and idling. Their project continuously runs inferencing on incoming data and then displays the action on both a screen and via two LEDs.
Categorizing trash with sound
In some circumstances, such as smart cities or home recycling, knowing what types of materials are being thrown away can provide a valuable datapoint for waste management systems. Students Jacky Wang and Gordonson Yan created their project, called SBTrashCat, to recognize trash types by the sounds they make when being thrown into a bin. Currently, the model can three different kinds, along with background noise and human voices to eliminate false positives.
Distributed edge machine learning
The abundance of Internet of Things (IoT) devices has meant an explosion of computational power and the amount of data needing to be processed before it can become useful. Because a single low-cost edge device does not possess enough power on its own for some tasks, Jong-Ik Park, Chad Taylor, and Anudeep Bolimera have designed a system where each device runs its own “slice” of an embedded model in order to make better use of available resources.
Predictive maintenance for electric motors
Motors within an industrial setting require constant smooth and efficient operation in order to ensure consistent uptime, and recognizing when one is failing often necessitates manual inspection before a problem can be discovered. By taking advantage of deep learning techniques and an IMU/camera combination, Abhishek Basrithaya and Yuyang Xu developed a project that could accurately identify motor failure at the edge.
Estimating inventory in real-time with computer vision
Warehouses greatly rely on having up-to-date information about the locations of products, inventory counts, and incoming/outgoing items. From these constraints, Netra Trivedi, Rishi Pachipulusu, and Cathy Tungyun collaborated to gather a dataset of 221 images labeled with the percentage of space remaining on the shelf. This enables the Nano 33 BLE Sense to use an attached camera to calculate empty shelf space in real-time.
Dog movement tracking
Fitness trackers such as the FitBit and Apple Watch have revolutionized personal health tracking, but what about our pets? Ajith Potluri, Eion Tyacke, and Parker Crain addressed this hole in the market by building a dog collar that uses the Nano’s IMU to recognize daily activities and send the results to a smartphone via Bluetooth. This means the dog’s owner has the ability to get an overview of their pet’s day-to-day activity levels across weeks or months.
Intelligent bird feeding system
Owners of backyards everywhere encounter the same problem: “How do I keep the squirrels away from a birdfeeder while also allowing birds?” Eric Wu, Harry Rosmann, and Blaine Huey worked together on a Nano 33 BLE Sense-based system that employs a camera module to identify if the animal at the feeder is a bird or a squirrel. If it is the latter, an alarm is played from a buzzer. Otherwise, the bird’s species is determined through another model and an image is saved to an SD card for future viewing.
Improving one’s exercise form
Exercise, while being essential to a healthy lifestyle, must also be done correctly in order to avoid accidental injuries or chronic pain later on, and maintain proper form is an easy way to facilitate this. By using both computer vision on an NVIDIA Jetson Nano and anomaly detection via an IMU on a Nano 33 BLE Sense, Addesh Bhargava, Varun Jain, and Rohan Paranjape built a project that was more accurate than typical approaches to squatting form detection.
Optimizing manufacturing processes is a requirement in any industry today, with electricity consumption in particular representing a major concern due to increased costs and instability. Analyzing energy use has therefore become a widespread need – and one that can also lead to early identification of anomalies and predictive maintenance: two important activities to put in place in order to minimize unexpected downtime and repair costs.
In particular, this approach can be applied to DC motors: used in a wide range of applications, from small household appliances to heavy industrial equipment; these motors are critical components that require regular maintenance to ensure optimal performance and longevity. Unfortunately, traditional maintenance practices based on fixed schedules or reactive repairs can be time-consuming, expensive, and unreliable. This is where energy monitoring-based anomaly detection comes in: it can provide a crucial solution for the early detection of potential issues and malfunctions before they can cause significant damage to the motor.
This more proactive approach to maintenance continuously monitors the energy consumption of the motor and analyzes the data to identify any deviations from normal operating conditions. By tracking energy usage patterns over time, the system can detect early warning signs of potential problems, such as excessive wear and tear, imbalances or misalignments, and alert maintenance personnel to take corrective actions before the issue escalates.
Our solution
This Arduino-powered solution implements an energy monitoring-based anomaly detection system using a current sensor and machine learning models running on edge devices. By capturing the electricity flowing in and out of a machine, it can collect large amounts of data on energy usage patterns over time. This data is then used to train a machine learning model capable of identifying anomalies in energy consumption behaviors and alerting operators to potential issues. The solution offers a cost-effective and scalable method for maintaining equipment health and maximizing energy efficiency, while also reducing downtime and maintenance costs.
Motor Current Signature Analysis (MCSA)
In this application, a technique called Motor Current Signature Analysis is used. MCSA involves monitoring the electrical signature of the motor’s current overtime to detect any anomalies that may indicate potential issues or faults. To acquire real-time data, a Hall effect current sensor is attached in series with the supply line of the DC motor. The data are then analyzed using machine learning algorithms to identify patterns and trends that might indicate a faulty motor operation. MCSA can be used to detect a number of issues like bearings wear, rotor bar bendings or even inter-turn short circuits.
Depending on the dimensions of the motor, using a non-invasive clamp-style current sensor – also known as a Split-Core Current Transformer – is recommended if a larger current draw is expected.
Edge ML
To monitor the current fluctuation and run the anomaly-detecting ML model, the solution uses anArduino Opta WiFi: a micro PLC suitable for Industrial IoT, which is excellent for this project because of its real-time data classification capabilities, based on a powerful STM32H747XI dual-core Cortex®-M7 +M4 MCU. The Arduino Opta WiFi works with both analog and digital inputs and outputs, allowing it to interact with a multitude of sensors and actuators. The Arduino Opta WiFi also features an Ethernet port, an RS485 half duplex connectivity interface and WiFi/Bluetooth® Low Energy connectivity, which makes it ideal for industrial retrofitting applications. You can find the full datasheet here.
To train the anomaly detection model, the project leverages the Edge Impulse platform: being integrated within the Arduino ecosystem, it makes it easy to develop, train, and deploy machine learning models on Arduino devices.
Connectivity
Once the machine learning model was successfully deployed on the Arduino Opta, the anomaly detection results were forwarded via Wi-Fi to the Arduino IoT Cloud. This enables easy monitoring and analysis of the data from multiple sensor nodes in real time.
Solving it with Arduino Pro
Let’s take a look at how we can put all of this together and what hardware and software solutions we would need for deployment. The Arduino Pro ecosystem is the most recent version of Arduino solutions, offering users the benefits of easy integration along with a range of scalable, secure, and professionally supported services.
The Arduino IDE 2.0 was used to program the Arduino Opta WiFi using C/C++. To train the Edge Impulse model, data was gathered from the current sensor for two classes: Normal Operation and Machine Off. The Motor Current Signature Analysis (MCSA) technique was implemented by extracting the frequency and power characteristics of the signal through a Spectral Analysis block. Additionally, an anomaly detection block was incorporated to identify any abnormal patterns in the data.
Here is a screenshot from a dashboard created directly in the Arduino Cloud, showcasing data received from the sensor nodes:
Here is an overview of the software stack and how a minimum deployment with one of each hardware module communicates to fulfil the proposed solution:
Conclusion
Through the implementation of a predictive maintenance system on an Arduino Opta WiFi PLC, using Edge Impulse ML models and the Arduino Cloud, this solution demonstrates the powerful potential of IoT technologies in industrial applications. With the use of current sensors and AI-driven anomaly detection models, the system enables real-time monitoring and fault detection of DC motors, providing valuable insights for predictive maintenance. The flexibility and scalability of the Arduino Opta WiFi platform make it a robust and cost-effective solution for implementing predictive maintenance systems in various industrial processes. Overall, the project highlights the significant advantages that MCSA and machine learning can offer in promoting efficiency, productivity, and cost savings for industrial processes.
The task of gathering enough data to classify distinct sounds not captured in a larger, more robust dataset can be very time-consuming, at least until now. In his write-up, Nurgaliyev Shakhizat describes how he used an array of AI tools to automatically create a keyword spotting dataset without the need for speaking into a microphone.
The pipeline is split into three main parts. First, the Piper text-to-speech engine was downloaded and configured via a Python script to output 904 distinct samples of the TTS model saying Shakhizat’s last name in a variety of ways to decrease overfitting. Next, background noise prompts were generated with the help of ChatGPT and then fed into AudioLDM which produces the audio files based on the prompts. Finally, all of the WAV files, along with “unknown” sounds from the Google Speech Commands Dataset, were uploaded to an Arduino ML project.
Training the model for later deployment on a Nicla Voice board was accomplished by adding a Syntiant audio processing block and then generating features to train a classification model. The resulting model could accurately determine when the target word was spoken around 96% of the time — all without the need for manually gathering a dataset.
Planet Arduino is, or at the moment is wishing to become, an aggregation of public weblogs from around the world written by people who develop, play, think on Arduino platform and his son. The opinions expressed in those weblogs and hence this aggregation are those of the original authors. Entries on this page are owned by their authors. We do not edit, endorse or vouch for the contents of individual posts. For more information about Arduino please visit www.arduino.cc
You are currently browsing the archives for the machine learning category.
