Tiny Machine Learning (TinyML) 

Abate: What should pro engineers and serious makers know about TinyML? And can you share an example or two of an ultra-low-power machine learning application?

Situnayake: Our world is crowded with billions of sensors, in all sorts of places, from home appliances to smart factories and the vehicles we drive. All of this sensor data represents a golden opportunity for developers to build devices that understand the world around them and make intelligent decisions that help people out.
 
Historically, the only way to make use of this data was to send it across the Internet to a powerful computer able to crunch the numbers, interpret the data, and trigger actions based on the results. This works just fine for some applications. For example, it’s OK if data from a weather station takes a few seconds to get to a server. But there are many applications where this won’t work.
 
One of my favorite examples of this is the camera trap. Scientists use these to monitor animals in the wild, often in remote places that have no Internet connection available. When an animal walks by, it triggers the camera trap’s motion sensor, and a photograph is taken. Researchers will stop by periodically and pick up a memory card filled with animal photos.
 
The problem is, the motion sensor can be triggered by all sorts of things, like leaves moving in the wind, or species the researchers aren’t interested in. It takes a long time to sift through all the photos, and the memory card gets filled up quickly, meaning people have to frequently go out and change it. Even if an Internet connection were available, the extra power drain would mean that the camera’s batteries would run out and need to be changed.
 
By adding a TinyML model that is trained to recognize the correct type of animal, the trap can avoid taking photos of leaves moving in the wind. This means traps can be left out longer, which saves time and money for researchers.
 
There are four main constraints that make an application ideal for TinyML. If an application has limited connectivity, low tolerance for latency, a limited source of power, or a requirement for strong privacy, TinyML can potentially help, by avoiding the need to send data from the device.
 
It’s worth noting that only certain problems can be solved by machine learning. They need to be problems where a bit of fuzziness is OK, since ML can rarely give absolute answers. It’s perfect for taking animal photos, where the occasional mistake is acceptable, but it might not be reliable enough in some safety-critical applications.
 
Abate: You co-authored the book, TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers. Since you cover Arduino, it leads me to believe that TinyML technology is accessible to non-experts and pro engineers alike. Besides reading your book, how can engineers and innovators who are unfamiliar with TinyML get started?
 
Situnayake: As always, the best way to learn is to build some projects! A good place to start is by following some of the tutorials we provide in the Edge Impulse documentation:
   
We’ve built Edge Impulse so you can use your mobile phone to collect data and test out models, so you can get started without needing any special hardware. Once you’re familiar with the machine learning flow, you can grab your favorite dev board and start hacking!
 
I’d recommend the Arduino Nano 33 BLE Sense as a good starter board for TinyML. It has a speedy but low-power Cortex M-4F processor, and enough RAM for some interesting models. It’s fully supported in Edge Impulse.

 

Edge Impluse

Abate: You joined San Jose-based Edge Impulse in January 2020. How did that opportunity come about?
 
Situnayake: I met the CEO, Zach Shelby, during my time at Google. He was previously a VP at ARM, who work very closely with the TensorFlow Lite team. When Zach announced he was leaving ARM to found a TinyML company, it caught my attention, and when they posted their first job, I had to apply!
 
Abate: What is your workday like? Are you coding all day? Educating? Business development?

Situnayake: One of the things I’m enjoying the most about life at Edge Impulse is I’m once again primarily an engineer. I spend most of my time hacking. I do still get to meet with customers, which is pretty rewarding, and crucial for knowing how to focus my engineering time. I’m also very active in the TinyML community, so I regularly give talks and help organize TinyML meetups.
 
That said, I usually have a couple of meetings in the morning, then I can get my head down and write some code for the rest of the day. It’s great!
 
Abate: I see that the Edge Impulse team started COVID-19-related project in May — "Cough Detection with TinyML on Arduino." The project has been viewed more than 5,400 times. How has the response been from the engineering community? What is the status of the project?
 
Situnayake: Even though Edge Impulse has only been around a few months, we already have a fantastic community from a diverse set of backgrounds. When COVID-19 hit, one of our community members — Kartik Thakore, a data scientist and biomedical engineer — was interested in building some projects to help. We thought this would be a great opportunity to showcase what can be done with TinyML, so we sent him some hardware and he put together a project to detect coughs using a TinyML model. The result was this project, which we hope inspires makers to tackle similar challenges! 
Cough Detection with TinyML on Arduino (Source: Edge Impulse)
Cough Detection with TinyML on Arduino (Source: Edge Impulse)

TinyML Projects

Abate: Any other TinyML projects you want to highlight for the Elektor community?

Situnayake: At Edge Impulse, we’re strong believers in technology as a force for good, and some of my favorite projects are in that category. For example, Smart Parks are training TinyML models to detect wild elephants, so that villagers can be warned when they’re headed towards town.

Another favorite category of mine is people enabling computers to do crazy things you never would have thought. For example, Benjamin Cabé created this amazing project that uses a gas sensor to recognize different brands of alcoholic spirits! If you’d like to reproduce it, there’s an awesome tutorial from Seeed Studio.