AS THE INTERNET OF THINGS and artificial intelligence (AI) advance and become inexorably linked, there’s a growing need to handle computation on actual devices rather than in the cloud or in data centers.
Machine learning typically relies on computing power in the cloud to tackle complex math problems. However, IoT solutions often need embedded intelligence on drones, cameras, vehicles, industrial machines, and sensors. “The traditional approach doesn’t work well on the edge,” says Zach Shelby, CEO of Edge Impulse, a machine learning development platform provider.
Enter Tiny Machine Learning (TinyML), which refers to an emerging group of open source techniques and approaches that support on-device AI workloads “in resource-constrained, ultra-low-power edge devices,” says James Kobielus, principal analyst at Franconia Research.
TinyML is a significant step forward in the evolution of machine learning, Kobielus notes. “Many device-level AI operations—such as calculations for training and inferencing—must occur serially, thereby placing a priority on fast local execution,” he explains. Within such scenarios, constant roundtrips to the cloud undermine performance—or render a process useless.
5G and inexpensive Raspberry Pi RP2040 microcontrollers now offer a computing and communications framework that fully supports TinyML. “There’s been a perfect storm of better and more efficient machine learning, better batteries, and more efficient embedded compute capabilities. We’ve arrived at machine learning on a milliwatt,” says Shelby. He calls these advances “a complete game changer.”
Devices Get Smarter
TinyML lies at the intersection of ubiquitous edge processing, powerful analytics, and AI. It addresses two primary challenges. First, it consumes fewer computational resources. Second, it reduces latency. “TinyML can boost the speed and efficiency at which AI models can be run by several orders of magnitude,” Kobielus notes. “[It enables] fast AI models to run on edge devices for hours, leveraging only a single CPU core without appreciably draining device batteries.”
The technology framework changes the stakes in several ways. For example, TinyML can automate the tuning of neural network architectures, hyperparameters, and other features within AI to fit the hardware constraints of target platforms.
It also allows device-resident AI models to perform search queries, compute counts, and other operations on efficiently compressed and cached local sensor data. And it can compress local AI models by pruning the less important neural-network connections, reweighting the connections, and applying a more efficient model for encoding, Kobielus explains.