Developing TPU Based AI Solutions Using TensorFlow Lite
June 06, 2022
Story
AI has become ubiquitous today from personal devices to enterprise applications, you see them everywhere. The advent of IoT clubbed with rising demand for data privacy, low power, low latency, and bandwidth constraints has increasingly pushed for AI models to be running at the edge instead of the cloud.
According to Grand View Research, the global edge artificial intelligence chips market was valued at USD 1.8 billion in 2019 and is expected to grow at a CAGR of 21.3 percent from 2020 to 2027. At this onset, Google introduced Edge TPU, also known as Coral TPU, which is its purpose-built ASIC for running AI at the edge. It's designed to give excellent performance while taking up minimal space and power.
When we train an AI model, we end up with AI models that have high storage requirements and GPU processing power. We cannot execute them on devices that have low memory and processing footprints. TensorFlow Lite is useful in this situation. TensorFlow Lite is an open-source deep learning framework that runs on the Edge TPU and allows for on-device inference and AI model execution. Also note that TensorFlow Lite is only for executing inference on the edge, not for training a model. For training an AI model, we must use TensorFlow.
Combining Edge TPU and TensorFlow Lite
When we talk about deploying an AI model on Edge TPU, we just cannot deploy any AI model.
The Edge TPU supports NN (Neural Network) operations and designs to enable high-speed neural network performance with low power consumption. Apart from specific networks, it only supports 8-bit quantized and compiled TensorFlow Lite models for Edge TPU.
For a quick summary, TensorFlow Lite is a lightweight version of TensorFlow specially designed for mobile and embedded devices. It achieves low latency results with small storage size. There is a TensorFlow Lite converter that allows converting a TensorFlow-based AI model file (. pb) to a TensorFlow Lite file (.tflite). Below is a standard workflow for deploying applications on Edge TPU.
Application deployment on Edge TPU
Let’s look at some interesting real-world applications that can be built using TensorFlow Lite on edge TPU.
- Human Detection and Counting
This solution has so many practical applications, especially in malls, retail, government offices, banks, and enterprises. One may wonder what one can do with detecting and counting humans. Data now has the value of time and money. Let us see how the insights from human detection and counting can be used.
- Estimating Footfalls: For the retail industry, this is important as it gives an idea if their stores are doing well. Whether their displays are attracting customers to enter the shops. It also helps them to know if they need to increase or decrease support staff. For other organizations, they help in taking adequate security measures for people.
- Crowd Analytics and Queue Management: For government offices and enterprises, queue management via human detection and counting helps them manage longer queues and save people’s time. Studying queues can attribute to individual and organizations' performance. Crowd detection can help analyze crowd alerts for emergencies, security incidents, etc., and take appropriate actions. Such solutions give the best results when deployed on edge, as required actions can be taken close to real-time.
- Age and Gender-based Targeted Advertisements.
This solution mainly has practical applications in the retail and advertisement industry. Imagine you walking towards the advertisement display which was showing a women’s shoe ad and then suddenly the advertisement changes to a male’s shoe ad as it determined you being male. Targeted advertisements help retailers and manufacturers target their products better and create brand awareness that a normal person would never get to see in his busy life.
This cannot be restricted to only advertisements, age and gender detection can also help businesses in taking quick decisions by managing appropriate support staff in retail stores, what age and gender of people prefer visiting your store, businesses, etc. All this is more powerful and effective if you are very quick to determine and act. So, even more, a reason to have this solution on Edge TPU.
- Face Recognition
The very first face recognition system was built in 1970, and to date this is still being developed, being made more robust and effective. The main advantage of having face recognition on edge is real-time recognition. Another advantage is having face encryption and feature extraction on edge, and just sending encrypted and extracted data to the cloud for matching, thereby protecting PII level privacy of face images (as you don’t save face images on edge and cloud) and complying with stringent privacy laws.
Edge TPU combined with the TensorFlow Lite framework opens several edge AI application opportunities. As the framework is open-source the Open-Source Software (OSS) community also supports it, making it even more popular for machine learning use cases. The overall platform of TensorFlow Lite enhances the environment for the growth of edge applications for embedded and IoT devices.
Rakesh R Nakod is an Associate Principal Engineer at Softnautics, an AI proficient having experience in developing and deploying AI solutions across computer vision, NLP, audio intelligence, and document mining. He also has vast experience in developing AI-based enterprise solutions and strives to solve real-world problems with AI. He is an avid food lover, passionate about sharing knowledge, and enjoys gaming, and playing cricket in his free time.
Rakesh R Nakod is an Associate Principal Engineer at Softnautics, an AI proficient having experience in developing and deploying AI solutions across computer vision, NLP, audio intelligence, and document mining. He also has vast experience in developing AI-based enterprise solutions and strives to solve real-world problems with AI. He is an avid food lover, passionate about sharing knowledge, and enjoys gaming, and playing cricket in his free time.
