LogoLogo
HomeDocsAPI & SDKsProjectsForumStudio
  • Welcome
    • Featured Machine Learning Projects
      • Getting Started with the Edge Impulse Nvidia TAO Pipeline - Renesas EK-RA8D1
      • Smart City Traffic Analysis - NVIDIA TAO + Jetson Orin Nano
      • ROS 2 Pick and Place System - Arduino Braccio++ Robotic Arm and Luxonis OAK-D
      • Optimize a cloud-based Visual Anomaly Detection Model for Edge Deployments
      • Rooftop Ice Detection with Things Network Visualization - Nvidia Omniverse Replicator
      • Surgery Inventory Object Detection - Synthetic Data - Nvidia Omniverse Replicator
      • NVIDIA Omniverse - Synthetic Data Generation For Edge Impulse Projects
      • Community Guide – Using Edge Impulse with Nvidia DeepStream
      • Computer Vision Object Counting - Avnet RZBoard V2L
      • Gesture Appliances Control with Pose Detection - BrainChip AKD1000
      • Counting for Inspection and Quality Control - Nvidia Jetson Nano (TensorRT)
      • High-resolution, High-speed Object Counting - Nvidia Jetson Nano (TensorRT)
    • Prototype and Concept Projects
      • Renesas CK-RA6M5 Cloud Kit - Getting Started with Machine Learning
      • TI CC1352P Launchpad - Getting Started with Machine Learning
      • OpenMV Cam RT1062 - Getting Started with Machine Learning
      • Getting Started with Edge Impulse Experiments
  • Computer Vision Projects
    • Workplace Organizer - Nvidia Jetson Nano
    • Recyclable Materials Sorter - Nvidia Jetson Nano
    • Analog Meter Reading - Arduino Nicla Vision
    • Creating Synthetic Data with Nvidia Omniverse Replicator
    • SonicSight AR - Sound Classification with Feedback on an Augmented Reality Display
    • Traffic Monitoring - Brainchip Akida
    • Multi-camera Video Stream Inference - Brainchip Akida
    • Industrial Inspection Line - Brainchip Akida
    • X-Ray Classification and Analysis - Brainchip Akida
    • Inventory Stock Tracker - FOMO - BrainChip Akida
    • Container Counting - Arduino Nicla Vision
    • Smart Smoke Alarm - Arduino Nano 33
    • Shield Bot Autonomous Security Robot
    • Cyclist Blind Spot Detection - Himax WE-I Plus
    • IV Drip Fluid-Level Monitoring - Arduino Portenta H7
    • Worker PPE Safety Monitoring - Nvidia Jetson Nano
    • Delivered Package Detection - ESP-EYE
    • Bean Leaf Disease Classification - Sony Spresense
    • Oil Tank Measurement Using Computer Vision - Sony Spresense
    • Object Counting for Smart Industries - Raspberry Pi
    • Smart Cashier with FOMO - Raspberry Pi
    • PCB Defect Detection with Computer Vision - Raspberry Pi
    • Bicycle Counting - Sony Spresense
    • Counting Eggs with Computer Vision - OpenMV Cam H7
    • Elevator Passenger Counting - Arduino Nicla Vision
    • ESD Protection using Computer Vision - Seeed ReComputer
    • Solar Panel Defect Detection - Arduino Portenta H7
    • Label Defect Detection - Raspberry Pi
    • Dials and Knob Monitoring with Computer Vision - Raspberry Pi
    • Digital Character Recognition on Electric Meter System - OpenMV Cam H7
    • Corrosion Detection with Computer Vision - Seeed reTerminal
    • Inventory Management with Computer Vision - Raspberry Pi
    • Monitoring Retail Checkout Lines with Computer Vision - Renesas RZ/V2L
    • Counting Retail Inventory with Computer Vision - Renesas RZ/V2L
    • Pose Detection - Renesas RZ/V2L
    • Product Quality Inspection - Renesas RZ/V2L
    • Smart Grocery Cart Using Computer Vision - OpenMV Cam H7
    • Driver Drowsiness Detection With FOMO - Arduino Nicla Vision
    • Gastroscopic Image Processing - OpenMV Cam H7
    • Pharmaceutical Pill Quality Control and Defect Detection
    • Deter Shoplifting with Computer Vision - Texas Instruments TDA4VM
    • Smart Factory Prototype - Texas Instruments TDA4VM
    • Correct Posture Detection and Enforcement - Texas Instruments TDA4VM
    • Visual Anomaly Detection with FOMO-AD - Texas Instruments TDA4VM
    • Surface Crack Detection and Localization - Texas Instruments TDA4VM
    • Surface Crack Detection - Seeed reTerminal
    • Retail Image Classification - Nvidia Jetson Nano
    • SiLabs xG24 Plus Arducam - Sorting Objects with Computer Vision and Robotics - Part 1
    • SiLabs xG24 Plus Arducam - Sorting Objects with Computer Vision and Robotics - Part 2
    • Object Detection and Visualization - Seeed Grove Vision AI Module
    • Bike Rearview Radar - Raspberry Pi
    • Build a Self-Driving RC Vehicle - Arduino Portenta H7 and Computer Vision
    • "Bring Your Own Model" Image Classifier for Wound Identification
    • Acute Lymphoblastic Leukemia Classifier - Nvidia Jetson Nano
    • Hardhat Detection in Industrial Settings - Alif Ensemble E7
    • Motorcycle Helmet Identification and Traffic Light Control - Texas Instruments AM62A
    • Import a Pretrained Model with "Bring Your Own Model" - Texas Instruments AM62A
    • Product Inspection with Visual Anomaly Detection - FOMO-AD - Sony Spresense
    • Visual Anomaly Detection in Fabric using FOMO-AD - Raspberry Pi 5
    • Car Detection and Tracking System for Toll Plazas - Raspberry Pi AI Kit
    • Visual Anomaly Detection - Seeed Grove Vision AI Module V2
    • Object Counting with FOMO - OpenMV Cam RT1062
    • Visitor Heatmap with FOMO Object Detection - Jetson Orin Nano
    • Vehicle Security Camera - Arduino Portenta H7
  • Audio Projects
    • Occupancy Sensing - SiLabs xG24
    • Smart Appliance Control Using Voice Commands - Nordic Thingy:53
    • Glass Window Break Detection - Nordic Thingy:53
    • Illegal Logging Detection - Nordic Thingy:53
    • Illegal Logging Detection - Syntiant TinyML
    • Wearable Cough Sensor and Monitoring - Arduino Nano 33 BLE Sense
    • Collect Data for Keyword Spotting - Raspberry Pi Pico
    • Voice-Activated LED Strip - Raspberry Pi Pico
    • Snoring Detection on a Smart Phone
    • Gunshot Audio Classification - Arduino Nano 33 + Portenta H7
    • AI-Powered Patient Assistance - Arduino Nano 33 BLE Sense
    • Acoustic Pipe Leakage Detection - Arduino Portenta H7
    • Location Identification using Sound - Syntiant TinyML
    • Environmental Noise Classification - Nordic Thingy:53
    • Running Faucet Detection - Seeed XIAO Sense + Blues Cellular
    • Vandalism Detection via Audio Classification - Arduino Nano 33 BLE Sense
    • Predictive Maintenance Using Audio Classification - Arduino Nano 33 BLE Sense
    • Porting an Audio Project from the SiLabs Thunderboard Sense 2 to xG24
    • Environmental Audio Monitoring Wearable - Syntiant TinyML - Part 1
    • Environmental Audio Monitoring Wearable - Syntiant TinyML - Part 2
    • Keyword Spotting - Nordic Thingy:53
    • Detecting Worker Accidents with Audio Classification - Syntiant TinyML
    • Snoring Detection with Syntiant NDP120 Neural Decision Processor - Arduino Nicla Voice
    • Recognize Voice Commands with the Particle Photon 2
    • Voice Controlled Power Plug with Syntiant NDP120 (Nicla Voice)
    • Determining Compressor State with Audio Classification - Avnet RaSynBoard
    • Developing a Voice-Activated Product with Edge Impulse's Synthetic Data Pipeline
    • Enhancing Worker Safety using Synthetic Audio to Create a Dog Bark Classifier
  • Predictive Maintenance and Defect Detection Projects
    • Predictive Maintenance - Nordic Thingy:91
    • Brushless DC Motor Anomaly Detection
    • Industrial Compressor Predictive Maintenance - Nordic Thingy:53
    • Anticipate Power Outages with Machine Learning - Arduino Nano 33 BLE Sense
    • Faulty Lithium-Ion Cell Identification in Battery Packs - Seeed Wio Terminal
    • Weight Scale Predictive Maintenance - Arduino Nano 33 BLE Sense
    • Fluid Leak Detection With a Flowmeter and AI - Seeed Wio Terminal
    • Pipeline Clog Detection with a Flowmeter and AI - Seeed Wio Terminal
    • Refrigerator Predictive Maintenance - Arduino Nano 33 BLE Sense
    • Motor Pump Predictive Maintenance - Infineon PSoC 6 WiFi-BT Pioneer Kit + CN0549
    • BrickML Demo Project - 3D Printer Anomaly Detection
    • Condition Monitoring - Syntiant TinyML Board
    • Predictive Maintenance - Commercial Printer - Sony Spresense + CommonSense
    • Vibration Classification with BrainChip's Akida
    • AI-driven Audio and Thermal HVAC Monitoring - SeeedStudio XIAO ESP32
  • Accelerometer and Activity Projects
    • Arduino x K-Way - Outdoor Activity Tracker
    • Arduino x K-Way - Gesture Recognition for Hiking
    • Arduino x K-Way - TinyML Fall Detection
    • Posture Detection for Worker Safety - SiLabs Thunderboard Sense 2
    • Hand Gesture Recognition - OpenMV Cam H7
    • Arduin-Row, a TinyML Rowing Machine Coach - Arduino Nicla Sense ME
    • Fall Detection using a Transformer Model – Arduino Giga R1 WiFi
    • Bluetooth Fall Detection - Arduino Nano 33 BLE Sense
    • Monitor Packages During Transit with AI - Arduino Nano 33 BLE Sense
    • Smart Baby Swing - Arduino Portenta H7
    • Warehouse Shipment Monitoring - SiLabs Thunderboard Sense 2
    • Gesture Recognition - Bangle.js Smartwatch
    • Gesture Recognition for Patient Communication - SiLabs Thunderboard Sense 2
    • Hospital Bed Occupancy Detection - Arduino Nano 33 BLE Sense
    • Porting a Posture Detection Project from the SiLabs Thunderboard Sense 2 to xG24
    • Porting a Gesture Recognition Project from the SiLabs Thunderboard Sense 2 to xG24
    • Continuous Gait Monitor (Anomaly Detection) - Nordic Thingy:53
    • Classifying Exercise Activities on a BangleJS Smartwatch
  • Air Quality and Environmental Projects
    • Arduino x K-Way - Environmental Asthma Risk Assessment
    • Gas Detection in the Oil and Gas Industry - Nordic Thingy:91
    • Smart HVAC System with a Sony Spresense
    • Smart HVAC System with an Arduino Nicla Vision
    • Indoor CO2 Level Estimation - Arduino Portenta H7
    • Harmful Gases Detection - Arduino Nano 33 BLE Sense
    • Fire Detection Using Sensor Fusion and TinyML - Arduino Nano 33 BLE Sense
    • AI-Assisted Monitoring of Dairy Manufacturing Conditions - Seeed XIAO ESP32C3
    • AI-Assisted Air Quality Monitoring - DFRobot Firebeetle ESP32
    • Air Quality Monitoring with Sipeed Longan Nano - RISC-V Gigadevice
    • Methane Monitoring in Mines - Silabs xG24 Dev Kit
    • Smart Building Ventilation with Environmental Sensor Fusion
    • Sensor Data Fusion with Spresense and CommonSense
    • Water Pollution Detection - Arduino Nano ESP32 + Ultrasonic Scan
    • Fire Detection Using Sensor Fusion - Arduino Nano 33 BLE Sense
  • Novel Sensor Projects
    • 8x8 ToF Gesture Classification - Arduino RP2040 Connect
    • Food Irradiation Dose Detection - DFRobot Beetle ESP32C3
    • Applying EEG Data to Machine Learning, Part 1
    • Applying EEG Data to Machine Learning, Part 2
    • Applying EEG Data to Machine Learning, Part 3
    • Liquid Classification with TinyML - Seeed Wio Terminal + TDS Sensor
    • AI-Assisted Pipeline Diagnostics and Inspection with mmWave Radar
    • Soil Quality Detection Using AI and LoRaWAN - Seeed Sensecap A1101
    • Smart Diaper Prototype - Arduino Nicla Sense ME
    • DIY Smart Glove with Flex Sensors
    • EdgeML Energy Monitoring - Particle Photon 2
    • Wearable for Monitoring Worker Stress using HR/HRV DSP Block - Arduino Portenta
  • Software Integration Demos
    • Azure Machine Learning with Kubernetes Compute and Edge Impulse
    • ROS2 + Edge Impulse, Part 1: Pub/Sub Node in Python
    • ROS2 + Edge Impulse, Part 2: MicroROS
    • Using Hugging Face Datasets in Edge Impulse
    • Using Hugging Face Image Classification Datasets with Edge Impulse
    • Edge Impulse API Usage Sample Application - Jetson Nano Trainer
    • MLOps with Edge Impulse and Azure IoT Edge
    • A Federated Approach to Train and Deploy Machine Learning Models
    • DIY Model Weight Update for Continuous AI Deployments
    • Automate the CI/CD Pipeline of your Models with Edge Impulse and GitHub Actions
    • Deploying Edge Impulse Models on ZEDEDA Cloud Devices
Powered by GitBook
On this page
  • Heating Oil And Usage
  • Operational Inefficiencies
  • How AI On The Edge Can Help?
  • Edge Impulse And Visual Regression
  • Data Capture
  • Creating Impulse
  • Deploy
  • Connect To AWS IoT
  • System Architecture
  • Dashboard UI
  • Demo
  • Low-Powered Battery Operated
  • Challenges
  • Leak & Anomaly Detection
  • Code

Was this helpful?

Edit on GitHub
Export as PDF
  1. Computer Vision Projects

Oil Tank Measurement Using Computer Vision - Sony Spresense

Use a Sony Spresense and computer vision to convert an analog oil tank gauge to a digital logistics application.

PreviousBean Leaf Disease Classification - Sony SpresenseNextObject Counting for Smart Industries - Raspberry Pi

Last updated 1 year ago

Was this helpful?

Created By: Mithun Das

Public Project Link:

Heating Oil And Usage

Heating oil is mainly used for space heating. Some homes and residential commercial buildings also use heating oil to heat water but in much smaller amounts than what they use for space heating. Because cold weather affects heating demand, most heating oil use occurs during the heating season—October through March.

In the winter of 2020–2021, about 5.3 million households in the United States used heating oil (distillate fuel oil) as their main space heating fuel, and about 82% of those households were in the U.S. Northeast.

Operational Inefficiencies

Usually heating oil companies deliver the oil in a truck. Many heating oil companies monitor your usage to determine when you will need an oil delivery again. These calculations are performed using "degree days" and a K-factor. The degree days calculation adds the high and low temperatures on a given day, divides it by two, and subtracts 65 from the quotient. This number is added to your home’s K-factor.

The K-factor determines “degree days per gallon” (think something similar to “miles per gallon” here) to estimate how quickly you use heating oil. Over time, our heating oil company can settle on a relatively stable K-factor for your home. We can then calculate your next delivery using your unique K-factor and our own calculations.

But this calculation is not accurate all the time. Family can go on vacation, resulting in less oil consumption, or relatives visiting during holiday season could result in more oil consumption.

Okay, so what's the big deal?

Extra cost and additional CO2 emissions - If the delivery is made when the tank is still 3/4 full, it's a waste of money and extra emission of CO2 in the environment. Delivery during cold weather is expensive in terms of effort as well. It does not make sense to fill the tank when it's already 3/4 full.

No heat during cold - On the other hand, if consumption is high suddenly and the tank is empty, there would not be any oil to heat the house. This can be very challenging and life threatening as well.

With real time reading of oil tanks, delivery companies can schedule the delivery precisely when it's needed. This will eliminate unwanted visits to the resident, reducing delivery costs and lowering CO2 emissions.

How AI On The Edge Can Help?

Most oil tanks have analog meters which are not easy to read by machines. Moreover, adding a sensor in an existing tank to measure fuel level is expensive as it needs some poking and prodding, and oil tanks are typically old and there is a risk to damage the tank.

I am looking for a cost effective and non-invasive way to read the analog meter in real time.

Using visual regression, we can train a model to read an analog meter and predict a scalar value between 0 to 100. This value is then sent to the cloud using a cellular connection.

Edge Impulse And Visual Regression

Data Capture

With any ML project, data capture is the most important step. To capture data from our heating oil tank is a time consuming task as the meter moves very slow, and I may need to wait for a month to capture all the different readings. So I designed and 3D printed an analog meter similar to a real heating oil tank meter, to capture data.

Precise prediction isn't necessary for this use-case, I mean it's not a big deal if prediction is 5-10% off. So I choose the following readings: 0%, 13%, 25%, 37%, 50%, 65%, 87% and 100%.

Ideally we must refill our tank when it's around a quarter full, meaning around 25-30%.

Creating Impulse

Creating an "Impulse" in Edge Impulse, is basically creating your ML model.

I have collected 96x96 pixel images, but trained the model with 60x60 grayscale. This makes the model size small, but compromises the accuracy a bit. You should experiment with different options and see what works best for you.

Once you finish training, you may see lot of incorrect regression, but if you take a closer look (above image), you will realize those are not entirely incorrect. For example, 25 was predicted as 25.12, which we can happily accept for our use-case.

Deploy

I have decided to download the model as a quantized Arduino library, considering Flash memory & RAM usage. Then used that in my Arduino sketch.

#include <sony-analog-meter_inferencing.h>

Connect To AWS IoT

Once I get the model prediction, I connect to a cellular network and send data to AWS IoT Core over MQTT. Before you can connect to AWS IoT Core, you need to:

  • Create a policy

  • Create and download certificates ( Root CA, certificate and private key )

  • Attach certificates to the created policy

  • Create a Thing

  • Attach policy to the Thing

It can be overwhelming. So I created a python script which will do all this for you. Locate the register.py program, change the variables, and run the program. This will create the AWS resources for you, and download the certificates and private key under the certs folder. Now you need to copy those files to the SD card and insert it into the Spresense LTE board.

System Architecture

Once data is sent to the AWS IoT topic, the defined IoT Rule invokes a Lambda Function passing the value.

The Lambda Function reads the raw value coming from the Topic, maps that to 1/4, 1/2, 3/4 and full, and persists the data to a dynamoDB table through GraphQL mutation.

Mapping of raw data is to flatten the prediction error. Like in below image, raw values came from the Sony Spresense varied between 47 and 51. But in reality the meter did not change. Rounding off those would give us a reading of 50. The reading on the dashboard will change when there is a significant change in the reading, such as 10% decrease.

Dashboard UI

Before we install and start consuming data from the smart device, we need to register the device on the app.

Once the smart meter is installed and powered on, the app will start showing data on the dashboard.

Demo

Low-Powered Battery Operated

Another key aspect of this project is to run the smart meter on battery for several months before recharging. This is important, as there may not be a power source near the heating oil tank.

Things considered to maximize the use of battery are:

  • Sony Spresense supports 156, 32 or 8 MHz clock frequency. I set clock mode to 32 MHz, as the Camera can work on it. And power consumption is almost half compared to 156 MHz.

  • Stored AWS certificates in a header file rather than on the SD card. Typically it draws around 40mA for read/write operations, which is quite high compare to GNSS read (7mA).

  • Not reading GNSS to obtain GPS location of the smart meter as its location is static. The location is not going to change once it's installed. Smart meter location is inserted from the app during device registration. This helps save battery power further.

  • Reading data once every day. As the oil level is not going to decrease significantly every hour, it's overkill to read the meter very frequently like every few minutes or hours. The smart meter wakes up every 24 hours, takes a picture, runs the inference to predict the reading, connects to the cellular network, and sends data to AWS IoT Core over MQTT. The whole process takes around 150 seconds. The rest of the time the device goes to deep sleep mode, drawing about 300 μA.

Based on my brief research, with clock set to 32MHz, average current draw was around 75mA with a spike to 213 mA when the board connects to the cellular network and sends data to AWS. And total execution time was around 76 seconds.

With clock set to 156 MHz, average current draw was around 110 mA with one big spike to 264mA and few spikes to between 150-200mA. We may consider average of 115 mA. Total execution time was 64 seconds compared to 76 seconds.

Hourly consumption is calculated as:

Cawake × (Tawake / 3600) + Csleep × ((3600-Tawake) / 3600)

Assuming current draw is 75mA (32 MHz clock), with a 2500mAh battery, it will last about 55 days and with 115mA (156 MHz clock), it will last about 44 days.

But note, this is an estimate and actual battery life depends on a lot of environmental factors such as temperature, qualify of the battery pack, how fast it discharges, etc.

Challenges

Of course no project is without challenges. I am not going to discuss any technical challenges I faced to get the Sony Spresense working with LTE over a secured connection or setting up the Truphone Sim card as I believe it's very generic and common with any new development board we try for the first time. Other challenges to point out are:

  • Different background of the image has huge impact on the performance of the regression model. So I decided to have a static background. This is a feasible option in production as well.

  • Lighting is another aspect. Heating oil tanks are mostly located in the basement without any natural light. We should have an LED light turn on before taking the picture.

Leak & Anomaly Detection

As I am receiving readings from the board, we can run ML models on AWS to predict any abnormality such as too fast draining of oil, which may indicate some leakage or provide some feedback to the customer to review the usage.

Code

Heating oil is also used in many countries in Europe.

The Sony Spresense is supported by Edge Impulse out of the box, which means they have firmware to help you connect the board to the EI Studio and capture images. Please checkout this which explains how.

You may take a look at the EI project .

I am using AWS Amplify to create the GraphQL API and Lambda function. I am also using Amplify to host the web app. If you have not worked with , I would highly encourage you to check that out.

Read for more information.

Code for this project is located here:

Source
blog
here
AWS Amplify
this article
https://github.com/just4give/sony-analog-meter
https://studio.edgeimpulse.com/public/105298/latest
Custom 3D printed analog meter
Deployment
AWS IoT Rule
Data received by AWS IoT Core
1100mAh With Clock 32MHz
1100mAh Clock 156MHz