
Problem Statement
Monitoring fragile objects during shipment require more time, care, labor effort, and infrastructure support. In an ideal scenario, shipments would travel from the warehouse directly into the hands of the customer. However, that is not always the case. In between, a shipment / package will be handled many times, and you can’t always expect the people doing the transportation to handle them in a manner that will not lead to damage.
- In a warehouse, storing fragile objects in the wrong position can damage the product.
- No real time monitoring of the shipments inside a warehouse about placement of products.
- Handling the shipment in the wrong position during transferring to the destination might cause damage to the product.
- Computer Vision-based shipment monitoring requires infrastructure to support it. For example, poor lightning conditions might affect the prediction.
TinyML Solution
I have created a TinyML model which can be attached to a fragile package and it tracks and predicts the position of the package, with the results communicated to a Mobile application through BLE. For hardware, I have used the SiLabs Thunderboard Sense 2 and Edge Impulse to train the model and deploy it to the Thunderboard Sense 2. The predicted result will be one of four categories:- Correct position
- Incorrect position - Upside-down
- Incorrect position - Tilted forward
- Incorrect position - Tilted backward
Data Acquisition
Connect the Thunderboard Sense 2 board to the system and flash the firmware from this link.
edge-impulse-daemon
Now the board is connected to your Edge Impulse account. Then place the hardware in a case, and attach it to the shipment package. In the below picture, you’ll notice that the Thunderboard is placed inside the case and mounted on top of the package. Now collect the accelerometer data.

Correct Position
Let’s assume the package with fragile object is orientated correctly. Place the Thunderboard on top of it, for data acquisition. Now start moving the package with this position, while data acquisition is running. Collect a dataset of around 2 minutes 30 seconds.
Incorrect Position - Upside-down
Turn the package over so that it is upside down, with the hardware still on it. Now start moving the package in this position, while data acquisition is running. Collect a dataset of around 2 minutes 30 seconds.
Incorrect Position - Tilted Forward
Turn the package on it’s side, with the hardware still on it, facing forward. Now start moving the package in this position, while data acquisition is running. Collect a dataset of around 2 minutes 30 seconds.
Incorrect Position - Tilted Backward
Turn the package on it’s side, with the hardware still on it, facing backward. Now start moving the package in this position, while data acquisition is running. Collect a dataset of around 2 minutes 30 seconds.



Neural Network Training
In the Neural network training, I have used sequential layers with a Dense neural network layer and drop out of 0.1 to avoid the over fitting. I have chosen the learning rate as 0.0005 and 200 training cycles. Upon completion, the training model Accuracy is 100%:

Model Testing
In the Model testing section, I have tested the trained model with the 30-second dataset that we set aside earlier for each category.
Deployment
Go to the Deployment section, and select Firmware option “Thunderboard Sense 2”. This will generate and download the firmware to your local system.
.bin
file and paste it in the TB004 drive attached to your PC. This will flash the software onto the Thunderboard Sense 2 board.

Testing in the Real World
To test it out in a real scenario, download the “LightBlue” app from the iOS App Store or Google Play store. This app will be used to communicate to with the Thunderboard Sense 2. Open the App and connect to the Edge Impulse service (Make sure board is powered on). Input settings in the application:- Subscribe to the
2A56
characteristic. - Decode the message as UTF-8 (click on
HEX
in the top right corner in LightBlue to switch). - Place the package in different positions and start moving the package.
- The app will notify the Thunderboard.

Results



