Question/Issue:
I am using FOMO (Faster Objects, More Objects) with MobileNetV2 0.35 for object detection in a Driver Monitoring System project. My goal is to export the trained model as TensorFlow Lite and deploy it on an NXP i.MX RT1176 EVK.
The model finishes training successfully and reports approximately 70% accuracy, but when I test it using Launch in Browser, the detections are not reliable. The model often fails to detect the expected classes or produces inconsistent results.
I would like to understand what is the issue that causing this outcome.
Project ID:
1064575
Project Link:
Context/Use case:
I am developing a Driver Monitoring System for an embedded device using Edge Impulse.
The model is trained to detect the following classes:
Awake
Drowsy
Yawn
Each class currently contains approximately 600 labeled images.
The final model will be exported as a TensorFlow Lite model and deployed on an NXP i.MX RT1176 EVK.
Initially trained the model with a smaller dataset.
Increased the dataset size to approximately 600 images per class after reviewing the Edge Impulse documentation.
Trained the model using: Training cycles: 40 Learning rate: 0.001
Expected Outcome:
The model should reliably detect the Awake, Drowsy, Yawn and more lables classes during Live Classification and provide similar performance to the reported training metrics.
Actual Outcome:
While I was testing it always shows the Drowsy even though the eyes are open and yawning is detected sometimes only.
Hello @K-arthi1_51 first of all welcome to the Edge Impulse community.
My initial question is how did you train the model and how are you testing it? using the same conditions (background, light, camera,etc)? or they are different when testing from the browser?
let us know more details on how are you testing the model
I used a dataset downloaded from Roboflow that contains three classes: yawn, drowsy, and awake. I trained the model using FOMO with MobileNetV2 0.35.
For testing I will run the model in the browser where I will use myself.
Questions:
All the my datasets will be like the images I have attached below where the boxes covering whole face. Is this correct way ? or do I need to cover only the mouth/eye ?
For this project is it better to use Object Detection or Image Classification ?
When you tested the resulting model after the training, did you test with a similar angle and similar background? i can imagine that the background and the angle of the camera can affect the inference results.
Is it possible that you add some new images with the testing background and lighting conditions to explore if this is something that improves your inference results?
I have been comenting your case with some of my colleagues and I wanted to follow-up with you to learn what was the situation of your project.
After discussin internally, we don’t think FOMO is the right architecture for your use case/dataset.
Thanks for sharing your project, it made it easier to understand the use case !
Our FOMO architecture is best at detecting objects that are about 1/8th of the total image.
In your project, I see most of your “objects” take a large portion of the full image.
Also, FOMO is training on centroids, see the docs
Also, I feel that running an object detection model for this use case can be overkill, it adds unnecessary complexity of:
Find the object
Classify the object
In your case, the use-case seems pretty scoped, a camera attached to the upper mirror of a car, so the environment is more or less similar. I would suggest to try an image classification model and explore if this improves the results.
