Deployed model shows incorrect classification

Report bugs
#1

Question/Issue: My image classification works well on Edge Impulse website, but fails when I deploy to a Linux x86 computer.

Project ID: 107572 / DuckDetector

Context/Use case: I have an IP camera focused on a dock. It is looking for ducks. I bring in raw images via RSTP and Opencv. I collected 88 images of various ducks on the dock in different lighting conditions and there is 1 image with no ducks present. I developed a 320x320 image detection model and it works great on the Edge Impluse website.

I then clicked on Deployment, Linux (x86), Quantized model and then Build. This creates a *.eim model for download. I run this model with the provided Python example classify-image.py. It seems no matter what image I give it, it always says ducks are present with high confidence.

What am I doing wrong, or is there a bug in the system?

#2

Hello @mikeh9999 ,

Could you try with the float32 model and let me know if that works better.

Regards,

Louis

#3

I built two models v6 - Int8 and v7 - float32. They seem to perform the same on my linux computer.
image

One thing I notice is that both models are almost the same *.eim file size. v6.eim is 7,934KB and v7.eim is 7,934KB.

Is that normal for both models to be almost the same size?

I notice I can download other model formats on the dashboard such as the Keras model.

*Do you think the .eim files are corrupted? Should I download the Keras model above and try that instead?

A little bit more information…my model gets a F1 score of 95%. When I run the live classification online, it correctly identifies no ducks when there are none 100% of the time. The only mistake is that in one image, it identifies 1 duck when there are actually 2. On both downloaded *.eim models, it identifies there are two ducks when there are none.

#4

Hello @mikeh9999,

You can also use the command:

edge-impulse-linux-runner --download modelfile.eim

to download the .eim model.

I’m checking your project now to see if I see anything odd.

Regards,

Louis

#5

Also, is your camera positioned the same way for your real tests?

If it is still not working, I would recommend to add more data (like double your dataset size).

Regards,

Louis

#6

For right now, the camera stays the same. When I roll this out to a commercial application, then each site will be different. Eventually, with enough data, I will end up with a universal model. :slight_smile:

I will work on gathering more data. Thanks for your advice!

#7

I got my first working model ! :grin:

I trained a classifier instead of a detector. I have 100 images of ducks and 40 images of just the dock. These are captured at different times of the day and night (varied lighting conditions).

My model has 100% accuracy.

Next, will be to optimize the model and deploy to a small microcontroller.

1 Like