Hi guys,
I might be doing something wrong, but it is a bit suspicious, so prefer posting a message here.
I have got a binary image classification trained, the confusion matrix is not fantastic but it is not so bad either.
I so deployed that in my ESP32 Camera, but I always see the same prediction with high (> 0.9) probability regardless of what the camera sees.
The training set is produced with the same ESP32, so I was not expecting such a problem. Please note that if I use my phone (in classification mode), I see much better results.
Several people had the same issueā¦ I definitely need to take some time this week to have a deeper look.
Are you using the Basic Example or the Advanced one in the github repository?
Note that on the basic one, we just do a cutout of the data instead of a proper resize. This might explain the lack of accuracy.
Also, how many classes do you have on your project?
Hi guys,
sorry for the stalking in the last few days, but I am trying to finish my project. What I am doing
Binary image classification What am using as hardware
ESP32 Cam How I have created the training set
I have got around 1000 pictures directly from the ESP32 (Jpeg pictures) and sent those to my local PC in HTTP, then I have uploaded those to your platform. The dataset is pretty balanced. Which model I then trained
MobileNetV1 0.25 (no final dense layer, 0.1 dropout).
I am able to use the above model without out of memory problems. The confusion matrix is good enough. Which code am using?
A modified version of the advanced classification. I do not need a webserver and stream (saving memory), so doing being able to use the MobileNetV1 0.25, which is not usable with the classic advanced classification sketch.
code is here Which problem I have?
When uploading the model in the ESP32, the device always predicts one class (the same class). When I use my phone, switching to classification mode, the results are better. Please note that as said above, the training set comes directly from the ESP32 cam.
I see on your project that at the moment, your model is not trained.
Could you train it again so I can download your custom version of the arduino library and check your accuracy on Edge Impulse Studio please?
Sure mate.
If you want to use my code, there are a couple of things to change (SSIDā¦).
Also, there is a small part where I send the JPEG in a HTTP request (which does not make sense for you at the moment).
If we are able to find a solution, I would be happy to create a cleaner example and send a PR to your repo. The important thing to note is that, if we get rid of the streaming, we have more ram to allocate for tensorflow, which allow users to run bigger (and probably better) models.
BTW, the training is done and the accuracy in the validation set is good to me.
Hi @louis I have collected some more data, and redone the training, will let you know if the situation gets better. If you got any updates or tips please share.
As you can see I was trying to take a picture of your image displayed on my laptop so the conditions are definitely not ideal but I actually recognized a cat on the last try
I am using Arduino IDE v 1.8.15
In the board manager, i have installed the esp32 boards version 1.0.6 (source of the boards: https://dl.espressif.com/dl/package_esp32_index.json)
I have an AI Thinker board with the OV2640 camera module.
Also additional comment from @janjongboom.
To āseeā what your camera sees, you can print the raw data (like 0x383c2b, 0x343827, 0x2b2f1e,...) and then feed it into one useful tool from our CLI edge-impulse-framebuffer2jpg
ok, a bit weird, but BTW you can see a lot of ānoneā donāt you?
2 points:
Have you downloaded the 8bit integer or the float model? EON vs not EON?
How can I print the buffer?
I will try again because I feel am close to success!
How can I print the buffer?
I guess you could do something similar to this (I havenāt tried this piece of code but I guess it should work, add a delay(10) somewhere if it prints too fast and you loose information in the for loop) :
s = fmt2rgb888(fb->buf, fb->len, fb->format, out_buf);
Serial.print("raw data, total length: ");
Serial.println(EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE);
for (int i = 0; i < EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE; i += 3) {
Serial.print("0x");
Serial.print(out_buf[i], HEX);
Serial.print(out_buf[i+1], HEX);
Serial.print(out_buf[i+2], HEX);
Serial.print(", ");
}
Serial.println("");
edge-impulse-framebuffer2jpg --help
Usage: edge-impulse-framebuffer2jpg [options]
Dump framebuffer as JPG file
Options:
-V, --version output the version number
-b --framebuffer <base64> Framebuffer in base64 format or as a list of RGB888 values
-f --framebuffer-file <file> File with framebuffer in base64 format or as a list of RGB888 values
-w --width <n> Width of the framebuffer
-h --height <n> Height of the framebuffer
-o --output-file <file> Output file
-h, --help output usage information
