Compilation Failures running Object Detection Model on Arduino IDE for ESP32CAM board

Help
#1

Hi,
This is in relation to my EI project - anupamguptacal-project-1

I’ve been trying to use Edge Impulse to train an object detection model. I have a trained model and translated it to an Arduino Library for deployment. I’m deploying to an ESP32 CAM and have already validated that the example from here: https://github.com/edgeimpulse/example-esp32-cam works correctly post deployment and provides good accuracy.

All I’m doing post this point, is replacing their included library with the Arduino Library I created for my project but I keep facing this problem with some of the fields of the structs not being recognized(during compilation). Specifically the following:

Basic-Image-Classification:164:42: error: 'struct ei_impulse_result_t' has no member named 'classification'
     ei_printf("    %s: \t%f\r\n", result.classification[ix].label, result.classification[ix].value);

and

anupamguptacal-project-1_inferencing\src/edge-impulse-sdk/classifier/ei_run_classifier.h:588:93: error: 'post_process_classes' is not a member of 'tflite'
     fill_result_struct_f32(result, tflite::post_process_boxes, tflite::post_process_scores, tflite::post_process_classes, debug);

and

anupamguptacal-project-1_inferencing\src/edge-impulse-sdk/classifier/ei_run_classifier.h:588:64: error: 'post_process_scores' is not a member of 'tflite'
     fill_result_struct_f32(result, tflite::post_process_boxes, tflite::post_process_scores, tflite::post_process_classes, debug);

My understanding of Object Detection is that it does both Bounding Boxes as well as Classification (I’ve validated this is as expected under the Live Classification Section of the website).

I did some digging around and it seems that in the file ei_classifier_types.cpp in the library, ei_impulse_result_classification_t classification[EI_CLASSIFIER_LABEL_COUNT] is not set if the EI_CLASSIFIER_OBJECT_DETECTION flag is set as 1. I also see that in my model_metadata file, I have the following flags set:

#define EI_CLASSIFIER_OBJECT_DETECTION           1
#define EI_CLASSIFIER_OBJECT_DETECTION_COUNT     10
#define EI_CLASSIFIER_OBJECT_DETECTION_THRESHOLD 0.5
#define EI_CLASSIFIER_TFLITE_OUTPUT_LABELS_TENSOR 1
#define EI_CLASSIFIER_TFLITE_OUTPUT_SCORE_TENSOR  2

I haven’t dived into the tensor flow errors but maybe it’s something similar. I guess my question is: What am I doing wrong? Is there an example I can follow that defines how to use object detection models with the Arduino IDE? Or did I deploy my model incorrectly in the first place itself and I need to undo something?

Any advice in this regard would be really helpful. Thank you!

#2

I’m getting the same error as above when using the unedited model from here: https://docs.edgeimpulse.com/docs/object-detection as an arduino library. Compilation on Arduino IDE fails with the error that there is no post_process_boxes, post_process_classes and post_process_scores defined for tflite and 'struct ei_impulse_result_t' has no member named 'classification'. So maybe this is not specific to my library only?

ESP32 CAM error on image_util.h
#3

Hello @anupamguptacal,

Object Detection models cannot fit at the moment on the ESP32 CAM. We are working on smaller models at the moment as it has been announced on the opening keynote during our Imagine event (https://youtu.be/jTDjO1xf-yc?t=3025 around min 50).

Regards,

Louis

1 Like
#4

I’m getting this same compile error when trying to upload generated example project to my Arduino Portenta H7 with Camera Shield. What is the right way to go about fixing this?

#5

Hey @louis , are esp-cam’s still not supported?

Just came across this same error.

#6

Hello @nicosandller,

ESP32-CAM (AI Thinker) is not officially supported.
However, you can compile your edge impulse models using ESP IDF (not using Arduino IDE).
Doing this it will use the esp32 neural networks (esp-nn) hardware acceleration.

Just change the code lines as explained here: Espressif ESP-EYE (ESP32) - Edge Impulse Documentation and recompile the code.

If you need the default firmware to collect data directly from the AI Thinker board, here is the firmware that I already recompiled using the same technique: https://edgeimpulse-public.s3.fr-par.scw.cloud/espressif-esp32-cam-ai-thinker.zip
Just flash it as you would do for the ESP-EYE board.

I’ll probably put the .zip file somewhere else in the future.
Also, the won’t be up-to-date as we update our SDK constantly and the AI Thinker is not officially supported.

If you really need to use Arduino IDE. I have not tested if it works with FOMO models for object detection projects. I’d say it should work with small input images and it won’t use hardware acceleration so it would probably be very slow.

Regards,

Louis

2 Likes
Edge-impulse-deamon timeout (ESP32-CAM, Raspberrypi)
ESP32 Firmware on AI-Thinker (esp32cam) board Failure
#7

thanks for the firmware for data collection. Works great!

Regarding running predictions with FOMO, I’ll give it a try using esp-idf!

Thanks a lot!

2 Likes
#8

Hi @louis,

I’m getting the same error of:

/Users/xxx/Documents/GitHub/example-standalone-inferencing-espressif-esp32/main/main.cpp:54:44: warning: missing initializer for member 'ei_impulse_result_t::anomaly' [-Wmissing-field-initializers]
/Users/xxx/Documents/GitHub/example-standalone-inferencing-espressif-esp32/main/main.cpp:54:44: warning: missing initializer for member 'ei_impulse_result_t::timing' [-Wmissing-field-initializers]
/Users/xxx/Documents/GitHub/example-standalone-inferencing-espressif-esp32/main/main.cpp:54:44: warning: missing initializer for member 'ei_impulse_result_t::label_detected' [-Wmissing-field-initializers]
/Users/xxx/Documents/GitHub/example-standalone-inferencing-espressif-esp32/main/main.cpp:90:34: error: 'struct ei_impulse_result_t' has no member named 'classification'
         ei_printf("%.5f", result.classification[ix].value);
                                  ^~~~~~~~~~~~~~
[1053/1140] Building CXX object esp-idf/main/CMakeFiles/__idf_main.dir/__/edge-impulse-sdk/tensorflow/lite/core/api/flatbuffer_conversions.cc.obj
ninja: build stopped: subcommand failed.
ninja failed with exit code 1

I tried with both your firmware (https://edgeimpulse-public.s3.fr-par.scw.cloud/espressif-esp32-cam-ai-thinker.zip) and the esp32 (https://github.com/edgeimpulse/firmware-espressif-esp32).

Any idea what i’m missing?
Thanks