2 Questions about importing Edge Impulse Arduino : ESP32

[SunBeenMoon]

Hi,
I need some help about importing edge impulse on ESP-EYE board through Arduino.

• Target Board : ESP-EYE
  (I’ve check the board working on Arduino with Example ‘Scanning WiFi’)
  (Setting the Board name ‘DOIT ESP32 DEVKIT V1’)

• Model : Image Classification model
  (Project number : 29303)
  (Link : https://studio.edgeimpulse.com/public/29303/latest)

1. Importing Static Buffer Example
   I tried to import Static Buffer in Arduino.

static const float features[] = {

0.6627, 0.6235, 0.4549, 0.7608, 0.7490, 0.6471, 0.8588, 0.8353, 0.7569, 0.8549, 0.8471, 0.7686, …
// copy raw features here (for example from the ‘Live classification’ page)
// see https://docs.edgeimpulse.com/docs/running-your-impulse-arduino
};

I put raw data from Live Classification, as the Edge Impulse Documents says.

First Error message is like

Edge Impulse standalone inferencing (Arduino)
The size of your 'features' array is not correct. Expected 2304 items, but had 2000

So I randomly matched the item size. Then I got the error message is like

Is there solution for fix this error message?

2. Edge impulse Official git hub : example-esp32-cam
   Thank you for edge impulse I found the example that you offered.
   https://github.com/edgeimpulse/example-esp32-cam

I’ve downloaded the github and run in Arduino, I’ve got the error log

C:\ESP32\example-esp32-cam-main\Basic-Image-Classification\Basic-Image-Classification.ino: In function ‘esp_err_t page_handler(httpd_req_t*)’:
C:\ESP32\example-esp32-cam-main\Basic-Image-Classification\Basic-Image-Classification.ino:180:1: warning: no return statement in function returning non-void [-Wreturn-type]
}
^
In file included from C:\Users\NOTA1204\Documents\Arduino\libraries\ei-1-arduino-1.0.4\src/edge-impulse-sdk/dsp/speechpy/speechpy.hpp:27,
from C:\Users\NOTA1204\Documents\Arduino\libraries\ei-1-arduino-1.0.4\src/edge-impulse-sdk/classifier/ei_run_dsp.h:28,
from C:\Users\NOTA1204\Documents\Arduino\libraries\ei-1-arduino-1.0.4\src/edge-impulse-sdk/classifier/ei_run_classifier.h:31,
from C:\Users\NOTA1204\Documents\Arduino\libraries\ei-1-arduino-1.0.4\src/a1_inference.h:41,
from C:\ESP32\example-esp32-cam-main\Basic-Image-Classification\Basic-Image-Classification.ino:8:
C:\Users\NOTA1204\Documents\Arduino\libraries\ei-1-arduino-1.0.4\src/edge-impulse-sdk/dsp/speechpy/feature.hpp: In static member function ‘static int ei::speechpy::feature::mfe(ei::matrix_t*, ei::matrix_t*, ei::signal_t*, uint32_t, float, float, uint16_t, uint16_t, uint32_t, uint32_t, uint16_t)’:
C:\Users\NOTA1204\Documents\Arduino\libraries\ei-1-arduino-1.0.4\src/edge-impulse-sdk/dsp/speechpy/feature.hpp:323:5: error: insn does not satisfy its constraints:
}
^
(insn 2002 1866 30 72 (set (reg:SF 20 f1 [orig:266 prephitmp_402 ] [266])
(mem/u/c:SF (symbol_ref/u:SI ("*.LC258") [flags 0x2]) [0 S4 A32])) “C:\Users\NOTA1204\Documents\Arduino\libraries\ei-1-arduino-1.0.4\src/edge-impulse-sdk/dsp/speechpy/feature.hpp”:154 47 {movsf_internal}
(nil))
during RTL pass: postreload
C:\Users\NOTA1204\Documents\Arduino\libraries\ei-1-arduino-1.0.4\src/edge-impulse-sdk/dsp/speechpy/feature.hpp:323:5: internal compiler error: in extract_constrain_insn, at recog.c:2210
Please submit a full bug report,
with preprocessed source if appropriate.
See https://gcc.gnu.org/bugs/ for instructions.
exit status 1

Is there any solution about this problem?

Thanks
SunBeenMoon

[louis]

Hello @SunBeenMoon,

1. For your image processing (under the Create Impulse tab) the resize is set to 96 * 96 px.
   The ESP may not handle this size. Try to set 48x48 and retrain your model. It should solve the Failed to allocate tensor arena issue.

2. Could you try the Advance Image Classification sketch to see if you have the same issue?
   And are you able to compile the code if you choose the AI Thinker ESP32-CAM board?

Regards,

[janjongboom]

Note @louis the new MobileNetV1 models use less RAM even at 96x96 pixels, would be good to test on the ESP32

[SunBeenMoon]

Thank you for your reply @louis

1. Ok. I’ll retrain my model with 48*48.

2. I tried to compile the Advance Image Classification sketch with AI Thinker ESP32-CAM board.
   I stuck in the header issue.
   

   
   
   Do you know about this header issue?
   I’ve tried other example with using ‘esp_camer.h’ and ‘sensor.h’ header file, the example worked well.

Thanks,
SunBeen Moon

[louis]

Hello @SunBeenMoon,

You can have a look at the Framesize available here: https://github.com/espressif/esp32-camera/blob/master/driver/include/sensor.h

Can you try changing to FRAMESIZE_QVGA?

    FRAMESIZE_96X96,    // 96x96
    FRAMESIZE_QQVGA,    // 160x120
    FRAMESIZE_QCIF,     // 176x144
    FRAMESIZE_HQVGA,    // 240x176
    FRAMESIZE_240X240,  // 240x240
    FRAMESIZE_QVGA,     // 320x240
    FRAMESIZE_CIF,      // 400x296
    FRAMESIZE_HVGA,     // 480x320
    FRAMESIZE_VGA,      // 640x480
    FRAMESIZE_SVGA,     // 800x600
    FRAMESIZE_XGA,      // 1024x768
    FRAMESIZE_HD,       // 1280x720
    FRAMESIZE_SXGA,     // 1280x1024
    FRAMESIZE_UXGA,     // 1600x1200 

Best regards,

Louis

[SunBeenMoon]

Thank you for your reply @louis

I tried two examples

1. Basic-Image-Classification
   Select camera model : CAMERA_MODEL_ESP_EYE
   Change framesize : FRAMESIZE_240*240 -> FRAMESIZE_QVGA
   Progress : model uploaded in the board
   I’ve got the message like this.
   

[SunBeenMoon]

2. Advanced-Image-Classification
   Select camera model : CAMERA_MODEL_AI_THINKER
   Change framesize : FRAMESIZE_240*240 -> FRAMESIZE_QVGA
   Progress : model build failed
   I’ve got the memory problem.
   
   

I can’t find the solution about this problem.

[louis]

Hello @SunBeenMoon,

1. Can you make sure you used 9600 as the Serial Speed in the Serial Console?
   Note that on the Advanced Example, I’ve used 115200 on the serial console. You can change this directly in the code Serial.begin(115200); if you want to keep the same between the two models.

2. It seems that the model is too big, can you try with a different model?
   I would try with the MobileNet v2 0.05 or the MobileNet v1 0.01 or 0.02. It should reduce the size of your model.

Regards,

[louis]

And also, here is a tool I’ve been using a lot when debugging the ESP: https://github.com/me-no-dev/EspExceptionDecoder

[SunBeenMoon]

Thank you for your kind reply @louis.

I tried again in the direction you suggested. Thanks for giving advice in the direction of designing the model. By reducing the size(Use MobileNetV1) and making it 48*48, it was compiled and deployed on the board in both examples below.
However, when deploying on the board in both ‘Advanced-Image-Classification’ and ‘Basic Image Classification’ and viewing the result in a prompt window, the following error occurred.

[SunBeenMoon]

Thanks!!! I’ll try it.

[louis]

Unfortunately I do not have the same board you have to try.

Have you tried to set #define CAMERA_MODEL_ESP_EYE instead of #define CAMERA_MODEL_AI_THINKER in the Advanced-Image-Classification sketch

On your previous message it seems that you are not using your board definition:

@SunBeenMoon

Select camera model : CAMERA_MODEL_AI_THINKER

Also, could you try to run the default example called CameraWebServer.ino (it should be provided with the ESP board):

Let me know how it goes

Regards

[SunBeenMoon]

Hi,
Thank you for your reply @louis

First, I installed EspExceptionDecoder that you told me. Thank you very much. By the way, I have one question. When you press Esp Decoder in Tool.Tell them to add an elf file. Is this tool not available in the process of compilation, but is it available after compiling and deploying on the board?

And I got the library conflict problem. (Other example got the same problem)

Board Manager version : 2.0.0-alpha1
I choose : ESP-Dev-Module

Could you give me some advice for this compiling problem?

[louis]

Hey @SunBeenMoon,

I tried to compile with the board ESP-Dev-Module and it works fine…

Maybe try to uninstall the ESP32 Arduino boards from your board manager and install it again.
Let see if it can compile then…

And for the ESP Exception Decoder usually what I do is:

• Copy the trace from the Serial Console (after the code has been deployed, when the module encounter an error while running)
• Paste the trace in the tool
• Check the code at the location given by the ESP Exception Decoder

I hope this can help,

Regards,

Louis

[SunBeenMoon]

Wow, @louis
Thank you so much! As shown in the picture below, two examples (Basic-Image-Classification and Advanced-Image-Classification) were operated.

However, if you go to ‘http://172.20.10.6’ as stated above, you don’t go to the website.
By any chance, what am I doing wrong?

[louis]

@SunBeenMoon, you confirm that your laptop is on the same network as your ESP?
Maybe it is a network config, can you try another network (by sharing the connection with your phone for instance). It happened some times with my students and the university network (which is too restrictive).

Regards,

[SunBeenMoon]

Thank you for your kind reply @louis

I’ll do it again in some other place with less regulated Wi-Fi.

Regards,