Question/Issue:
I have created a FOMO image classification model and exported it as an Arduino Library. I am following this tutorial. I have successfully compiled and uploaded the model to the board. Better still, I can see from the serial output from one of the examples (“nicla_vision_camera”) that the model can successfully classify the object I’m looking for, which is a box shape. That shows me things are very close to working properly!
The issue is that following this first classification, I get a stream of errors:
ERR: failed to allocate tensor arena
Failed to initialize the model (error code 1)
ERR: Failed to run classified (-6)
Project ID:
672920
Context/Use case:
I am getting the same errors whether I run the “static_buffer” example, or, the “nicla_vision_camera” example. Both throw the same errors above. However, the “nicla_vision_camera” example does run properly at least once before throwing those errors after detecting an object.
Upon pressing the board’s reset button (or setting fin 5 on the board to low), the classification model successfully runs again, but encounters the same issue upon detecting an object. Perhaps something needs to be included in the code to reset the board? Or, perhaps there is an issue with the memory allocation?
Steps Taken:
Export arduino library and run per tutorial steps.
Open Examples >> Examples from custom library >> Compile and run the “nicla_vision_camera.ino” or “static_buffer”. Note: I’ve made no edits to these other than to paste in the features to the static buffer example per instructions.
See above photo… errors only for the static buffer example, but initial success followed by errors for the “nicla_vision_camera.ino” example.
Expected Outcome:
After successfully ID the object, I’d expect the serial output to return to it default state, again trying to run the model and find an object.
The only work around I currently have available is to trigger the reset pin, which seems clunky.
Actual Outcome:
Repeated errors as above
Reproducibility:
[ X] Always
[ ] Sometimes
[ ] Rarely
Environment:
Platform: Arduino Nicla Vision
Build Environment Details: [Arduino IDE 2.3.4
OS Version: Windows 11
Edge Impulse Version (Firmware): In powershell typing edge-impulse-daemon --version gives “Edge impulse runner v1.30.6”
Edge Impulse CLI Version: How do I determine this? How is it different from above?
Project Version: 1.0.0 initial commit
Custom Blocks / Impulse Configuration: No custom blocks
Still hoping for a response, but in the absence of one I’m going to head down the route of working with what I have, and triggering a reset of the board once a detection is made. Fingers crossed that someone will be able to provide a more practical path to troubleshoot and resolve this.
@BrianMacG try defining EI_CLASSIFIER_ALLOCATION_STATIC 1
for your entire project. This will allocate the tensor at compile time rather than runtime via heap. If you’re right on the bubble RAM wise, you might be getting bit by heap fragmentation. It won’t make a difference to final footprint unless you’re malloc/free something large in between running inference.
I should say that Ive been attracted to exploring this in part by the notion that those with little coding experience can interact with your platform and have functional results… unfortunately that describes me… Im not yet well versed in memory allocation (malloc, free etc). I think I’m on the cusp of success, but, could I ask for a few more specific instructions on how exactly i should go about defining that?
Do you mean including a specific line of code in the sketch, and if so what exaclty? E.g. “#define EI_CLASSIFIER_ALLOCATION_STATIC 1;” at the start of my sketch? Are other changes to ither files required?
(i) within the start of the arduino sketch
(ii) within the “boards.txt” file i found in "C:\Users<USER>\AppData\Local\Arduino15\packages\arduino\hardware\mbed_nicla\4.2.4
(iii) with in a “boards.locat.txt” file in that same folder mentioned above.
I am not seeing a change in the behavior. I still get excellent performance first time detecting followed by that error. Any suggestions for next trouble shooting steps? Would reducing the number or type of images in the training dataset I created help? Perhaps I should reduce the image size, which is currently 96x96?
Is there anyway to just start over by writing a software command. Right now my hack is to use a second arduino to trigger the reset on the vision, and that’s less than elegant!
Hi, @BrianMacG
It does indeed look like heap fragmentation issue.
For EI_CLASSIFIER_ALLOCATION_STATIC, I think in case with Arduino library, you need to add it to boards.local.txt as described here Advanced Arduino IDE Configuration – Rick's Measurement for Mechatronics Notes for example (Injecting Definitions at Compile Time).
There is yet another way to force static allocation: in your downloaded deployment (the Arduino library you downloaded from Studio and placed in your Arduino libraries folder) find src/tflite-model/tflite_learn_22_compiled.cpp (the number will be different). Open the file and insert #define EI_CLASSIFIER_ALLOCATION_STATIC 1
at the top of the file, right after copyright note.
Great guidance, thank you! I have found the file you mentioned and added the line you provided. Following that I have recompiled my arduino sketch but I am unfortunately running into the same issue.
I appreciate all the help. I am going to focus on how I might reduce the memory requirements of my exported impulse. I’ve already exported the int8 quantised version and it’s 239.5K. I did that with the EON Compiler. I don’t have access to the enterprise version which can export smaller code. Should I try the tensorflowlite compiler as a next step?
There is no such option as “tensorflowlite compiler” in Edge Impulse, you can have either EON Compiler or Tensorflow Lite - the latter being optimized version, so even more RAM is used.
I’ll try your project with Nicla Vision this week to see if I can reproduce the issue.
Hi there, and thanks for continuing to try to help. In truth I’m getting a bit desperate! It is possible to pay for an account type that could get me more support on this?
Recent activities:
I’ve started a new project number from scratch to try to resolve this: see version 2 of project ID 618058.
I’ve also purchased another Arduino Nicla Vision board and I am continuing efforts to get some level of object detection functional. I just need to demonstrate basic object detection to successfully complete an academic project.
I have scaled back my model so that it is now trying to detect only 1 item, and I have set the image to black and white rather than RGB. I have 30 images of 96x96 with the item to be detected, and 30 of background.
I have made efforts to implement the “static ei classifier fix mentioned above”, however the instructions I am getting are not clear and I am not confident that I have implemented it correctly. Or, I have implemented it and it doesn’t work.
Unfortunately, I am seeing similar behavior still. Typically, the detection seems to run once, but then runs into a “Failed to run classifier” error.
The images below illustrate the issue, which is that the board is initially working, detecting an object, but then is stalling/crashing, or erroring out.
Based on the size of the code (227,000 bytes) I don’t think this is a memory issue, but cant be certain.
My next step is to abandon the object detection impulse and attempt again with an image classification impulse.
Thanks! Any guidance you can give is much appreciated.
@BrianMacG ,
I’ve attempted to reproduce the error you are describing on my Nicla Vision with Arduino Mbed Nicla Boards package version 4.1.5 (the version we use for building Nicla Vision firmware in Studio).
The deployment from your project 618058 (image classification), works correctly for me - the inference runs for prolonged periods of time.
The deployment from your project 672920 (object detection FOMO), exhibits the same issue as you described, failure to allocate arena following a successful single inference. Interesting thing is that if there are no objects detected, there is no error. Makes me think the issue is not in the inferencing code, but rather post-processing.
I’m back from PTO and will be investigating it.
Okay, to unblock you, there is a workaround:
In src/edge-impulse-sdk/classifier/ei_fill_result_struct.h (which you find inside your Arduino folder/libraries/name of your deployment)
find fill_result_struct_from_cubes function
and change
static std::vector<ei_impulse_result_bounding_box_t> results;
to
std::vector<ei_impulse_result_bounding_box_t> results;
After this change, the inference runs uninterrupted for me with your object detection deployment.
However this is not a solution I can push to production, since we actually want this vector to be static, so it and the data inside of it persists between inferences.
Let me know if the workaround can unblock you.
What can I say… that worked like a CHARM!! I’m thrilled.
Ok, to be specific. My image classification inference went from “Failed to run classified (-6)” to working perfectly. I have not tried it with the Object detection effort, but for now that is all I need. I’m back on track for this project!
