Transfer Learning Classification Model works fine for some inferences and then “gets stuck” at 1.00 probability

Help
, ,
#1

Hello there,
i am trying to run a classification model (160 x 160 rgb using transfer learning) to distinguish between apples, banana and potatoes as a test project for image classification.
I am using the Arduione IDE and the Arduino Library for deployment in the quantized int8 version.

The inferencing works good for a few runs and switching between images of the 3 classes will produce the desired classification.
Then it seems like the board gets stuck on one classification at 1.00 probability.

I am using an seeed xiao esp32 s3 cam board - this is the project in question:

Does anybody has an idea what might be at play here?

This is the Serial Monitor Output:
Camera OK
Put object in front of camera
Classified as: banana with probability: 0.48 Resetting Frame Now
Classified as: potato with probability: 0.70 Resetting Frame Now
Classified as: potato with probability: 0.98 Resetting Frame Now
Classified as: potato with probability: 0.98 Resetting Frame Now
Classified as: apple with probability: 0.99 Resetting Frame Now (which is actually an apple)
Classified as: apple with probability: 0.99 Resetting Frame Now (which is actually an apple)
Classified as: potato with probability: 1.00 Resetting Frame Now
Classified as: potato with probability: 1.00 Resetting Frame Now
Classified as: potato with probability: 1.00 Resetting Frame Now
Classified as: potato with probability: 1.00 Resetting Frame Now (still potato, although already switching back to images of bananas or apples)
…
it will stay potato with 1.00 probability from here on

Thanks a lot

#2

Hi @clemens_edge

You are running out of resources somehow, and must clear. What example did you base this on continuous inference?

Questions to clarify and troubleshoot:

  1. Example Base: Did you base your implementation on an example project or tutorial? Knowing this can help pinpoint where the issue might be arising, especially if there’s a difference in resource management between your implementation and the example.
  2. Resource Monitoring: Are you able to monitor the memory usage on your ESP32 during inference? It’s possible that memory isn’t being freed correctly between inferences, leading to eventual exhaustion.
  3. Continuous Inference Strategy: If you’re running continuous inference (i.e., running the model in a loop without restarting the board), the model might not be clearing out previous inferences correctly, causing it to get stuck. Make sure that your implementation clears any buffers or states that could be holding on to previous inference data.
  4. Model Size and Complexity: The complexity of your model could also be a factor. If your model is too large or complex for the ESP32 to handle continuously, consider simplifying it or using a more efficient model architecture.
  5. Memory Management: In embedded environments, careful management of memory is crucial. Ensure that your inference loop properly manages memory by:
  • Clearing or resetting buffers after each inference.
  • Avoiding memory leaks by checking if any dynamically allocated memory is being freed correctly.

Best

Eoin