Esp32 cam object detection to gpio high help

Help
#1

Hi again! I would like help in how to best learn how to do the following:

I want to train a esp32 cam (that I flashed with edge impulse fw) to recognise an object. When the object is recognized I want the esp32 to make a GPIO pin HIGH and send a signal to another board that plays audio (this board can be triggered with a pulse from a microcontroller). I think I know how to train the actual model (tried before with a raspberry pi, so im comfortable with the edge impulse studio). My question is:

How and where do I write the code so that I can make the board send a digital high on a gpio output pin when the model inferes that my trained object is in the image?

I am a coding beginner, so any tips on how I can learn to do this is appreciated a lot. It is for a design school project (I am a design student, but we mostly work with traditional materials and techniques so I can not ask school teachers for help on this). I only know a little bit of basic python and micropython, so I would very much be happy if someone could explain in beginners terms how to code this so it works with the edge impulse fw and ML model.

Thanks,
Jonas

1 Like
#2

Hello @Instant_exit,

You might want to have a look at this thread:

The concept is basically the same, instead of an LED, just add whatever GPIO you want.

Best,

Louis

1 Like
#3

Ok, thank you Louis! I think I understand the concept. But how do I adapt this to work on object detection using the camera? (With FOMO?)

Also: If using the edge impulse firmware for esp32, can I still use .ino files? (programming the esp32 cam with the arduino ide) Or should I use stock esp32 cam fw and connect esp32 with arduino ide? (I believe you wrote a tutorial on this earlier, but I dont know if its still relevant since you added support and custom fw for esp32 boards).

Please keep in mind, I am a beginner to most of this. I have dabbled a bit with raspberry pi and python, but I am pretty much learning from scratch here.

I just want a gpio to turn high when a detection of an object is made. Say i train my model on dogs. Whenever i see a dog i want to turn the gpio to High. I have a feeling this should be not that hard (hoping at least!) :slight_smile:

#4

Hello @Instant_exit,

Indeed you can achieve that using FOMO.

If you download the Arduino Library of your project from the deployment page, include that library in Arduino IDE, you can then load an example sketch (.ino). Search for the esp32-camera example.

At the beginning of your file, declare your GPIO

#define YOUR_GPIO 4 // or whatever pins it is attached to

Then in your setup function add:

pinMode(YOUR_GPIO, OUTPUT);

And then in your loop function, after you get the inference results add something like:

#if EI_CLASSIFIER_OBJECT_DETECTION == 1
    bool bb_found = result.bounding_boxes[0].value > 0;
    for (size_t ix = 0; ix < result.bounding_boxes_count; ix++) {
        auto bb = result.bounding_boxes[ix];
        if (bb.value == 0) {
            continue;
        }
        ei_printf("    %s (%f) [ x: %u, y: %u, width: %u, height: %u ]\n", bb.label, bb.value, bb.x, bb.y, bb.width, bb.height);

        //Add your custom logic here for example
        if (bb.label == "dog" && bb.value >= 0.8) {
            digitalWrite(YOUR_GPIO, HIGH);
        }
    }
    if (!bb_found) {
        ei_printf("    No objects found\n");
    }

Here your GPIO will stay on the HIGH state forever once it detected a dog with a probability higher than 80%, make sure to turn it off at the end of your loop or after a certain time.

I hope that helps.

Best,

Louis

#5

That is very helpful! Thanks! :smiley: Now I just want to know if I should use the firmware (modified to fit esp32 cam) you supply for this, or use the stock fw? I have two esp32 here, one has the modified edge impulse fw, the other is stock atm.

I can use your modified esp32 fw to gather data from the actual camera, which is super helpful, so thanks for that!

/Jonas

#6

When you will flash your ESP32-Cam with the Arduino sketch, it will erase the official EI firmware.
So if you have two boards, keep one to send the data and the other to test your end project.

Once you will be comfortable with building Arduino/ESP programs, you can also write a piece of code or a function to upload the pictures to your project using Edge Impulse API (or to a cloud bucket and then retrieve those data in your project using the data pipelines).

Best,

Louis

1 Like