How can I use serial communication between my esp-32 cam once it has detected an object to ask the uno r3 to do something?

Help
, ,
#1

Question/Issue:
How do I make my esp32 cam command my uno r3 to move motors through serial communication after the esp 32 cam has detected an object above 0.995 confidence for 3 times in a row?(Hardware and Software issue)

Project ID:
601271

Context/Use case:
So my esp-32 cam is recognizing what it’s supposed to recognize pretty easily, but once it has detected something for 0.995 and above confidence for 3 times in a row, I tried to code it to communicate to the arduino to move some motors, however the code just stops working.

Steps Taken:

  1. I tried to do simple serial communication through a youtube tutorial, although the esp32 cam was saying hello and arduino saying message received they did not come up in the same serial monitor, just in the same baud rate but not in the same serial monitors. I should also mention that the esp 32 cam and arduino were in different ports.
  2. I tried a bunch of different schematics online to connect the esp 32 and arduino directly to see if it was a hardware issue in terms of connectivity, but the functionality of the aforementioned program was worse, the arduino was working saying hello, but my esp32 stop saying message received.
  3. I tried using ChatGPT to code for me for the original project but as always it didn’t work, this step was a done in a bit of frustration.

Expected Outcome:
Esp-32 cam detects something for 0.995 and above confidence 3 times in a row → arduino starts moving motors and executes the program.

Actual Outcome:
program stops working entirely

Reproducibility:

  • [ ] Always

Environment:

  • Platform: [Uno R3]
  • Build Environment Details: [e.g., Arduino IDE 1.8.19 ESP32 Core for Arduino 2.0.4]
  • OS Version: [Windows 11]
  • Edge Impulse Version (Firmware): [e.g., 1.2.3]
  • To find out Edge Impulse Version:
  • if you have pre-compiled firmware: run edge-impulse-run-impulse --raw and type AT+INFO. Look for Edge Impulse version in the output.
  • if you have a library deployment: inside the unarchived deployment, open model-parameters/model_metadata.h and look for EI_STUDIO_VERSION_MAJOR, EI_STUDIO_VERSION_MINOR, EI_STUDIO_VERSION_PATCH
  • Edge Impulse CLI Version: [e.g., 1.5.0]
  • Project Version: [1.0.2]

ESP32 CAM code:
// Include necessary headers
#include <Arduino.h>

// Add global variables
const float CONFIDENCE_THRESHOLD = 0.995;
const int REQUIRED_CONSECUTIVE_DETECTIONS = 3;
int detection_count = 0;
String last_detected_label = “”;

// Initialize serial communication with Arduino
void setup() {
Serial.begin(115200);
Serial1.begin(9600); // For communication with Arduino Uno
while (!Serial);
Serial.println(“Edge Impulse Inferencing Demo”);

if (ei_camera_init() == false) {
    ei_printf("Failed to initialize Camera!\r\n");
} else {
    ei_printf("Camera initialized\r\n");
}

ei_printf("\nStarting continuous inference in 2 seconds...\n");
ei_sleep(2000);

}

void loop() {
if (ei_sleep(5) != EI_IMPULSE_OK) {
return;
}

snapshot_buf = (uint8_t*)malloc(EI_CAMERA_RAW_FRAME_BUFFER_COLS * EI_CAMERA_RAW_FRAME_BUFFER_ROWS * EI_CAMERA_FRAME_BYTE_SIZE);
if (snapshot_buf == nullptr) {
    ei_printf("ERR: Failed to allocate snapshot buffer!\n");
    return;
}

ei::signal_t signal;
signal.total_length = EI_CLASSIFIER_INPUT_WIDTH * EI_CLASSIFIER_INPUT_HEIGHT;
signal.get_data = &ei_camera_get_data;

if (ei_camera_capture((size_t)EI_CLASSIFIER_INPUT_WIDTH, (size_t)EI_CLASSIFIER_INPUT_HEIGHT, snapshot_buf) == false) {
    ei_printf("Failed to capture image\r\n");
    free(snapshot_buf);
    return;
}

ei_impulse_result_t result = { 0 };
EI_IMPULSE_ERROR err = run_classifier(&signal, &result, debug_nn);
if (err != EI_IMPULSE_OK) {
    ei_printf("ERR: Failed to run classifier (%d)\n", err);
    return;
}

// Process object detection results
for (uint32_t i = 0; i < result.bounding_boxes_count; i++) {
    ei_impulse_result_bounding_box_t bb = result.bounding_boxes[i];
    if (bb.value >= CONFIDENCE_THRESHOLD) {
        if (last_detected_label == bb.label) {
            detection_count++;
        } else {
            last_detected_label = bb.label;
            detection_count = 1; // Reset count for a new label
        }

        if (detection_count >= REQUIRED_CONSECUTIVE_DETECTIONS) {
            Serial1.println("ACTIVATE_MOTOR"); // Signal to Arduino
            detection_count = 0; // Reset detection count
        }
    }
}

free(snapshot_buf);

}

UNO R3 code:

#include <Servo.h> // Example for controlling servos

Servo motor1; // Define motors
Servo motor2;

void setup() {
Serial.begin(9600); // Communication with ESP32-CAM
motor1.attach(9); // Attach motors to pins
motor2.attach(10);
}

void loop() {
if (Serial.available() > 0) {
String message = Serial.readStringUntil(‘\n’);

    if (message == "ACTIVATE_MOTOR") {
        activateMotors();
    }
}

}

void activateMotors() {
// Example motor movement havent gotten past the esp32 cam being able to properly communicate
motor1.write(90); // Rotate motor 1
motor2.write(180); // Rotate motor 2
delay(1000); // Hold for 1 second
motor1.write(0); // Reset motors
motor2.write(0);
}