Running Continuous mode sketch inferencing stops after some time

#1

I took the example of the accelerometer contionous example for arduino.

I am using it on raspberrry pi pico.

I was able to run the static buffer example without issues.

Instead of accelerometer data i get 3 values from my vl530x sensor and pass them to the buffer.

seems that the first inferences are good but then it just prints default values that never change

Starting... Reset of sensors
Starting Left...
Starting Center..
Starting Right..
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): uncertain  [ 0, 1, 0, 9, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): uncertain  [ 0, 2, 0, 8, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): uncertain  [ 0, 3, 0, 7, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): uncertain  [ 0, 4, 0, 6, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): uncertain  [ 0, 5, 0, 5, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): uncertain  [ 0, 6, 0, 4, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 7, 0, 3, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 8, 0, 2, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 9, 0, 1, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 10, 0, 0, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 10, 0, 0, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 10, 0, 0, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 10, 0, 0, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 10, 0, 0, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 10, 0, 0, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 10, 0, 0, 0, ]
Predictions (DSP: 15 ms., Classification: 1 ms., Anomaly: 0 ms.): noise  [ 0, 10, 0, 0, 0, ]```

any clues? maybe has to do something with the buffer ?
#2

Hey buton,

Would you provide the code snippet you are using to pass to the buffer and create the signal type? I’m wondering if there’s some accidental mis-allocation going on.

If you replace that sensor sampling with a fixed set of input features in the same static buffer, does inferencing still fail after some time?

Best,
David

#3

Hi @daschwar

Find the attached code

thanks

//continous
#include <Arduino.h>
#include "C:/Users/jorge/OneDrive/Documents/PlatformIO/Projects/RpiPico/.pio/libdeps/pico/Adafruit_VL53L0X/src/Adafruit_VL53L0X.h"
#include "C:\Users\jorge\OneDrive\Documents\PlatformIO\Projects\RpiPico\lib\Gestures_inferencing\src\Gestures_inferencing.h"


// addresses we will assign  to the sensors
#define XL_ADDRESS 0x30 //Left Sensor
#define XC_ADDRESS 0x31 //Center Sensor
#define XR_ADDRESS 0x32 // Right Sensor

// set Shutdown pins
#define SHT_XL 2
#define SHT_XC 3
#define SHT_XR 4
/*

/// Arduino default on raspberry pi GPIO

// SPI
#define PIN_SPI_MISO  (4u)
#define PIN_SPI_MOSI  (3u)
#define PIN_SPI_SCK   (2u)
#define PIN_SPI_SS    (5u)

// Wire
#define PIN_WIRE_SDA        (6u) GPIO 6
#define PIN_WIRE_SCL        (7u) GPIO 7
*/


// objects for the vl53l0x
Adafruit_VL53L0X XL = Adafruit_VL53L0X();
Adafruit_VL53L0X XC = Adafruit_VL53L0X();
Adafruit_VL53L0X XR = Adafruit_VL53L0X();
// this holds the measurement
VL53L0X_RangingMeasurementData_t measure1;
VL53L0X_RangingMeasurementData_t measure2;
VL53L0X_RangingMeasurementData_t measure3;

float R=0;
float C=0;
float L=0;

/* Private variables ------------------------------------------------------- */

static bool debug_nn = false; // Set this to true to see e.g. features generated from the raw signal
static uint32_t run_inference_every_ms = 200;
static rtos::Thread inference_thread(osPriorityLow);
static float buffer[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE] = { 0 };
static float inference_buffer[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE];

/* Forward declaration */
void run_inference_background();

//--Edge impulse print
void ei_printf(const char *format, ...) {
   static char print_buf[1024] = { 0 };

   va_list args;
   va_start(args, format);
   int r = vsnprintf(print_buf, sizeof(print_buf), format, args);
   va_end(args);

   if (r > 0) {
       Serial1.write(print_buf);
   }
}



void SetAddressSensors()
{

  //-- set shutdown pins as ouputs
  pinMode(SHT_XL, OUTPUT);
  pinMode(SHT_XC, OUTPUT);
  pinMode(SHT_XR, OUTPUT);

   Serial1.println("Starting... Reset of sensors");
   // Deactivate all Sensors
  digitalWrite(SHT_XL, LOW);    
  digitalWrite(SHT_XC, LOW);
  digitalWrite(SHT_XR, LOW);

  delay(10);

  // activating XL Left Sensor
  digitalWrite(SHT_XL, HIGH);
  digitalWrite(SHT_XC, LOW);
  digitalWrite(SHT_XR, LOW);
 
  delay(10);
   
   // initing Left
  Serial1.println("Starting Left...");
  if(!XL.begin(XL_ADDRESS,true)) {
    Serial1.println(F("Failed to boot first VL53L0X"));
    while(1);
  }

  delay(10);


  // activating XC Center Sensor
  digitalWrite(SHT_XC, HIGH);
  delay(10);

   // initing Center Sensor
   Serial1.println("Starting Center..");
  if(!XC.begin(XC_ADDRESS,true)) {
    Serial1.println(F("Failed to boot first VL53L0X"));
    while(1);
  }

  delay(10);

  // Activating XR Right Sensor
  digitalWrite(SHT_XR, HIGH);
  delay(10);

   // initing Right
  Serial1.println("Starting Right..");
  if(!XR.begin(XR_ADDRESS,true)) {
    Serial1.println(F("Failed to boot first VL53L0X"));
    while(1);
  }
  Serial1.println("All Sensors Initiated..");


}

void run_inference_background()
{
    // wait until we have a full buffer
    delay((EI_CLASSIFIER_INTERVAL_MS * EI_CLASSIFIER_RAW_SAMPLE_COUNT) + 100);

    // This is a structure that smoothens the output result
    // With the default settings 70% of readings should be the same before classifying.
    ei_classifier_smooth_t smooth;
    ei_classifier_smooth_init(&smooth, 10 /* no. of readings */, 7 /* min. readings the same */, 0.8 /* min. confidence */, 0.3 /* max anomaly */);

    while (1) {
        // copy the buffer
        memcpy(inference_buffer, buffer, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE * sizeof(float));

        // Turn the raw buffer in a signal which we can the classify
        signal_t signal;
        int err = numpy::signal_from_buffer(inference_buffer, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, &signal);
        if (err != 0) {
            ei_printf("Failed to create signal from buffer (%d)\n", err);
            return;
        }

        // Run the classifier
        ei_impulse_result_t result = { 0 };

        err = run_classifier(&signal, &result, debug_nn);
        if (err != EI_IMPULSE_OK) {
            ei_printf("ERR: Failed to run classifier (%d)\n", err);
            return;
        }

        // print the predictions
        ei_printf("Predictions ");
        ei_printf("(DSP: %d ms., Classification: %d ms., Anomaly: %d ms.)",
            result.timing.dsp, result.timing.classification, result.timing.anomaly);
        ei_printf(": ");

        // ei_classifier_smooth_update yields the predicted label
        const char *prediction = ei_classifier_smooth_update(&smooth, &result);
        ei_printf("%s ", prediction);
        // print the cumulative results
        ei_printf(" [ ");
        for (size_t ix = 0; ix < smooth.count_size; ix++) {
            ei_printf("%u", smooth.count[ix]);
            if (ix != smooth.count_size + 1) {
                ei_printf(", ");
            }
            else {
              ei_printf(" ");
            }
        }
        ei_printf("]\n");

        delay(run_inference_every_ms);
    }

    ei_classifier_smooth_free(&smooth);
}
void setup() {


  Serial1.begin(115200);  
  Serial1.println("Entering Setup");
  delay(2000);



  // Initialize  Led Pin
  pinMode(LED_BUILTIN, OUTPUT);
  digitalWrite(LED_BUILTIN, HIGH);

  //--Enumarate sensors and start
  SetAddressSensors();

   if (EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME != 3) {
        ei_printf("ERR: EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME should be equal to 3 (the 3 sensor axes)\n");
        return;
    }
     inference_thread.start(mbed::callback(&run_inference_background));

}

void loop() {
      


while (true) {
      digitalWrite(LED_BUILTIN, HIGH);
     
        // Determine the next tick (and then sleep later)
       uint64_t next_tick = micros() + (EI_CLASSIFIER_INTERVAL_MS * 1000);
  
        // roll the buffer -3 points so we can overwrite the last one
        numpy::roll(buffer, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, -3);
        // read to the end of the buffer

        //- Get sensor data
        XL.rangingTest(&measure1,false); // pass in 'true' to get debug data printout!
        L=measure1.RangeMilliMeter; 
        XC.rangingTest(&measure2,false); // pass in 'true' to get debug data printout!
        C=measure2.RangeMilliMeter;
        XR.rangingTest(&measure3,false); // pass in 'true' to get debug data printout
        R=measure3.RangeMilliMeter;
          
          // print sensor thre reading 
         /* Serial1.print(L);
          Serial1.print(",");
          Serial1.print(C);
          Serial1.print(",");
          Serial1.println(R);
         */
        
          buffer[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE - 1]=R;
          buffer[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE - 3]=L; 
          buffer[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE - 2]=C; 
        

         // and wait for next tick
        uint64_t time_to_wait = next_tick - micros();
        delay((int)floor((float)time_to_wait / 1000.0f));
        delayMicroseconds(time_to_wait % 1000);
        digitalWrite(LED_BUILTIN, LOW);

    }

  
 
}```
#4

Based on the output the inference thread is continuing to run, but it doesn’t block on waiting for the sampling to fully fill the buffer, so likely your sensor sampling loop is stalling after a while and then the inferencing just continues on periodically.

You can confirm this by adding a heartbeat print, enabling debug logging on the sensor reading, or monitoring the LED you have set up. I expect you’ll see it stop toggling after a while. If this line of debugging is in the right direction, I’d next try to disable all other threads and see if the loop still stalls just printing and filling up the sensor buffer.

If you make it to that point and it still stalls, its likely something VL53L0X specific. I’m not too familiar with the sensor driver, but a skimming through the library there are a few examples that look to show different methods for initializing the sensor. If you haven’t already, it may be useful to test out the different ones and compare:

https://www.arduino.cc/reference/en/libraries/vl53l0x/

2 Likes