Running Continuous Gesture Classification Locally

#1

I am currently working on the continuous gesture classification example. I have completed the impulse, but I am having some issues with running the impulse. There is documentation for running the impulse locally (https://docs.edgeimpulse.com/docs/running-your-impulse-mbed), however, this documentation requires raw features and will not run continuously. Could someone help me explain how to run a continuous classification program on my mbed enabled board?

#2

Hi @Andre923, yeah, we wanted to keep the Mbed example as generic as possible. But you can do this by filling the buffer with data collected by the accelerometer, and then call the run_classifier function continuously.

F.e. for the ST IoT Discovery Kit, do:

   static float accel_data[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE] = { 0 };

   // 1. capture raw data into the buffer
   Timer timer;
   timer.start();

   printf("Sampling data...\n");

   size_t accel_data_ix = 0;

   // capture data...
   while (1) {
       int64_t next_tick = timer.read_us() + static_cast<int64_t>(EI_CLASSIFIER_INTERVAL_MS * 1000);

       int16_t accel_data_i16[3] = { 0 };
       BSP_ACCELERO_AccGetXYZ(accel_data_i16);

       accel_data[accel_data_ix++] = static_cast<float>(accel_data_i16[0]) / 100.0f;
       accel_data[accel_data_ix++] = static_cast<float>(accel_data_i16[1]) / 100.0f;
       accel_data[accel_data_ix++] = static_cast<float>(accel_data_i16[2]) / 100.0f;

       // end of the buffer
       if (accel_data_ix == EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE) {
           break;
       }

       // let's sleep for (wait_time / 1000) - 1 ms. then busy loop from there
       uint64_t wait_time = next_tick - timer.read_us();

       // sleep OK (/1000 already floors it)
       ThisThread::sleep_for((wait_time / 1000) - 1);

       // busy loop til next tick
       while (next_tick > timer.read_us());
   }

   // run inferencing on the data
   signal_t features_signal;
   int r = numpy::signal_from_buffer(accel_data, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, &features_signal);
   if (r != 0) {
      printf("Failed to convert buffer into signal (%d)\n", r);
      return;
   }

   // invoke the impulse
   ei_impulse_result_t result;
   EI_IMPULSE_ERROR res = run_classifier(&features_signal, &result, false);

   // do something with the result

Then call this in a loop.

This however still captures data for the full window and then classifies it. To make this more robust you can use a sliding window to go over the buffer and do more inferences that way over the same data. There’s an example here: https://www.edgeimpulse.com/blog/adding-machine-learning-to-your-lorawan-device/ that does that, and I’ve explained some strategies of doing this also in https://www.edgeimpulse.com/blog/audio-based-shower-timer-with-a-phone-machine-learning-and-webassembly/.

Continuous Gesture Classification Real-time adquisition
#3

Thanks for the quick response! I have implemented the code, but now I get some compiling errors:

[ERROR] In file included from ./edge-impulse-sdk/dsp/spectral/processing.hpp:28:0,
from ./edge-impulse-sdk/dsp/spectral/spectral.hpp:27,
from ./edge-impulse-sdk/classifier/ei_run_dsp.h:27,
from ./edge-impulse-sdk/classifier/ei_run_classifier.h:30,
from .\source\main.cpp:2:
./edge-impulse-sdk/dsp/spectral/…/numpy.hpp: In static member function ‘static float ei::numpy::log(float)’:
./edge-impulse-sdk/dsp/spectral/…/numpy.hpp:1165:39: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
g = (int32_t) * ((int32_t )&a);
^
./edge-impulse-sdk/dsp/spectral/…/numpy.hpp:1168:35: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
m = (float) * ((float *)&g);
^
.\source\main.cpp: In function ‘int main()’:
.\source\main.cpp:55:10: error: return-statement with no value, in function returning ‘int’ [-fpermissive]
return;
^~~~~~
.\source\main.cpp:62:40: warning: format ‘%s’ expects argument of type 'char’, but argument 3 has type ‘ei_impulse_result_t’ [-Wformat=]
pc.printf(“Result: %s \n”, result);

For the top 2 errors, they seem to be problems with the natural log estimation in numpy.hpp. For the last error, I don’t know what variable type the impulse returns. Is it a string with the gesture classified?

#4

@Andrew923, ythe above two are warnings, but should be OK. The latter one is because your code is in the main function, which requires to return an integer. E.g. change the return; in my example with return 1;.