Integrating the trained model

Hi,
I’m doing a Gesture detection project in nRF52833 board with Segger embedded studio. I’ve connected the device to Edge Impulse through the data forwarder and trained a model. Now I’m integrating the C++ library into my project. I’ve forwarded the data to the features array but I’m getting the one gesture as always high.

Running neural network…
Predictions (time: 0 ms.):
Left-right: 0.99
Rotate: 0.00
Features (0 ms.): 0.00 0.00 0.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 0.00 0.00 0.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 0.00 0.00 0.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00

This is the log that I’m getting. I don’t know what value that feature is printing and from where it is printing, but the values are always the same, and I’m getting a prediction always high.
How can I solve this? Thanks in advance.

Hello @Nabeel,

Could you share how you passed the accelerometer data to the buffer, the issue is probably coming from there if it is always the same results.

Best,

Louis

Hello @louis,

void process_input(void) 
{
    static char sensor_values[10];
    sprintf(sensor_values,"%d,%d,%d\n", accX,accY,accZ);
    features[2] = sensor_values;
    signal_t signal;
    signal.total_length = FEATURES_SIZE;
    signal.get_data = &get_feature_data;

    ei_impulse_result_t result;
    EI_IMPULSE_ERROR res = run_classifier(&signal, &result, true);

    if (result.classification[0].value > 1) 
    {
      nrf_gpio_pin_set(LED2);
      nrf_gpio_pin_clear(LED1);
      NRF_LOG_INFO("Left-Right");
    }
    else if (result.classification[1].value > 0.9)
    {
      nrf_gpio_pin_set(LED1);
      nrf_gpio_pin_clear(LED2);
      NRF_LOG_INFO("Rotate");
    }
    else
    {
       NRF_LOG_INFO("Nothing");
       NRF_LOG_INFO("data received: %d" ,accX);
    }
}

This is the function that runs the classifier. And I gave if (result.classification[0].value > 1) because it always gives the value 0.99 and I want to execute the else part.
This function is called every time when the controller fetches the sensor data.

I am not sure as I don’t have the full code but maybe you want to have a look at this to get a better understanding of how we do with IMU data (at least for the logic): Data forwarder - Edge Impulse Documentation

Let me know if that helps.

Best,

Louis

Hi @louis,

Thanks for the reply. I’m a bit confused about passing the data to the features array. So I thought of sending the raw feature from the data collected in Edge Impulse to the Features array according to this article.

#include "edge-impulse-sdk/dsp/numpy_types.h"
#include "edge-impulse-sdk/porting/ei_classifier_porting.h"
#include "edge-impulse-sdk/classifier/ei_classifier_types.h"

static char features[FEATURES_SIZE];

int get_feature_data(size_t offset, size_t length, float *out_ptr) {
    memcpy(out_ptr, features + offset, length * sizeof(float));
    return 0;
}

EI_IMPULSE_ERROR run_classifier(signal_t *, ei_impulse_result_t *, bool);

void process_input(uint16_t length) 
{
    static const int features[] = {-12, 22, -23, -11, 23, -24, -10, 23, -24, .....};
    signal_t signal;
    signal.total_length = EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE;
    signal.get_data = &get_feature_data;

    ei_impulse_result_t result;

    // Calculate the length of the buffer
    size_t buf_len = sizeof(features) / sizeof(features[0]);

    // Make sure that the length of the buffer matches expected input length
    if (buf_len != EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE) {
        ei_printf("ERROR: The size of the input buffer is not correct.\r\n");
        ei_printf("Expected %d items, but got %d\r\n",
                EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE,
                (int)buf_len);
        return 1;
    }

    EI_IMPULSE_ERROR res = run_classifier(&signal, &result, true);

    if (result.classification[0].value > 1) 
    {
      nrf_gpio_pin_set(LED2);
      nrf_gpio_pin_clear(LED1);
      NRF_LOG_INFO("Left-Right");
    }
    else if (result.classification[1].value > 0.9)
    {
      nrf_gpio_pin_set(LED1);
      nrf_gpio_pin_clear(LED2);
      NRF_LOG_INFO("Rotate");
    }
    else
    {
       NRF_LOG_INFO("Nothing");
      NRF_LOG_INFO("data received: %d",buf_len); 
    }
}

This is the integration that I’ve made in the main.c. I call the process_input() function in a function where I fetch the sensor data. But still I’m getting the following log.

Features (0 ms.): 0.00 0.00 0.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 0.00 0.00 0.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 0.00 0.00 0.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 
Running neural network...
Predictions (time: 0 ms.):
Left-right:	0.99
Rotate:	0.00

Where I gave the raw feature data of rotate in the feature array but it still shows left_right as high. How can I solve this problem? Am I feeding the data correctly in the features array?

Hello @Nabeel,

I can see that the features buffer is declared twice:

static char features[FEATURES_SIZE];

and

static const int features[] = {...}

I am suspecting that the get_feature_data function uses the global variable and not the one scoped at your function level. Can you remove the declaration and only use:

features[] = {-12, 22, -23, -11, 23, -24, -10, 23, -24, .....};

instead of

static const int features[] = {-12, 22, -23, -11, 23, -24, -10, 23, -24, .....};

You might have an issue between your item types too but I’ll let you fix that.

Best,

Louis

@Nabeel You need to update the features variable.

  • In the 1st code listing you are updating the features array via features[2] = sensor_values. I am not sure what this is doing exactly but at least you are updating the features array.

  • In the 2nd code listing features is never getting updated.

    • When the line containing run_classifier() is executed it will call get_feature_data() that copies features in to a buffer that the Edge Impulse Inferencer uses to make predictions.

Maybe an easier to understand code listing is in nano_ble33_sense_accelerometer.ino. To view this file deploy an Arduino library and browse to the example folder. The INO file won’t run directly on your hardware but may give you guidance in how to code it.

Hi @louis @MMarcial ,

Apologies for my late reply. I’ve been working on it. Now I have changed my code to the following

#include "edge-impulse-sdk/dsp/numpy_types.h"
#include "edge-impulse-sdk/porting/ei_classifier_porting.h"
#include "edge-impulse-sdk/classifier/ei_classifier_types.h"

#define FEATURES_SIZE           294    //value of EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE is 294

static float features[FEATURES_SIZE];

int get_feature_data(size_t offset, size_t length, float *out_ptr) {
    memcpy(out_ptr, features + offset, length * sizeof(float));
    return 0;
EI_IMPULSE_ERROR run_classifier(signal_t *, ei_impulse_result_t *, bool);

void process_input(uint16_t length) 
{
    for (uint32_t i = 0; i < length; i++)
    {         
      features[i] = m_cdc_data_array[i];
    }   
       
    signal_t signal;
    signal.total_length = FEATURES_SIZE;
    signal.get_data = &get_feature_data;
    
    ei_impulse_result_t result;
    EI_IMPULSE_ERROR res = run_classifier(&signal, &result, true);
    
    // Print return code and how long it took to perform inference
    printf("run_classifier returned: %d\r\n", res);
    printf("Timing: DSP %d ms, inference %d ms, anomaly %d ms\r\n", 
            result.timing.dsp, 
            result.timing.classification, 
            result.timing.anomaly);

    if (result.classification[0].value > 1)
    {
      nrf_gpio_pin_set(LED2);
      nrf_gpio_pin_clear(LED1);
      NRF_LOG_INFO("Left-Right");
    }
    else if (result.classification[1].value > 0.9)
    {
      nrf_gpio_pin_set(LED1);
      nrf_gpio_pin_clear(LED2);
      NRF_LOG_INFO("Rotate");
    }  
    else
    {
      NRF_LOG_INFO("Nothing");
      NRF_LOG_INFO("data received: %s",features); 
    } 
}

Now the result shows as the following

Features (0 ms.): 8.26 5.35 27.00 3.10 2.92 2.57 1.98 0.97 1.46 1.80 1.88 8.57 5.28 26.33 3.14 2.95 2.58 1.93 0.77 1.52 1.82 1.88 8.00 5.49 28.39 3.03 2.88 2.60 2.09 0.79 1.54 2.07 2.21 
Running neural network...
Predictions (time: 0 ms.):
Left-right:	0.99
Rotate:	0.00
run_classifier returned: 0
Timing: DSP 0 ms, inference 0 ms, anomaly 0 ms
Features (0 ms.): 8.26 5.35 27.00 3.10 2.92 2.57 1.98 0.97 1.46 1.80 1.88 9.09 5.31 26.75 3.19 3.00 2.65 2.04 1.12 1.53 1.80 1.85 8.32 5.57 29.64 3.06 2.91 2.63 2.11 0.98 1.69 2.16 2.29 
Running neural network...
Predictions (time: 0 ms.):
Left-right:	0.15
Rotate:	0.84
run_classifier returned: 0
Timing: DSP 0 ms, inference 0 ms, anomaly 0 ms

Now the Features array is not constant as before, if I move the device I think it is predicting something but not as intended. I have few questions,

  1. Is the features array fed right? If so why the prediction, DSP, inference and anomaly time is 0 ms?

  2. The gesture Left-Right and Rotate which I have trained is just a momentary movement, Most of the time the device will be doing some other movement. If that’s the case do I have to add the anomaly block? Say if I’m getting

Left-right: 0.15
Rotate: 0.84

do I’ll get

Left-right: 0.00
Rotate: 0.00

if some other actions are made?

  1. I want the device to predict the gesture instantly when it is performed. How to reduce the prediction time? Do I have to reduce the interval time or I have to use the run_classifier_continuous() function?

Appreciate your help. Thanks in advance.