Spectrogram Event Length & Zero Pad Feature Differences

Report bugs
#1

Hi, I am running into an issue where I am using 3x spectrogram blocks for feature extraction on 3 sensor inputs prior to inference as a workaround to a previous (DSP Error -1004) issue here.

A constraint I have is that my sample size is most of the time smaller than the selected window size and therefore I have chosen to zero-pad the samples for training and test which has resulted in good training and test results online.

I then transfer the model to a C/C++ app which is running locally and because I was getting the error on size mismatch (“The size of your ‘features’ array is not correct. Expected %d items, but had %lu\n”), I have adapted the demo C++ code (https://github.com/edgeimpulse/example-standalone-inferencing) so that I now pad the remainder of the sample with zeros to get to a correct length (I assume this is the same as online zero-pad). The demo app now accepts the input.

I run this but I end up where the online calculated features compare well with the local features up to a point and then they start to deviate. This seems to result in differing output for inference compared with the web classification.

I have checked that my padding of the “raw_features” variable is correct - here is my code for reference

if (raw_features.size() != EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE) {
    printf("The size of your 'features' array is not correct. Expected %d items, but had %lu\n",
        EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, raw_features.size());

    const size_t nonPaddedSize = raw_features.size();
    const size_t numPaddingElements = EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE - nonPaddedSize;
    if (numPaddingElements > 0) {
        raw_features.resize(nonPaddedSize + numPaddingElements, 0);
    }

    //return 1;

    printf("Corrected the size. Expected %d items, have %lu\n",
        EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, raw_features.size());
}

What I have noticed is that the online generated features have a lot of zeros after a bulk of numbers up front, whereas the locally generated features does not have the bulk of zeros on the backend of the features that are generated.

E.G.
Sample size = 645
Window size = 3000 (therefore fill 646 > 2999 with zeros)
Calculated features = 9801
Online X features - values 1 to 726 have a value, after this up to 3267 has zero as value
Local X features - have values up to 3267 with zeros interspersed without big block

Please help - perhaps there is something simple that I am doing wrong in padding or other ?

#2

Hi @bw42, I just tested this quickly and there’s indeed something funky going on (at least I would write the code in the same way)… I’ve blocked some time to investigate on Thursday.

edit: I indeed see something weird going on when creating the features in the SDK, to be continued.

#3

Hi @bw42 it’s indeed a bug in our signal handling code, a patch is forthcoming (probably by the end of the week), but you can apply this diff if needed earlier:

diff --git a/edge-impulse-sdk/classifier/ei_signal_with_axes.h b/edge-impulse-sdk/classifier/ei_signal_with_axes.h
index 78257298e..c6b1fe53a 100644
--- a/edge-impulse-sdk/classifier/ei_signal_with_axes.h
+++ b/edge-impulse-sdk/classifier/ei_signal_with_axes.h
@@ -55,11 +55,12 @@ public:
     }
 
     int get_data(size_t offset, size_t length, float *out_ptr) {
+        size_t offset_on_original_signal = offset / _axes_count * EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME;
         size_t length_on_original_signal = length / _axes_count * EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME;
 
         size_t out_ptr_ix = 0;
 
-        for (size_t ix = 0; ix < length_on_original_signal; ix += EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME) {
+        for (size_t ix = offset_on_original_signal; ix < offset_on_original_signal + length_on_original_signal; ix += EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME) {
             for (size_t axis_ix = 0; axis_ix < this->_axes_count; axis_ix++) {
                 int r = _original_signal->get_data(ix + _axes[axis_ix], 1, &out_ptr[out_ptr_ix++]);
                 if (r != 0) {
@@ -103,11 +104,12 @@ public:
     }
 
     int get_data(size_t offset, size_t length, int16_t *out_ptr) {
+        size_t offset_on_original_signal = offset / _axes_count * EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME;
         size_t length_on_original_signal = length / _axes_count * EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME;
 
         size_t out_ptr_ix = 0;
 
-        for (size_t ix = 0; ix < length_on_original_signal; ix += EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME) {
+        for (size_t ix = offset_on_original_signal; ix < offset_on_original_signal + length_on_original_signal; ix += EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME) {
             for (size_t axis_ix = 0; axis_ix < this->_axes_count; axis_ix++) {
                 int r = _original_signal->get_data(ix + _axes[axis_ix], 1, &out_ptr[out_ptr_ix++]);
                 if (r != 0) {
#4

Hi @janjongboom - It works! I have applied your fix and all features now match the cloud output - thank you very much! :slight_smile:

1 Like