Ei_run_classifier returns EI_IMPULSE_DSP_ERROR

[Ranchpal]

Hi,

I am trying build a simple motion classification application on NRF52833 soc using nRF5SDK. I am able to generate a model and deploy it. But when I run the classifier function ‘ei_run_classifier’ it is giving me a response of ‘EI_IMPULSE_DSP_ERROR’.

I am using ‘spectogram’ preprocessing block. Are there any other settings that need to be taken care of when using pre-processing blocks ? I did not have this issue when I used ‘RAW data’.

Kindly help !!

Thanks,
Rakesh

[louis]

Hello @Ranchpal

I just had a look at your project and I think your issue is similar to this one:

ERR: Failed to run DSP process (-1004) spectrogram

Hello @davidib19, @rjames and I have been looking a bit deeper into it. First, spectrogram does not accept at the moment more than one axis. We’ll work either on supporting it or add a option to select which axis you want to work with and you’ll need to duplicate your spectrogram block for each axis. We have an internal ticket that has been created. In the meantime, you can do something similar to this: [Screenshot 2021-12-21 at 14.32.18] This will let you run the C++ library from the exam…

Let me know if it solves your issue.

Best regards,

Louis

[Ranchpal]

Hi Louis, I tested it adding spectogram to each of the axis, the issue still present. Let me know what else I can test, or if you need any more information from me.

Thanks,
Rakesh

[Ranchpal]

Hi Louis,

Actually we like to add STFT but standard block not available so tried spectrogram, which is also giving similar accuracy results. We are sampling data @50Hz and window size is 1000 ms with 200ms hop. what are ideal parameters you suggest we need to choose while generating spectrogram? I mean Frame Length - 1, Frame stride- 0.2, Frequency Band - 50, Noise Floor(dB): -150.

Thanks,

Rakesh

[louis]

Hello @Ranchpal ,

That’s a million dollar question
It will very depend on your data, your project and your target. This is why we have released the EON tuner to test many parameters in parallel and help you choose the right DSP parameters and the right NN architecture for your specific application.

From what I see in your project, you are having an error on the EON tuner saying:
An error occurred during the last EON Tuner run: Every label in the training dataset requires at least 5 data samples
It seems that you have long data samples that could be split in several smaller samples.

I wrote a python script few weeks ago to split long CSV into smaller CSV, let me know if you need it, I’d be happy to share it.

Regards,

Louis

[Ranchpal]

Hi Louis,

Please do share but we are using 1/5 of data we have for training. Still what we can do about spectrogram EI_IMPULSE_DSP_ERROR. That model is fine for us but giving this error but not if run a model made by using RAW data.

Thanks,

Rakesh

[louis]

The csv-splitter script is the following:

csv-splitter.py:

import csv

def write_new_csv(csv_file_write, column_headers, split_number):

    writer = csv.DictWriter(csv_file_write, fieldnames=column_headers)
    writer.writeheader()

    return writer

def loop_through_csv(csv, split_threshold, column_headers, output_name):

    row_counter = 0
    split_number = 0
    write_csv = None

    for row in csv:

        if (row_counter == 0) or (row_counter == split_threshold):

            # close file if it already exists before writing to a new csv file
            if write_csv:
                write_csv.close()

            split_number += 1
            write_csv = open(f'{output_name}.{split_number}.csv', 'w')
            writer = write_new_csv(write_csv, column_headers, split_number)
            writer.writerow(row)
            row_counter = 0
        else:
            writer.writerow(row)

        row_counter += 1

def main():

    # grab inputs from user
    csv_file_path = input('Enter csv file path: ')
    split_threshold = int(input('Enter how many rows per CSV file you would like: '))
    output_name = input('Name prefix of the output files: ')

    with open(csv_file_path) as csv_file:
        csv_reader = csv.DictReader(csv_file)
        column_headers = csv_reader.fieldnames
        loop_through_csv(csv_reader, split_threshold, column_headers, output_name)

if __name__ == "__main__":
    main()

And to execute it:

$> python3 csv-splitter.py
Enter csv file path: your-file.csv
Enter how many rows per CSV file you would like: 200
Name prefix of the output files: your-label

For the DSP error, unfortunately, I don’t have an nRF52833 SoC with me at the moment.
I’ll try to check internally if someone has more info.

Regards,

Louis

[Ranchpal]

Hi Louis,

Do you mean DSP error is related to an nRF52833 SoC?

But why not when run model trained with raw data?

Thanks,

Rakesh

[louis]

@Ranchpal,

Is the version called 150db noise drop spectrogram on your project the one with the error?
I’d like to run the standalone example by downloading the C++ library but I see that you modified your project.

The DSP error could be related to the resource on your board or another issue within our SDK using the Spectrogram, I’d like to make sure where does the error comes from

Regards,

Louis

[Ranchpal]

Dear Louis,

yes 150db noise drop spectrogram is the version which is giving dsp_error.

I am trying different ways if that works.

Best Regards,

Rakesh

[Ranchpal]

Hi Louis,

I made this again correct project and named it spectrogram FQ50. Actually I have reduced the data to 1/5 but still EON tuner gives same error.

Appreciate your help!

Best Regards,

Rakesh

[Ranchpal]

Hi Louis,

Also do have code to add Normalisation by Standard Scalar Pre-Processing Block?

We want to add Normalisation block.

Thanks!

Rakesh

[louis]

Hello @Ranchpal

EON tuner gives same error.

EON tuner needs at least 5 samples per label, just split your samples into smaller files and import it again.

do have code to add Normalisation by Standard Scalar Pre-Processing Block?

You can create your own pre-processing block, see https://docs.edgeimpulse.com/docs/custom-blocks
All our processing blocks are available here if you need to add normalisation on top of your pre-processing: https://github.com/edgeimpulse/processing-blocks
Custom blocks can either be hosted on our side with the enterprise offer or you can host it on your servers.

Let me know if you want me to put you in contact with our business developers so they can present you this offer.

Also, I tested your project with the standalone inferencing example and I don’t have the DSP error but the classification results differ from the Studio Live Classification. I am asking the Core Engineering in they can have a look.

Regards,

Louis

[Ranchpal]

Hi Louis,

Please ask your core engineering team to have a look as we are still stuck at this.

Thanks!

Rakesh

[Ranchpal]

Hi Louis,

any leads on this. what we can do?

thanks,

Rakesh

[louis]

Hello @Ranchpal,

I have asked our Core Engineering to look into this, I will come back to you as soon as I have more info.

Regards,

Louis

[louis]

Hello @Ranchpal,

It seems that you don’t have any more data in your project.
@rjames was trying to investigate further where the issue could come from.

Have you changed anything in your project recently?

Regards,

Louis

[Ranchpal]

Hi Louis,

yes tried reducing data if that change something and run EON tuner but nothing changed.

Anyway will add data and run same model is next 1-2 hours.

Best Regards,

Rakesh

[louis]

Hello @Ranchpal,

@rjames found the issue, a fix is being pushed. It will be merged soon.

To give you more context of where the issue came from, the C/C++ and spectrogram/dsp.py differed between the python implementation and the C++ implementation.
In the python implementation it always scales the signal while in C/C++ implementation it only does so if it’s values are outside of the range [-1, 1].

Thanks for reporting this issue.

Regards,

Louis

[Ranchpal]

Hi Louis,

That is great insight. Let us know when this through. I run a fresh model and check out.

Appreciate the effort.

Best Regards,

Rakesh