Inference Error on STM32 Proteus simulation

ookosoft · April 21, 2021, 8:39am

I am trying to simulate different models for STM32 on proteus but getting the same wrong classification for all models (al giving an output of -5 for any label). Live classification gives correct results. Below is a sample output

janjongboom · April 21, 2021, 9:59am

@ookosoft Could this be an error with printing floats? If you do a normal printf of e.g. 0.35f does it print correctly?

louis · April 21, 2021, 10:33am

Hello @ookosoft,

Could you share your the model_metadata.h file so I can have a look please?
And do you mind if I join your project to run the Live Classification?

Regards,

Louis

ookosoft · April 21, 2021, 10:43am

/* Generated by Edge Impulse
*

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the “Software”), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/

#ifndef EI_CLASSIFIER_MODEL_METADATA_H
#define EI_CLASSIFIER_MODEL_METADATA_H

#include <stdint.h>

#define EI_CLASSIFIER_NONE 255
#define EI_CLASSIFIER_UTENSOR 1
#define EI_CLASSIFIER_TFLITE 2
#define EI_CLASSIFIER_CUBEAI 3
#define EI_CLASSIFIER_TFLITE_FULL 4
#define EI_CLASSIFIER_TENSAIFLOW 5
#define EI_CLASSIFIER_TENSORRT 6

#define EI_CLASSIFIER_SENSOR_UNKNOWN -1
#define EI_CLASSIFIER_SENSOR_MICROPHONE 1
#define EI_CLASSIFIER_SENSOR_ACCELEROMETER 2
#define EI_CLASSIFIER_SENSOR_CAMERA 3

// These must match the enum values in TensorFlow Lite’s “TfLiteType”
#define EI_CLASSIFIER_DATATYPE_FLOAT32 1
#define EI_CLASSIFIER_DATATYPE_INT8 9

#define EI_CLASSIFIER_PROJECT_ID 27622
#define EI_CLASSIFIER_PROJECT_OWNER “SAMSON OTIENO OOKO”
#define EI_CLASSIFIER_PROJECT_NAME “Edge AI for COPD”
#define EI_CLASSIFIER_PROJECT_DEPLOY_VERSION 2
#define EI_CLASSIFIER_NN_INPUT_FRAME_SIZE 56
#define EI_CLASSIFIER_RAW_SAMPLE_COUNT 500
#define EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME 8
#define EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE (EI_CLASSIFIER_RAW_SAMPLE_COUNT * EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME)
#define EI_CLASSIFIER_INPUT_WIDTH 0
#define EI_CLASSIFIER_INPUT_HEIGHT 0
#define EI_CLASSIFIER_INTERVAL_MS 2
#define EI_CLASSIFIER_LABEL_COUNT 2
#define EI_CLASSIFIER_HAS_ANOMALY 0
#define EI_CLASSIFIER_FREQUENCY 500
#define EI_CLASSIFIER_USE_QUANTIZED_DSP_BLOCK 0

#define EI_CLASSIFIER_OBJECT_DETECTION 0

#define EI_CLASSIFIER_TFLITE_ARENA_SIZE 3692
#define EI_CLASSIFIER_TFLITE_INPUT_DATATYPE EI_CLASSIFIER_DATATYPE_INT8
#define EI_CLASSIFIER_TFLITE_INPUT_QUANTIZED 1
#define EI_CLASSIFIER_TFLITE_INPUT_SCALE 0.0363069511950016
#define EI_CLASSIFIER_TFLITE_INPUT_ZEROPOINT -45
#define EI_CLASSIFIER_TFLITE_OUTPUT_DATATYPE EI_CLASSIFIER_DATATYPE_INT8
#define EI_CLASSIFIER_TFLITE_OUTPUT_QUANTIZED 1
#define EI_CLASSIFIER_TFLITE_OUTPUT_SCALE 0.00390625
#define EI_CLASSIFIER_TFLITE_OUTPUT_ZEROPOINT -128
#define EI_CLASSIFIER_INFERENCING_ENGINE EI_CLASSIFIER_TFLITE
#define EI_CLASSIFIER_COMPILED 1
#define EI_CLASSIFIER_HAS_TFLITE_OPS_RESOLVER 1

#define EI_CLASSIFIER_SENSOR EI_CLASSIFIER_SENSOR_UNKNOWN
#define EI_CLASSIFIER_SLICE_SIZE (EI_CLASSIFIER_RAW_SAMPLE_COUNT / EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW)
#ifndef EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW
#define EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW 4
#endif // EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW

#if EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_TFLITE && EI_CLASSIFIER_USE_FULL_TFLITE == 1
#undef EI_CLASSIFIER_INFERENCING_ENGINE
#undef EI_CLASSIFIER_HAS_TFLITE_OPS_RESOLVER
#define EI_CLASSIFIER_INFERENCING_ENGINE EI_CLASSIFIER_TFLITE_FULL
#define EI_CLASSIFIER_HAS_TFLITE_OPS_RESOLVER 0
#if EI_CLASSIFIER_COMPILED == 1
#error “Cannot use full TensorFlow Lite with EON”
#endif
#endif // EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_TFLITE && EI_CLASSIFIER_USE_FULL_TFLITE == 1

const char* ei_classifier_inferencing_categories[] = { “COPD”, “Healthy” };

typedef struct {
uint16_t implementation_version;
int axes;
float scale_axes;
bool average;
bool minimum;
bool maximum;
bool rms;
bool stdev;
bool skewness;
bool kurtosis;
} ei_dsp_config_flatten_t;

typedef struct {
uint16_t implementation_version;
int axes;
const char * channels;
} ei_dsp_config_image_t;

typedef struct {
uint16_t implementation_version;
int axes;
int num_cepstral;
float frame_length;
float frame_stride;
int num_filters;
int fft_length;
int win_size;
int low_frequency;
int high_frequency;
float pre_cof;
int pre_shift;
} ei_dsp_config_mfcc_t;

typedef struct {
uint16_t implementation_version;
int axes;
float frame_length;
float frame_stride;
int num_filters;
int fft_length;
int low_frequency;
int high_frequency;
int win_size;
} ei_dsp_config_mfe_t;

typedef struct {
uint16_t implementation_version;
int axes;
float scale_axes;
} ei_dsp_config_raw_t;

typedef struct {
uint16_t implementation_version;
int axes;
float scale_axes;
const char * filter_type;
float filter_cutoff;
int filter_order;
int fft_length;
int spectral_peaks_count;
float spectral_peaks_threshold;
const char * spectral_power_edges;
} ei_dsp_config_spectral_analysis_t;

typedef struct {
uint16_t implementation_version;
int axes;
float frame_length;
float frame_stride;
int fft_length;
bool show_axes;
} ei_dsp_config_spectrogram_t;

typedef struct {
uint16_t implementation_version;
int axes;
float frame_length;
float frame_stride;
int num_filters;
int fft_length;
int low_frequency;
int high_frequency;
float pre_cof;
} ei_dsp_config_audio_syntiant_t;

ei_dsp_config_flatten_t ei_dsp_config_23 = {
1,
8,
1.00000f,
true,
true,
true,
true,
true,
true,
true
};

#endif // EI_CLASSIFIER_MODEL_METADATA_H

ookosoft · April 21, 2021, 10:49am

Hi @louis

I have shared its okay you can join I do not mind, Live classification is running well, but getting the error on device simulations

janjongboom · April 21, 2021, 10:54am

@ookosoft I’ve tested this quickly on one of my targets, and the outputs match the studio. If you print like this:

      // print the predictions
      ei_printf("[");
      for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
          ei_printf("%d ", (int)(result.classification[ix].value * 1000.0f));
  #if EI_CLASSIFIER_HAS_ANOMALY == 1
          ei_printf(", ");
  #else
          if (ix != EI_CLASSIFIER_LABEL_COUNT - 1) {
              ei_printf(", ");
          }
  #endif
      }

What does it say? I’m still suspecting the float printing.

marvin · April 21, 2021, 10:58am

Hi i have been experiencing the same issue, is it possible to see the output of the predictions you are getting ? @janjongboom

ookosoft · April 21, 2021, 11:04am

Tried with this still getting an output similar to the one from the earlier model

marvin · April 21, 2021, 11:05am

janjongboom · April 21, 2021, 11:15am

@marvin @ookosoft Interesting… If you run this on a real device, do you see proper results? Also, which Proteus model are you using? (Never used, but will send @louis on a mission to try it out )

marvin · April 21, 2021, 11:19am

@janjongboom

proteus version 8.9. selection of the board is STM32F401VE an arm cortex M4, 84MHz, 512K/96K should meet hardware requirements yes?

ookosoft · April 21, 2021, 11:20am

Trying with some of these models. Yet to try with a board

janjongboom · April 21, 2021, 11:31am

@marvin Yes, the capabilities shouldn’t be a problem, we run this on a wide variety of targets, and also in Renode without issue, so interesting to see what’s the difference with Proteus.

louis · April 21, 2021, 11:46am

Sure @janjongboom, let me get a Windows Virtual Machine to install Proteus on.

@marvin @ookosoft, I let you know as soon as I have tested.

Regards

ookosoft · April 21, 2021, 11:51am

Thanks @louis and @janjongboom I appreciate the efforts

louis · April 21, 2021, 2:08pm

@ookosoft,

Do you mind sharing your stm32cube project.
I just installed a brand new Windows VM with Proteus on it

Best,

Louis

ookosoft · April 21, 2021, 2:32pm

Yes I can share,do you need entire project or that the HEX to execute on proteus? I am also not seeing the option to attach files

louis · April 21, 2021, 2:35pm

I guess both would be best.

You can send me a zip file at louis@edgeimpulse.com

Once we will have the solution, we will share the solution here so everyone can enjoy it.

louis · April 21, 2021, 2:40pm

And could you explain a bit what steps you followed to run your program on Proteus so I can reproduce them please?

ookosoft · April 21, 2021, 2:54pm

Followed the steps on running the model on edge impulse, However before generating code you also need to enable UART under connectivity.

On project settings C++ build settings, MCU post build outputs enable convert to HEX file

You will use the HEX when simulating in Proteus…Do you need the proteus steps?