Doesn't work "static_buffer" sketch in Spresense

Question/Issue:
Hello, I using Edge Impulse object detection in Spresense, a model was created to identify the color of a magnet.

After deploying the Arduino library for the model and using the “static_buffer” sketch, the same raw features that were identified in the web classfication are not being identified in Spresense.

Any advice would be appreciated.
Thank you.


Live Classification Result


Classification Result in Spresense

Edge Impulse standalone inferencing (Arduino)
run_classifier returned: 0
Timing: DSP 31 ms, inference 3584 ms, anomaly 0 ms
Object detection bounding boxes:

Hi @ts1040,

Which features did you copy into the following portion of the static_buffer sketch?

static const float features[] = {
    // copy raw features here (for example from the 'Live classification' page)
    // see https://docs.edgeimpulse.com/docs/running-your-impulse-arduino
};

Could you provide your project ID so I can take a look at your model?

Thank you for your reply.

I send my projectID.

“245421”

If you have any problems please contact me.
Thank you

Hi @ts1040,

The project looks OK to me. What values did you copy into the features[] array?

What value did you use for the Score Threshold? Try lowering the value.

To get there:

  • Model Testing page
  • Click 3 vertical dots to the right of the Classify All button
  • The Score Threshold textbox appears.
1 Like

Thank you @shawn_edgeimpulse

The data I copied are “Raw features” that can be obtained from the “Live classification” page on the web.

I referred to the reference page of the official Edge Impulse Arduino Library.

Is there a difference in usage between audio data and image data?

A part of the features source code is attached.

static const float features[] = {
    // copy raw features here (for example from the 'Live classification' page)
    // see https://docs.edgeimpulse.com/docs/running-your-impulse-arduino
    0x5d5644, 0x5e5745, 0x5e5746, 0x5e5743, 0x5e5842, 0x5f5945, 0x605946, 0x605946, 0x615946, 0x615946, 0x615946, 0x615a47, 0x615a47, 0x615944, 0x625a45, 0x625a46, 0x635b48, 0x625b48, 0x655b49, 0x665a49, 0x655a48, 0x675c48, 0x665b47, 0x675b49, 0x685b4b,...
};

Thank you

Thank you @MMarcial

I tryed decrease the “Score Threshold” from 0.5 to 0.3, but doesn’t detection the model.

Thank you.

Hi @ts1040,

Thanks, that features array looks correct. Could you try two other things:

  1. Try changing the debug flag on run_classifier() to true and running again to see if you see any helpful insights in the serial output (paste them here if you can):
EI_IMPULSE_ERROR res = run_classifier(&features_signal, &result, true /* debug */);
  1. Paste the contents of your <arduino directory>/libraries/<project>_inferencing/src/model-parameters/model_metadata.h so we can take a look at how the project was exported.

Hi @shawn_edgeimpulse
Sorry late reply.

I tryed two things.

First.
I changed the run_classifier() debug flag from false to true.
The following is the data returned in serial.

Edge Impulse standalone inferencing (Arduino)
Features (31 ms.): 0.364706 0.337255 0.266667 0.368627 0.341176 0.270588 0.368627 0.341176 0.274510 0.368627 ... (very long features datas)
Predictions (time: 3585 ms.):
run_classifier returned: 0
Timing: DSP 31 ms, inference 3585 ms, anomaly 0 ms
Object detection bounding boxes:

Seconds.
The following is the contents of model_metadata.h.

#ifndef _EI_CLASSIFIER_MODEL_METADATA_H_
#define _EI_CLASSIFIER_MODEL_METADATA_H_

#include <stdint.h>
#include <stdbool.h>

#define EI_CLASSIFIER_NONE                       255
#define EI_CLASSIFIER_UTENSOR                    1
#define EI_CLASSIFIER_TFLITE                     2
#define EI_CLASSIFIER_CUBEAI                     3
#define EI_CLASSIFIER_TFLITE_FULL                4
#define EI_CLASSIFIER_TENSAIFLOW                 5
#define EI_CLASSIFIER_TENSORRT                   6
#define EI_CLASSIFIER_DRPAI                      7
#define EI_CLASSIFIER_TFLITE_TIDL                8
#define EI_CLASSIFIER_AKIDA                      9
#define EI_CLASSIFIER_SYNTIANT                   10
#define EI_CLASSIFIER_ONNX_TIDL                  11
#define EI_CLASSIFIER_MEMRYX                     12

#define EI_CLASSIFIER_SENSOR_UNKNOWN             -1
#define EI_CLASSIFIER_SENSOR_MICROPHONE          1
#define EI_CLASSIFIER_SENSOR_ACCELEROMETER       2
#define EI_CLASSIFIER_SENSOR_CAMERA              3
#define EI_CLASSIFIER_SENSOR_9DOF                4
#define EI_CLASSIFIER_SENSOR_ENVIRONMENTAL       5
#define EI_CLASSIFIER_SENSOR_FUSION              6

// These must match the enum values in TensorFlow Lite's "TfLiteType"
#define EI_CLASSIFIER_DATATYPE_FLOAT32           1
#define EI_CLASSIFIER_DATATYPE_UINT8             3
#define EI_CLASSIFIER_DATATYPE_INT8              9

#define EI_CLASSIFIER_PROJECT_ID                 245421
#define EI_CLASSIFIER_PROJECT_OWNER              "TS"
#define EI_CLASSIFIER_PROJECT_NAME               "FOMO_Spresense"
#define EI_CLASSIFIER_PROJECT_DEPLOY_VERSION     4
#define EI_CLASSIFIER_NN_INPUT_FRAME_SIZE        76800
#define EI_CLASSIFIER_RAW_SAMPLE_COUNT           25600
#define EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME      1
#define EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE       (EI_CLASSIFIER_RAW_SAMPLE_COUNT * EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME)
#define EI_CLASSIFIER_INPUT_WIDTH                160
#define EI_CLASSIFIER_INPUT_HEIGHT               160
#define EI_CLASSIFIER_INPUT_FRAMES               1
#define EI_CLASSIFIER_NN_OUTPUT_COUNT            2800
#define EI_CLASSIFIER_INTERVAL_MS                1
#define EI_CLASSIFIER_LABEL_COUNT                6
#define EI_CLASSIFIER_HAS_ANOMALY                0
#define EI_CLASSIFIER_FREQUENCY                  0
#define EI_CLASSIFIER_HAS_MODEL_VARIABLES        1



#define EI_CLASSIFIER_OBJECT_DETECTION             1
#define EI_CLASSIFIER_OBJECT_DETECTION_LAST_LAYER  EI_CLASSIFIER_LAST_LAYER_FOMO
#warning 'EI_CLASSFIER_OBJECT_DETECTION_COUNT' is used for the guaranteed minimum number of objects detected. To get all objects during inference use 'bounding_boxes_count' from the 'ei_impulse_result_t' struct instead.
#define EI_CLASSIFIER_OBJECT_DETECTION_COUNT       10
#define EI_CLASSIFIER_OBJECT_DETECTION_THRESHOLD   0.3
#define EI_CLASSIFIER_TFLITE_OUTPUT_DATA_TENSOR    0
#define EI_CLASSIFIER_TFLITE_OUTPUT_LABELS_TENSOR  1
#define EI_CLASSIFIER_TFLITE_OUTPUT_SCORE_TENSOR   2


#define EI_CLASSIFIER_TFLITE_INPUT_DATATYPE         EI_CLASSIFIER_DATATYPE_INT8
#define EI_CLASSIFIER_TFLITE_OUTPUT_DATATYPE        EI_CLASSIFIER_DATATYPE_INT8


#define EI_CLASSIFIER_INFERENCING_ENGINE            EI_CLASSIFIER_TFLITE

#define EI_CLASSIFIER_QUANTIZATION_ENABLED          1

#define EI_CLASSIFIER_COMPILED                      1
#define EI_CLASSIFIER_HAS_TFLITE_OPS_RESOLVER       0

#define EI_CLASSIFIER_LOAD_IMAGE_SCALING         0


#define EI_CLASSIFIER_HAS_FFT_INFO               1
#define EI_CLASSIFIER_LOAD_FFT_32                0
#define EI_CLASSIFIER_LOAD_FFT_64                0
#define EI_CLASSIFIER_LOAD_FFT_128               0
#define EI_CLASSIFIER_LOAD_FFT_256               0
#define EI_CLASSIFIER_LOAD_FFT_512               0
#define EI_CLASSIFIER_LOAD_FFT_1024              0
#define EI_CLASSIFIER_LOAD_FFT_2048              0
#define EI_CLASSIFIER_LOAD_FFT_4096              0

#define EI_DSP_PARAMS_GENERATED 1


#define EI_CLASSIFIER_SENSOR                     EI_CLASSIFIER_SENSOR_CAMERA
#define EI_CLASSIFIER_FUSION_AXES_STRING         "image"
#define EI_CLASSIFIER_CALIBRATION_ENABLED        0

#ifndef EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW
#define EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW    4
#endif // EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW
#define EI_CLASSIFIER_SLICE_SIZE                 (EI_CLASSIFIER_RAW_SAMPLE_COUNT / EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW)


#if ((EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_TFLITE) ||      (EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_DRPAI)) &&      EI_CLASSIFIER_USE_FULL_TFLITE == 1

#if EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_TFLITE
#undef EI_CLASSIFIER_INFERENCING_ENGINE
#define EI_CLASSIFIER_INFERENCING_ENGINE          EI_CLASSIFIER_TFLITE_FULL
#endif

#undef EI_CLASSIFIER_HAS_TFLITE_OPS_RESOLVER
#define EI_CLASSIFIER_HAS_TFLITE_OPS_RESOLVER     0

#if EI_CLASSIFIER_COMPILED == 1
#error "Cannot use full TensorFlow Lite with EON"
#endif
#endif // ((EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_TFLITE) || (EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_DRPAI)) && EI_CLASSIFIER_USE_FULL_TFLITE == 1

typedef struct {
    uint32_t block_id;
    uint16_t implementation_version;
    int axes;
    float scale_axes;
    bool average;
    bool minimum;
    bool maximum;
    bool rms;
    bool stdev;
    bool skewness;
    bool kurtosis;
} ei_dsp_config_flatten_t;

typedef struct {
    uint32_t block_id;
    uint16_t implementation_version;
    int axes;
    const char * channels;
} ei_dsp_config_image_t;

typedef struct {
    uint32_t block_id;
    uint16_t implementation_version;
    int axes;
    int num_cepstral;
    float frame_length;
    float frame_stride;
    int num_filters;
    int fft_length;
    int win_size;
    int low_frequency;
    int high_frequency;
    float pre_cof;
    int pre_shift;
} ei_dsp_config_mfcc_t;

typedef struct {
    uint32_t block_id;
    uint16_t implementation_version;
    int axes;
    float frame_length;
    float frame_stride;
    int num_filters;
    int fft_length;
    int low_frequency;
    int high_frequency;
    int win_size;
    int noise_floor_db;
} ei_dsp_config_mfe_t;

typedef struct {
    uint32_t block_id;
    uint16_t implementation_version;
    int axes;
    float scale_axes;
} ei_dsp_config_raw_t;

typedef struct {
    uint32_t block_id;
    uint16_t implementation_version;
    int axes;
    float scale_axes;
    int input_decimation_ratio;
    const char * filter_type;
    float filter_cutoff;
    int filter_order;
    const char * analysis_type;
    int fft_length;
    int spectral_peaks_count;
    float spectral_peaks_threshold;
    const char * spectral_power_edges;
    bool do_log;
    bool do_fft_overlap;
    int wavelet_level;
    const char * wavelet;
    bool extra_low_freq;
} ei_dsp_config_spectral_analysis_t;

typedef struct {
    uint32_t block_id;
    uint16_t implementation_version;
    int axes;
    float frame_length;
    float frame_stride;
    int fft_length;
    int noise_floor_db;
    bool show_axes;
} ei_dsp_config_spectrogram_t;

typedef struct {
    uint32_t block_id;
    uint16_t implementation_version;
    int axes;
    float frame_length;
    float frame_stride;
    int num_filters;
    int fft_length;
    int low_frequency;
    int high_frequency;
    float pre_cof;
    const char * extractor;
} ei_dsp_config_audio_syntiant_t;

typedef struct {
    uint32_t block_id;
    uint16_t implementation_version;
    int axes;
    bool scaling;
} ei_dsp_config_imu_syntiant_t;

#endif // _EI_CLASSIFIER_MODEL_METADATA_H_

Please contact us if you want any other information.
Thank you.

Everything here looks correct. @shawn_edgeimpulse do you agree?

@ts1040 Publish a version of your project, i.e., make it Public and I will try running on my Spresense.

Hi @ts1040,

Those settings look correct to me. The only thing I noticed is that from post #6 your features do not match your screenshot. The screenshot shows 0x615a47, 0x605946, 0x5f5846 ... whereas your feature array is 0x5d5644, 0x5e5745, 0x5e5746 ... Where did you copy that array from?

Hi @shawn_edgeimpulse

Sorry, I have posted other image raw features. (In sourcecode, I posted PICT017’s raw features)

I tried use correct raw features, but doesn’t classification in Spresense.

Hi @MMarcial

I publish the project. Please try running in Spresense.

Sorry late reply.
Thank you.

  • I was able to run static_buffer.ino with expected results using:

    • Arduino IDE 2.1.1, Spresense Board Library 3.1.0
  • I reproduced your issue using:

    • Arduino IDE 1.8.19, Spresense Board Library 3.1.0
    • The EdgeImpulse int8 library would not run with the default memory at 768kB so I increased to 1536kB and was able to reproduce your results.
    • The EdgeImpulse float32 library would not run even with 1536kB.

So my advice is to use Arduino IDE 2.1.1 for this mission.

1 Like

Hi @MMarcial

I already use the Arduino IDE 2.1.1. but part of board library was older than latest.
I tried after update but doesn’t change results.
I research more information this probrem.

Thank you.

Short of doing a screen sharing session I can only recommend that you start a document written like a procedure that documents each step you are doing.

A procedure does not necessarily document the WHY. Write it so a person unfamiliar wit the process can sit down at a keyboard executing the procedure.

  • Do this…
    • Do this…
  • IF do this AND this happens
    • Do this
    • ELSE
    • Do this

See section 4.2 in this doc for procedure writing guidelines.

Maybe check the Spresense Memory setting:

1 Like