ts1040
July 18, 2023, 4:58am
#1
Question/Issue:
After deploying the Arduino library for the model and using the “static_buffer” sketch, the same raw features that were identified in the web classfication are not being identified in Spresense.
Any advice would be appreciated.Live Classification Result
Classification Result in Spresense
Edge Impulse standalone inferencing (Arduino)
run_classifier returned: 0
Timing: DSP 31 ms, inference 3584 ms, anomaly 0 ms
Object detection bounding boxes:
Hi @ts1040 ,
Which features did you copy into the following portion of the static_buffer sketch?
static const float features[] = {
// copy raw features here (for example from the 'Live classification' page)
// see https://docs.edgeimpulse.com/docs/running-your-impulse-arduino
};
Could you provide your project ID so I can take a look at your model?
ts1040
July 19, 2023, 2:21am
#3
Thank you for your reply.
I send my projectID.
“245421”
If you have any problems please contact me.
Hi @ts1040 ,
The project looks OK to me. What values did you copy into the features[] array?
What value did you use for the Score Threshold? Try lowering the value.
To get there:
Model Testing page
Click 3 vertical dots to the right of the Classify All button
The Score Threshold textbox appears.
1 Like
ts1040
July 20, 2023, 2:19am
#6
Thank you @shawn_edgeimpulse
The data I copied are “Raw features” that can be obtained from the “Live classification” page on the web.
I referred to the reference page of the official Edge Impulse Arduino Library .
Is there a difference in usage between audio data and image data?
A part of the features source code is attached.
static const float features[] = {
// copy raw features here (for example from the 'Live classification' page)
// see https://docs.edgeimpulse.com/docs/running-your-impulse-arduino
0x5d5644, 0x5e5745, 0x5e5746, 0x5e5743, 0x5e5842, 0x5f5945, 0x605946, 0x605946, 0x615946, 0x615946, 0x615946, 0x615a47, 0x615a47, 0x615944, 0x625a45, 0x625a46, 0x635b48, 0x625b48, 0x655b49, 0x665a49, 0x655a48, 0x675c48, 0x665b47, 0x675b49, 0x685b4b,...
};
Thank you
ts1040
July 20, 2023, 2:22am
#7
Thank you @MMarcial
I tryed decrease the “Score Threshold” from 0.5 to 0.3, but doesn’t detection the model.
Thank you.
Hi @ts1040 ,
Thanks, that features array looks correct. Could you try two other things:
Try changing the debug flag on run_classifier() to true and running again to see if you see any helpful insights in the serial output (paste them here if you can):
EI_IMPULSE_ERROR res = run_classifier(&features_signal, &result, true /* debug */);
Paste the contents of your <arduino directory>/libraries/<project>_inferencing/src/model-parameters/model_metadata.h so we can take a look at how the project was exported.
ts1040
July 22, 2023, 3:18am
#9
Hi @shawn_edgeimpulse
I tryed two things.
First.run_classifier() debug flag from false to true.
Edge Impulse standalone inferencing (Arduino)
Features (31 ms.): 0.364706 0.337255 0.266667 0.368627 0.341176 0.270588 0.368627 0.341176 0.274510 0.368627 ... (very long features datas)
Predictions (time: 3585 ms.):
run_classifier returned: 0
Timing: DSP 31 ms, inference 3585 ms, anomaly 0 ms
Object detection bounding boxes:
Seconds.model_metadata.h.
#ifndef _EI_CLASSIFIER_MODEL_METADATA_H_
#define _EI_CLASSIFIER_MODEL_METADATA_H_
#include <stdint.h>
#include <stdbool.h>
#define EI_CLASSIFIER_NONE 255
#define EI_CLASSIFIER_UTENSOR 1
#define EI_CLASSIFIER_TFLITE 2
#define EI_CLASSIFIER_CUBEAI 3
#define EI_CLASSIFIER_TFLITE_FULL 4
#define EI_CLASSIFIER_TENSAIFLOW 5
#define EI_CLASSIFIER_TENSORRT 6
#define EI_CLASSIFIER_DRPAI 7
#define EI_CLASSIFIER_TFLITE_TIDL 8
#define EI_CLASSIFIER_AKIDA 9
#define EI_CLASSIFIER_SYNTIANT 10
#define EI_CLASSIFIER_ONNX_TIDL 11
#define EI_CLASSIFIER_MEMRYX 12
#define EI_CLASSIFIER_SENSOR_UNKNOWN -1
#define EI_CLASSIFIER_SENSOR_MICROPHONE 1
#define EI_CLASSIFIER_SENSOR_ACCELEROMETER 2
#define EI_CLASSIFIER_SENSOR_CAMERA 3
#define EI_CLASSIFIER_SENSOR_9DOF 4
#define EI_CLASSIFIER_SENSOR_ENVIRONMENTAL 5
#define EI_CLASSIFIER_SENSOR_FUSION 6
// These must match the enum values in TensorFlow Lite's "TfLiteType"
#define EI_CLASSIFIER_DATATYPE_FLOAT32 1
#define EI_CLASSIFIER_DATATYPE_UINT8 3
#define EI_CLASSIFIER_DATATYPE_INT8 9
#define EI_CLASSIFIER_PROJECT_ID 245421
#define EI_CLASSIFIER_PROJECT_OWNER "TS"
#define EI_CLASSIFIER_PROJECT_NAME "FOMO_Spresense"
#define EI_CLASSIFIER_PROJECT_DEPLOY_VERSION 4
#define EI_CLASSIFIER_NN_INPUT_FRAME_SIZE 76800
#define EI_CLASSIFIER_RAW_SAMPLE_COUNT 25600
#define EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME 1
#define EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE (EI_CLASSIFIER_RAW_SAMPLE_COUNT * EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME)
#define EI_CLASSIFIER_INPUT_WIDTH 160
#define EI_CLASSIFIER_INPUT_HEIGHT 160
#define EI_CLASSIFIER_INPUT_FRAMES 1
#define EI_CLASSIFIER_NN_OUTPUT_COUNT 2800
#define EI_CLASSIFIER_INTERVAL_MS 1
#define EI_CLASSIFIER_LABEL_COUNT 6
#define EI_CLASSIFIER_HAS_ANOMALY 0
#define EI_CLASSIFIER_FREQUENCY 0
#define EI_CLASSIFIER_HAS_MODEL_VARIABLES 1
#define EI_CLASSIFIER_OBJECT_DETECTION 1
#define EI_CLASSIFIER_OBJECT_DETECTION_LAST_LAYER EI_CLASSIFIER_LAST_LAYER_FOMO
#warning 'EI_CLASSFIER_OBJECT_DETECTION_COUNT' is used for the guaranteed minimum number of objects detected. To get all objects during inference use 'bounding_boxes_count' from the 'ei_impulse_result_t' struct instead.
#define EI_CLASSIFIER_OBJECT_DETECTION_COUNT 10
#define EI_CLASSIFIER_OBJECT_DETECTION_THRESHOLD 0.3
#define EI_CLASSIFIER_TFLITE_OUTPUT_DATA_TENSOR 0
#define EI_CLASSIFIER_TFLITE_OUTPUT_LABELS_TENSOR 1
#define EI_CLASSIFIER_TFLITE_OUTPUT_SCORE_TENSOR 2
#define EI_CLASSIFIER_TFLITE_INPUT_DATATYPE EI_CLASSIFIER_DATATYPE_INT8
#define EI_CLASSIFIER_TFLITE_OUTPUT_DATATYPE EI_CLASSIFIER_DATATYPE_INT8
#define EI_CLASSIFIER_INFERENCING_ENGINE EI_CLASSIFIER_TFLITE
#define EI_CLASSIFIER_QUANTIZATION_ENABLED 1
#define EI_CLASSIFIER_COMPILED 1
#define EI_CLASSIFIER_HAS_TFLITE_OPS_RESOLVER 0
#define EI_CLASSIFIER_LOAD_IMAGE_SCALING 0
#define EI_CLASSIFIER_HAS_FFT_INFO 1
#define EI_CLASSIFIER_LOAD_FFT_32 0
#define EI_CLASSIFIER_LOAD_FFT_64 0
#define EI_CLASSIFIER_LOAD_FFT_128 0
#define EI_CLASSIFIER_LOAD_FFT_256 0
#define EI_CLASSIFIER_LOAD_FFT_512 0
#define EI_CLASSIFIER_LOAD_FFT_1024 0
#define EI_CLASSIFIER_LOAD_FFT_2048 0
#define EI_CLASSIFIER_LOAD_FFT_4096 0
#define EI_DSP_PARAMS_GENERATED 1
#define EI_CLASSIFIER_SENSOR EI_CLASSIFIER_SENSOR_CAMERA
#define EI_CLASSIFIER_FUSION_AXES_STRING "image"
#define EI_CLASSIFIER_CALIBRATION_ENABLED 0
#ifndef EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW
#define EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW 4
#endif // EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW
#define EI_CLASSIFIER_SLICE_SIZE (EI_CLASSIFIER_RAW_SAMPLE_COUNT / EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW)
#if ((EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_TFLITE) || (EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_DRPAI)) && EI_CLASSIFIER_USE_FULL_TFLITE == 1
#if EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_TFLITE
#undef EI_CLASSIFIER_INFERENCING_ENGINE
#define EI_CLASSIFIER_INFERENCING_ENGINE EI_CLASSIFIER_TFLITE_FULL
#endif
#undef EI_CLASSIFIER_HAS_TFLITE_OPS_RESOLVER
#define EI_CLASSIFIER_HAS_TFLITE_OPS_RESOLVER 0
#if EI_CLASSIFIER_COMPILED == 1
#error "Cannot use full TensorFlow Lite with EON"
#endif
#endif // ((EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_TFLITE) || (EI_CLASSIFIER_INFERENCING_ENGINE == EI_CLASSIFIER_DRPAI)) && EI_CLASSIFIER_USE_FULL_TFLITE == 1
typedef struct {
uint32_t block_id;
uint16_t implementation_version;
int axes;
float scale_axes;
bool average;
bool minimum;
bool maximum;
bool rms;
bool stdev;
bool skewness;
bool kurtosis;
} ei_dsp_config_flatten_t;
typedef struct {
uint32_t block_id;
uint16_t implementation_version;
int axes;
const char * channels;
} ei_dsp_config_image_t;
typedef struct {
uint32_t block_id;
uint16_t implementation_version;
int axes;
int num_cepstral;
float frame_length;
float frame_stride;
int num_filters;
int fft_length;
int win_size;
int low_frequency;
int high_frequency;
float pre_cof;
int pre_shift;
} ei_dsp_config_mfcc_t;
typedef struct {
uint32_t block_id;
uint16_t implementation_version;
int axes;
float frame_length;
float frame_stride;
int num_filters;
int fft_length;
int low_frequency;
int high_frequency;
int win_size;
int noise_floor_db;
} ei_dsp_config_mfe_t;
typedef struct {
uint32_t block_id;
uint16_t implementation_version;
int axes;
float scale_axes;
} ei_dsp_config_raw_t;
typedef struct {
uint32_t block_id;
uint16_t implementation_version;
int axes;
float scale_axes;
int input_decimation_ratio;
const char * filter_type;
float filter_cutoff;
int filter_order;
const char * analysis_type;
int fft_length;
int spectral_peaks_count;
float spectral_peaks_threshold;
const char * spectral_power_edges;
bool do_log;
bool do_fft_overlap;
int wavelet_level;
const char * wavelet;
bool extra_low_freq;
} ei_dsp_config_spectral_analysis_t;
typedef struct {
uint32_t block_id;
uint16_t implementation_version;
int axes;
float frame_length;
float frame_stride;
int fft_length;
int noise_floor_db;
bool show_axes;
} ei_dsp_config_spectrogram_t;
typedef struct {
uint32_t block_id;
uint16_t implementation_version;
int axes;
float frame_length;
float frame_stride;
int num_filters;
int fft_length;
int low_frequency;
int high_frequency;
float pre_cof;
const char * extractor;
} ei_dsp_config_audio_syntiant_t;
typedef struct {
uint32_t block_id;
uint16_t implementation_version;
int axes;
bool scaling;
} ei_dsp_config_imu_syntiant_t;
#endif // _EI_CLASSIFIER_MODEL_METADATA_H_
Please contact us if you want any other information.
Everything here looks correct. @shawn_edgeimpulse do you agree?
@ts1040 Publish a version of your project, i.e., make it Public and I will try running on my Spresense.
Hi @ts1040 ,
Those settings look correct to me. The only thing I noticed is that from post #6 your features do not match your screenshot. The screenshot shows 0x615a47, 0x605946, 0x5f5846 ... whereas your feature array is 0x5d5644, 0x5e5745, 0x5e5746 ... Where did you copy that array from?
ts1040
July 25, 2023, 1:39am
#12
Hi @shawn_edgeimpulse
Sorry, I have posted other image raw features. (In sourcecode, I posted PICT017’s raw features)
I tried use correct raw features, but doesn’t classification in Spresense.
ts1040
July 25, 2023, 6:54am
#13
Hi @MMarcial
I publish the project. Please try running in Spresense.
Sorry late reply.
So my advice is to use Arduino IDE 2.1.1 for this mission.
1 Like
ts1040
July 26, 2023, 5:34am
#15
Hi @MMarcial
I already use the Arduino IDE 2.1.1. but part of board library was older than latest.
Thank you.
Short of doing a screen sharing session I can only recommend that you start a document written like a procedure that documents each step you are doing.
A procedure does not necessarily document the WHY. Write it so a person unfamiliar wit the process can sit down at a keyboard executing the procedure .
Do this…
IF do this AND this happens
See section 4.2 in this doc for procedure writing guidelines.
Maybe check the Spresense Memory setting:
1 Like
