It would be great if there were tutorials or examples of how to deploy a edge impulse model to the nano 33 ble sense.
One of the things that is great about Edge Impulse is how it makes training these models easy to do and accessible to everyone. It would be great if there were some end to end examples of running the models on an Arduino environment. Expecting users to know how to use mbed os, something that most people have not heard of or used, and is not that well documented, is a tall order.
Hi @oveddan, very interesting timing, as I just got this board in over the weekend. FYI, we are working on a proper port for the Nano 33 BLE Sense - with similar support as the ST IoT Discovery Kit - but this will be a few weeks off.
You can already run the compiled impulse on the board using Arduino CLI.
Update the Arduino core:
$ arduino-cli core update-index
Install the Arduino core for Mbed:
$ arduino-cli core install arduino:mbed
From Edge Impulse, export as C++ library using TensorFlow Lite, and place the output (three folders) in the src/ directory.
Run the following script to change the structure of the SDK to something the Arduino build tools understand:
rm -rf ./src/edge-impulse-sdk/utensor/CMakeFiles/
rm -rf ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/mbed/
rm -rf ./src/edge-impulse-sdk/porting/posix/
rm -rf ./src/edge-impulse-sdk/classifier/ei_run_classifier_c*
rm -rf ./src/edge-impulse-sdk/utensor/TESTS/
rm -rf ./src/edge-impulse-sdk/utensor/examples/
rm -f ./src/edge-impulse-sdk/CMSIS-DSP/Source/TransformFunctions/arm_bitreversal2.S
rm -rf ~/Library/Arduino15/packages/arduino/hardware/mbed/1.1.4/cores/arduino/mbed/features/unsupported/dsp/
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/MatrixFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/TransformFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/CommonTables/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/FilteringFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/StatisticsFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/BasicMathFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/SupportFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/DistanceFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/ControllerFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/FastMathFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/BayesFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/SVMFunctions/*.c
rm -rf ./src/edge-impulse-sdk/CMSIS-DSP/Source/ComplexMathFunctions/*.c
mv ./src/edge-impulse-sdk/tensorflow/lite/kernels/kernel_util.cc ./src/edge-impulse-sdk/tensorflow/lite/kernels/kernel_util.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/kernels/internal/quantization_util.cc ./src/edge-impulse-sdk/tensorflow/lite/kernels/internal/quantization_util.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/conv.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/conv.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/depthwise_conv.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/depthwise_conv.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/softmax.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/softmax.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/all_ops_resolver.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/all_ops_resolver.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/round.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/round.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/strided_slice.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/strided_slice.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/prelu.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/prelu.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/split.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/split.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/fully_connected.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/fully_connected.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/mul.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/mul.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/dequantize.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/dequantize.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/unpack.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/unpack.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/arg_min_max.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/arg_min_max.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/reshape.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/reshape.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/activations.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/activations.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/logistic.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/logistic.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/ceil.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/ceil.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/elementwise.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/elementwise.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/pooling.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/pooling.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/floor.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/floor.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/svdf.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/svdf.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/maximum_minimum.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/maximum_minimum.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/pack.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/pack.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/comparisons.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/comparisons.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/quantize.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/quantize.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/logical.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/logical.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/neg.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/neg.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/add.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/kernels/add.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/test_helpers.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/test_helpers.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/simple_memory_allocator.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/simple_memory_allocator.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_error_reporter.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_error_reporter.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_optional_debug_tools.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_optional_debug_tools.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/debug_log_numbers.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/debug_log_numbers.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_interpreter.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_interpreter.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/memory_planner/greedy_memory_planner.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/memory_planner/greedy_memory_planner.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/memory_planner/linear_memory_planner.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/memory_planner/linear_memory_planner.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_utils.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_utils.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_mutable_op_resolver.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_mutable_op_resolver.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_allocator.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/micro_allocator.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/memory_helpers.cc ./src/edge-impulse-sdk/tensorflow/lite/experimental/micro/memory_helpers.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/core/api/error_reporter.cc ./src/edge-impulse-sdk/tensorflow/lite/core/api/error_reporter.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/core/api/tensor_utils.cc ./src/edge-impulse-sdk/tensorflow/lite/core/api/tensor_utils.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/core/api/flatbuffer_conversions.cc ./src/edge-impulse-sdk/tensorflow/lite/core/api/flatbuffer_conversions.cpp
mv ./src/edge-impulse-sdk/tensorflow/lite/core/api/op_resolver.cc ./src/edge-impulse-sdk/tensorflow/lite/core/api/op_resolver.cpp
mv ./src/edge-impulse-sdk/porting/mbed/debug_log.cc ./src/edge-impulse-sdk/porting/mbed/debug_log.cpp
Add the following sketch which invokes the inferencing SDK to classify a sample:
using namespace mbed;
#include "ei_run_classifier.h"
float features[] = {
1.5100, 3.7600, 10.2400, 1.8200, 5.4800, 9.9500, 2.0400, 6.9800, 10.2900, 1.5100, 6.5000, 9.9300, 0.2500, 4.6300, 10.0000, -1.2100, 2.4600, 10.0500, -1.2100, 2.4600, 10.0500, -2.1000, 1.0100, 9.7800, -2.2600, -0.8700, 10.1400, -1.6200, -0.8400, 10.1600, -0.9100, -0.2000, 10.2300, -0.3500, 0.3600, 10.0600, 1.8100, 0.8200, 9.4900, 1.8100, 0.8200, 9.4900, 2.5600, -1.9800, 9.1300, 2.0700, -4.9000, 10.1000, 0.3800, -7.3400, 10.5400, 0.6000, -5.7300, 9.9900, 0.8400, -3.4300, 10.1800, 1.0900, -3.7900, 9.7000, 1.0900, -3.7900, 9.7000, 1.3500, -4.2600, 10.0400, 1.6000, -3.5800, 9.9400, 1.2400, -1.3200, 9.6000, 1.3500, -1.2600, 9.7600, 1.1300, -1.9200, 10.1400, 0.3200, -0.7300, 10.2100, 0.3200, -0.7300, 10.2100, 2.9500, -0.8200, 9.9000, 3.4400, 2.1700, 10.0200, 2.5300, 3.6700, 10.1800, 1.6400, 3.9100, 9.7000, 0.5100, 4.5900, 9.9800, 0.1600, 5.0400, 10.0900, 0.1600, 5.0400, 10.0900, 1.0600, 4.7000, 10.3300, 1.5900, 4.4000, 9.9400, 0.3900, 3.0200, 10.2600, 0.1000, 1.7800, 9.9900, -0.3700, 1.2800, 10.1500, -0.3700, 1.2800, 9.9100, -0.2000, 1.5200, 9.9100, -0.3300, 2.5200, 9.9600, 0.3600, 3.2800, 10.0300, 0.2300, 3.3400, 10.3300, -1.1100, 1.7400, 9.9100, -2.6200, -1.1100, 10.2900, -2.6200, -1.1100, 10.2900, -1.8700, -3.1100, 10.0000, -2.0900, -4.0700, 9.9400, -2.1200, -4.0100, 10.2100, -1.4100, -4.2000, 10.0900, -1.6900, -3.5100, 10.1500, 0.3500, -3.0000, 10.1300, 0.3500, -3.0000, 10.1300, -0.0500, -2.7300, 9.9400, 0.0200, -3.1800, 10.0700, 0.9200, -1.9000, 10.1700, 2.2100, -1.8700, 9.9700, 2.9500, -2.5600, 9.9800, 1.9400, -3.2000, 10.2200, 1.9400, -3.2000, 10.2200, 1.3400, -2.5700, 9.6700, 1.4300, -0.1200, 10.1100, 1.4300, 1.7100, 9.8200, 1.0400, 3.2000, 9.7500, 1.4900, 4.2400, 10.0100, 0.4400, 4.5200, 9.9900, 0.4400, 4.5200, 9.9900, 0.2000, 4.6100, 10.0700, -1.7500, 4.5500, 10.1900, -2.7800, 3.8400, 9.9700, -2.4100, 6.8100, 10.2100, -2.6200, 6.6100, 10.0100, -2.5500, 3.6900, 9.9800, -2.5500, 3.6900, 9.9800, -1.5300, -0.1100, 10.0300, -0.7700, -1.3900, 10.1900, -0.7500, -0.2500, 9.9300, -0.5000, 1.5100, 10.2800, -0.3600, 1.0900, 9.8600, 0.0000, -0.0900, 9.8100, 0.0000, -0.0900, 9.8100, 0.4700, -1.2800, 9.8700, -0.3000, -1.4700, 9.8900, 0.0100, -1.0100, 10.0300, 0.4000, -0.3100, 9.3500, 1.4400, -1.5900, 10.1700, 3.4200, -3.8200, 9.3300, 3.4200, -3.8200, 9.3300, 3.1300, -6.0200, 9.7800, 2.7100, -5.0600, 9.9900, 2.6100, -4.3400, 9.5700, 2.8000, -2.4700, 10.1500, 2.1400, -1.5300, 10.0500, 2.1000, -0.7400, 10.1800, 2.1000, -0.7400, 10.1800, 2.2400, -0.9800, 10.0100, 1.4600, -0.8800, 9.5800, 1.1500, 0.6100, 9.9500, 0.2300, 2.2000, 9.9000, -1.6100, 3.3300, 10.1100, -1.6100, 3.3300, 10.1100, -2.0800, 4.9400, 9.4500, -2.5800, 6.0800, 9.6200, -3.5000, 4.8000, 9.5500, -3.4500, 1.6200, 9.9600, -1.4000, 1.1200, 9.3100, 0.0100, 0.4700, 10.1500, 0.0100, 0.4700, 10.1500, 0.8300, 0.7500, 9.8500, 0.4000, 2.8100, 10.0700, 1.2100, 1.7800, 9.6500, 1.5000, 0.7100, 10.0000, 0.8300, 1.3800, 9.9000, 0.4700, 1.3400, 9.7300, 0.4700, 1.3400, 9.7300, 1.0200, 1.4500, 9.9100, 0.4200, 0.5000, 10.0600, -0.0500, -0.7300, 9.6900, 0.1500, -1.0500, 9.6400, 1.0700, -1.8400, 10.1000, 1.1500, -2.8100, 9.7100, 1.1500, -2.8100, 9.7100, 0.3500, -3.4200, 9.9100
};
int raw_feature_get_data(size_t offset, size_t length, float *out_ptr) {
memcpy(out_ptr, features + offset, length * sizeof(float));
return 0;
}
void setup() {
Serial.begin(115200);
}
void loop() {
ei_impulse_result_t result = { 0 };
// the features are stored into flash, and we don't want to load everything into RAM
signal_t features_signal;
features_signal.total_length = sizeof(features) / sizeof(features[0]);
features_signal.get_data = &raw_feature_get_data;
// invoke the impulse
EI_IMPULSE_ERROR res = run_classifier(&features_signal, &result, true);
Serial.print("run_classifier returned: ");
Serial.println(res);
if (res != 0) return;
Serial.print("Predictions (DSP: ");
Serial.print(result.timing.dsp);
Serial.print(" ms., Classification: ");
Serial.print(result.timing.classification);
Serial.print(" ms., Anomaly: ");
Serial.print(result.timing.anomaly);
Serial.println(" ms.): ");
// print the predictions
Serial.print("[");
for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
Serial.print(result.classification[ix].value, 5);
#if EI_CLASSIFIER_HAS_ANOMALY == 1
Serial.print(", ");
#else
if (ix != EI_CLASSIFIER_LABEL_COUNT - 1) {
Serial.print(", ");
}
#endif
}
#if EI_CLASSIFIER_HAS_ANOMALY == 1
Serial.print(result.anomaly, 3);
#endif
Serial.println("]");
delay(1000);
}
Build the application:
$ arduino-cli compile --fqbn arduino:mbed:nano33ble --build-properties build.extra_flags="-I./src/ -I./src/edge-impulse-sdk/anomaly -I./src/model-parameters -I./src/edge-impulse-sdk -I./src/edge-impulse-sdk/porting -I./src/edge-impulse-sdk/classifier -I./src/edge-impulse-sdk/third_party -I./src/edge-impulse-sdk/third_party/flatbuffers/include -I./src/tflite-model -I./src/edge-impulse-sdk/utensor -I./src/edge-impulse-sdk/CMSIS-DSP/Include -I./src/edge-impulse-sdk/third_party/gemmlowp -DARDUINOSTL_M_H -DMBED_HEAP_STATS_ENABLED=1 -DMBED_STACK_STATS_ENABLED=1 -Og -DMBED_DEBUG -g3" .
Flash the application:
$ arduino-cli upload -p $(arduino-cli board list | grep Arduino | cut -d ' ' -f1) --fqbn arduino:mbed:nano33ble .
This doesn’t use any of the sensors on the board so you’d still need to wire those in from the sketch, but at least it’s possible 
As said, we’re working on a proper package for this board that has accelerometer and microphone support for both data acquisition and inferencing, so if you have some time to wait, that’d be a good option.
(edit: just realized that the C++ export has not received the latest inferencing SDK update, so for the above to work, remove edge-impulse-sdk and replace with the latest version from GitHub. Will be fixed in the next few days.)
Wow - super amazing, thank you! Looking forward to trying this out.
@janjongboom https://github.com/edgeimpulse/inferencing-sdk-cpp/blob/master/porting/mbed/debug_log.cc#L21
When porting, is there only an output function, not an input function? Also I see that you use AT commands during communication, I don’t see this part of the code. How does this part work?
Hi @Boazhu, yeah the inferencing SDK only has an output function; as the library is supposed to be as minimal and portable as possible.
AT commands are in the firmware for the ST board here: https://github.com/edgeimpulse/firmware-st-b-l475e-iot01a/tree/master/source/repl. My suggestion would be to reimplement this on your target using the libraries you already have, because that code is very platform dependent (the way UART IRQs are set up, and signaling using RTX). That’s why we didn’t split that part out. All commands and their responses are listed here: https://docs.edgeimpulse.com/reference#remote-mgmt-serial-protocol
@janjongboom great work on this! Super easy to gather and train data from an Arduino with the Data Forwarder code 
I’ve just used the Arduino deployment option with the Static Buffer example and it seems to be working well - @oveddan have you seen that since posting?
If I wanted to have continuous live classification of an analog input on the Nano 33 BLE Sense what would be the best way to approach this and would you have any examples? Currently unpicking the accelerometer example to see if that will do the job.
If I wanted to have continuous live classification of an analog input on the Nano 33 BLE Sense what would be the best way to approach this and would you have any examples? Currently unpicking the accelerometer example to see if that will do the job.
The accelerometer does discrete samples right now - we’ll be pushing one that does continuous inferencing somewhere in the near future. I assume inferencing is pretty fast on your buffer, so I’d have a rolling buffer (e.g. a few seconds of data, shift the buffer so the first bytes drop off, fill the last bytes with new readings) and then run inferencing every 100 ms. or so.
Thanks, that’s great. I just switched out the IMU specific parts from the exported sketches and it’s working a treat 
I think every 100ms will be more than sufficient for my purposes!
I’m also curious about using the Arduino Nano 33 IoT for on-the-fly training, as it has WiFi. Would it work? Looks like they have different architectures (Mbed and SAMD), so I guess it would need a bit more work to port it?
It will probably run as-is (I’ve tested this on the MKR1000) but maybe not with hardware optimizations enabled, so might be slower.
Could you explain how you got live classification on the nano 33 ble sense running?
EDIT: My bad, it is all there. Instead of opening the “static buffer” from the library, I just had to use the example sketch “nano_ble33_sense_accelerometer”. It works like a charm now!
@janjongboom
I might be helpful for beginners to add the information, that 1: the library Arduino_LSM9DS1 must be installed and 2: that the above mentioned sketch must be run for live classification. Thanks for all this great work!
@Marty21 thanks! The library is actually a dependency of the library we generate but didn’t realise the Arduino Ide doesn’t automatically resolve those dependencies. Will add it to the docs!
Do you to have Arduino Nano 33 BLE Sense connected via Wire all the time or can we use Bluetooth 5 communication to collect data and do live infrancing?
@kmashal, we don’t currently have data collection over BLE, but you can use ArduinoBLE in your sketch to send the inference results anywhere. See Incompatible with ArduinoBLE? for some background.
I have added to the default example code for the BLE Sense to get it to send the classifications over BLE as a notification. I’ve put in a gist in case it’s of use - warning it is rough!