After downloading the model as a C++ library we have all the required files and dependencies apart from the main.cpp file.
This main.cpp file is platform agnostic, correct me if am wrong.
My doubt is, how to generate this main.cpp, is there any resource available ??
I have not tested it myself but from the documentation I found this link: https://docs.edgeimpulse.com/docs/deploy-your-model-as-a-c-library#create-an-application
Thank you @noone, even I found the same but that code seems to be specific to object detection use-case.
I want the main.cpp for audio classification or atleast a template that will help me create the main.cpp.
Ok, hopefully someone else will chime in an help you further.
Hello @LakshmiNarayanan,
That main.cpp is generic, you can test it directly with your raw features by copy pasting them here:
static float input_buf[] = {
/* Paste your raw features here! */
};
It should work with all the project types (images, audio, time-series).
As this is a template, you will need to implement how you want to sense or grab the raw data from the sensors to fill that input_buf[]
There is indeed a if statement if you use object detection it will not enter in that statement if you use a different project type
Regards,
Louis
Thank you so much @louis, the main.cpp is generic like u had mentioned and it worked perfectly for my audio classification model.
But I had to manually enter the processed features as an input to the input_buf[], how can I directly gather input from the sensor ? I want to perform the inferencing live.
The document says " For performing inference live, you would want to fill the features[] array with values from a connected sensor."
How do I do this step, could you help me out with this please ?
I am using an Arduino Nano 33 BLE sense board to demonstrate a usecase on Audio Classification.
Also, how to use run_classifier_continuous in the main.cpp ?
You’ll have to capture data from the sensor and pass it to run_classifier_continuous(). For an example of how to do this for the Arduino Nano 33 BLE sense you can export a Arduino Library from the deployment page and take a look at the examples (nano_ble33_sense_microphone_continuous.ino). Or you can also look at our firmware example for ingestion and inference with sensor support: https://github.com/edgeimpulse/firmware-arduino-nano-33-ble-sense
Raul
I am new to this TinyML and Edge Impulse and also the mentioned suggestions had more depth for me to consume.
Please take out a few minutes to go through this completely
Let me share a clear picture on what I am trying to do,
#include “edge-impulse-sdk/classifier/ei_run_classifier.h”
// Callback function declaration
static int get_signal_data(size_t offset, size_t length, float *out_ptr);
// Raw features copied from test sample (Edge Impulse > Model testing)
static float input_buf[650] = {
-1.2499, 2.6768, 1.1546, 1.2468, 0.1183, -0.8606, 0.5514, -0.2750, 0.0352, 0.3802, 0.5430, 0.7830, -1.2270, -1.2546, 1.0519, 0.9000, 0.9674, -1.1848, 2.5841, 0.0741, 2.5454, -1.0782, -0.5412, 0.2570, 0.4586, 0.9596, -1.2188, 1.8183, -0.0931, -0.3918, -0.8104, 0.1562, -0.4991, 2.0208, 0.1922, -0.1200, 1.0671, -0.0562, 0.7494, -1.2340, 1.1815, 0.5990, 0.5382, 0.5212, 1.4408, 0.1510, 0.8330, 0.7513, -1.3334, -0.0265, 0.1758, -0.4882, -1.2242, 1.4035, 0.9996, 1.1769, 1.3072, 0.0218, -0.2021, -0.4501, 0.4524, -0.5441, -0.1965, -0.0056, 0.3877, -1.1437, 4.2140, -2.4306, 0.2165, 0.3267, -1.7997, 0.5458, -1.2066, -0.3113, 0.2499, 0.6801, 0.5837, -0.9795, 2.6413, -0.0557, 2.2754, 1.4062, -0.8807, -0.1946, -0.6090, 1.2311, 0.3741, -0.0561, -0.0350, -0.8655, 0.1114, 1.8805, -0.5963, 1.1900, 1.0974, 0.6903, 0.1867, -1.4703, 1.4858, 0.5481, -1.1022, -0.2616, -2.2635, -0.2390, 1.4730, -0.2513, 0.9505, 0.6225, 1.4774, 0.9229, -2.3754, -0.6783, 0.1134, -0.1160, -0.1137, 0.1516, 0.9437, 0.8827, -1.3946, -0.4132, 0.5484, 1.2453, 1.1125, -0.8156, 1.8219, 2.3342, -0.7384, -1.1011, -1.5087, 1.2643, 0.3536, -0.3045, -0.0212, 0.1655, 0.8864, -1.0856, -0.9615, 0.4886, 1.2751, 0.7456, 0.5725, 0.2442, 0.4768, 0.1025, 0.0006, -0.0542, -0.8683, 0.3927, -0.5324, -1.2179, 1.2343, -0.6452, -0.8230, 0.4758, 0.3758, 1.6095, -0.0201, -0.5008, 0.1145, -0.6909, -1.0810, 0.5425, -2.0149, 1.1232, 0.9431, 0.3365, 0.8646, 0.5684, 1.3273, -0.4915, -0.3801, -0.5082, -0.0903, 1.0598, -0.9106, -2.5140, 0.1106, 2.2969, 1.6695, 0.5234, 0.3436, 0.1380, -0.7403, -0.8652, -0.9501, -0.4303, 0.4751, 0.5735, -0.5013, 1.0516, 2.1936, 0.9314, 0.2836, -0.0919, -0.0361, -0.8775, -0.2045, 0.7949, 1.1270, 1.9294, 1.2358, -1.6967, 0.0842, 0.2908, -0.0840, -0.0643, -1.8292, -1.5515, -0.8808, 0.5194, 2.1695, 0.4121, 0.2605, -0.4362, -2.1628, 0.5926, 1.1842, -0.3003, 0.0787, -0.6281, 0.5580, -0.1636, -0.9416, 1.7522, 0.6324, 0.1720, -1.5335, -0.7481, -0.6037, -0.5189, -0.9699, 0.1952, 0.4837, 0.9873, -0.1894, -0.2698, -0.6465, -1.2588, -1.1057, -1.9926, 0.3584, 1.3303, 1.6969, 1.6630, 2.3969, 1.0194, 1.7365, 0.2493, -1.3367, -0.7779, -2.1001, -1.1804, -0.6956, 0.0188, -0.6577, -0.3948, -1.6594, 0.4544, 1.7831, -0.6245, 0.1458, -0.8502, -1.2709, -1.6817, -0.6232, 0.3814, 0.8332, -0.3402, 0.3171, 0.9163, 0.9521, 1.4266, -0.1704, -0.3607, -0.1349, 0.2583, 0.1369, -0.0131, -0.0818, 0.4627, -0.6596, -0.1702, -0.9831, -0.3631, 1.0384, 0.7707, -0.4708, -0.2704, -0.1136, -0.8103, 0.0101, 0.3382, 0.4356, -0.2629, -0.1764, -1.2728, -1.0845, 2.1899, 0.5767, 1.1988, -0.3980, 0.2624, -2.9597, -1.5673, -1.3922, -0.7308, -0.5957, -0.9519, 0.2713, 2.0375, 0.2142, -2.9868, 2.3510, -0.7834, 1.7764, 1.2349, 1.9651, 1.5906, -0.3835, -0.0288, -3.1145, -0.8341, 0.9734, 0.8341, 1.1604, 1.3021, -0.5905, 0.6965, 0.3989, -0.5648, -0.0349, 0.8043, 0.8938, 0.4020, -0.1462, -0.0393, -0.4184, -0.6744, 1.8611, -1.4521, -0.7900, -1.0773, -1.7050, 0.2634, 0.7486, 0.6555, -0.4314, 0.7569, 1.0457, -0.0088, 0.9132, 1.4880, -0.7121, 0.2905, -0.6281, 0.7429, 0.8565, -0.5003, -0.8603, -0.5170, -0.0183, 0.7349, 0.8274, -0.4880, 0.8017, -0.4514, -0.3329, -2.0800, -1.0348, -0.6140, 0.3866, -0.1326, -0.2271, -0.1285, -0.3148, -0.4459, -0.1219, 0.6354, -0.1489, 0.4691, -0.9354, -0.9899, 0.3824, 0.1153, 0.3246, -0.3345, -1.2648, 0.1997, 0.6062, 1.1702, 0.0790, 0.5600, 0.4204, -0.5946, 0.2163, -0.0961, 0.2981, -0.4108, -0.2814, -0.0356, 1.3450, -0.4295, -1.2381, 0.0949, -0.6943, -1.4402, -0.9571, 0.5443, 1.3988, 1.7599, -0.3452, 0.1238, 1.2957, 0.0297, 0.9006, 1.1569, -0.2574, -0.4848, -1.0703, -0.3549, -0.0611, 0.0307, 0.9121, -0.4069, -0.4893, 1.9344, -0.5394, -1.4208, 0.3240, -0.4135, -0.3014, -0.1434, 0.0769, -0.8301, -0.2821, 1.8794, -1.5452, 0.1619, 2.2248, -0.7793, -1.8576, 0.3719, -0.7187, 0.3127, -0.8896, -0.3688, 0.5978, -0.2874, 1.4370, -1.9091, -1.6616, -0.4741, -1.2252, -0.4728, 1.1965, -0.9816, 0.5569, -0.9560, 0.2558, 1.3678, -0.8981, -0.1574, -1.2985, -0.4120, -0.2215, -0.3412, -0.2679, -0.0667, -1.1195, 0.5197, -0.7390, 0.4548, 1.2960, 1.2947, 0.8558, -0.5529, -0.3056, -0.7635, -0.3799, -1.0532, -0.7795, -1.1192, 0.4260, -1.0613, 0.9841, 0.7645, -0.3807, 0.4231, -1.0814, -1.4607, -0.9601, 0.0613, 0.0268, -0.9215, -1.1432, 0.8836, -0.7038, -0.6130, -0.1840, 0.4796, 0.2611, 0.0499, -0.8922, -0.9245, 0.0673, 0.2462, 0.0818, -1.0521, 0.4610, -0.9995, 0.3457, 0.9988, 1.2092, 0.8116, -1.3501, -0.5111, -0.6645, -1.4294, -2.7762, -0.8232, -0.9438, -0.3502, -1.6607, 0.4585, -0.4028, 1.3806, 0.7213, -0.8204, 0.2284, 2.6747, 0.6207, 0.6294, -0.3174, 0.1515, -0.5546, 0.4160, 0.6910, 0.1201, 0.2506, 0.5360, 0.5020, -0.1456, 0.9495, -0.9575, 0.0657, -0.0933, 0.3204, 0.0795, 0.0935, -0.5939, 0.6895, 0.6346, 0.3688, 0.1165, -1.0502, 0.1651, -1.8505, -0.9234, 0.1954, -0.2101, 0.5405, 0.1018, 0.6296, 0.0792, 0.3710, 0.4573, -0.2430, -0.6287, 0.2346, -0.4585, 0.8887, 0.5686, 0.0048, -0.2555, 1.3553, 2.3282, -0.4329, -0.4439, -0.3061, -2.1576, -0.1185, 2.0488, -2.7107, -1.2404, -1.2380, -0.1493, -0.3911, 1.0617, 0.4128, -2.2894, -2.4464, 0.6353, 0.3029, 0.9588, -0.1824, -2.6501, 0.6866, 0.2896, 1.1264, -1.1399, -1.1297, 0.2234, -1.3528, -0.2605, 0.6773, -0.2977, -1.4831, -1.0822, -0.9874, -0.6242, -2.1775, 0.5092, -0.1660, -0.1334, 0.7685, -0.0576, -0.2275, 0.5766, 1.0440, 0.7241, -0.8571, -0.8936, -0.2402, -0.0721, 0.1206, -0.6613, -0.6396, -0.6692, -1.9594, -0.8309, 1.1808, 0.3220, -0.3420, 0.0383, 1.1603, 0.8848, -1.5100, -0.2180, 0.6974, -0.1491, 0.6866, 0.0809, -1.3089, 0.5330, -0.9887, 1.0428, -0.2611, 1.1262, 0.9829, -1.2083
};
int main(int argc, char **argv) {
signal_t signal; // Wrapper for raw input buffer
ei_impulse_result_t result; // Used to store inference output
EI_IMPULSE_ERROR res; // Return code from inference
// Calculate the length of the buffer
size_t buf_len = sizeof(input_buf) / sizeof(input_buf[0]);
// Make sure that the length of the buffer matches expected input length
if (buf_len != EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE) {
printf("ERROR: The size of the input buffer is not correct.\r\n");
printf("Expected %d items, but got %d\r\n",
EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE,
(int)buf_len);
return 1;
}
// Assign callback function to fill buffer used for preprocessing/inference
signal.total_length = EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE;
signal.get_data = &get_signal_data;
// Perform DSP pre-processing and inference
res = run_classifier(&signal, &result, false);
// Print return code and how long it took to perform inference
printf("run_classifier returned: %d\r\n", res);
printf("Timing: DSP %d ms, inference %d ms, anomaly %d ms\r\n",
result.timing.dsp,
result.timing.classification,
result.timing.anomaly);
// Print the prediction results (object detection)
#if EI_CLASSIFIER_OBJECT_DETECTION == 1
printf(“Object detection bounding boxes:\r\n”);
for (uint32_t i = 0; i < EI_CLASSIFIER_OBJECT_DETECTION_COUNT; i++) {
ei_impulse_result_bounding_box_t bb = result.bounding_boxes[i];
if (bb.value == 0) {
continue;
}
printf(" %s (%f) [ x: %u, y: %u, width: %u, height: %u ]\r\n",
bb.label,
bb.value,
bb.x,
bb.y,
bb.width,
bb.height);
}
// Print the prediction results (classification)
#else
printf(“Predictions:\r\n”);
for (uint16_t i = 0; i < EI_CLASSIFIER_LABEL_COUNT; i++) {
printf(" %s: “, ei_classifier_inferencing_categories[i]);
printf(”%.5f\r\n", result.classification[i].value);
}
#endif
// Print anomaly result (if it exists)
#if EI_CLASSIFIER_HAS_ANOMALY == 1
printf(“Anomaly prediction: %.3f\r\n”, result.anomaly);
#endif
return 0;
}
// Callback: fill a section of the out_ptr buffer when requested
static int get_signal_data(size_t offset, size_t length, float *out_ptr) {
for (size_t i = 0; i < length; i++) {
out_ptr[i] = (input_buf + offset)[i];
}
return EIDSP_OK;
}
What changes do I have to make to this main.cpp in-order to perform live inferencing with Arduino Nano 33 BLE.
My final goal is to build the end application and its dependencies into a firmware and flash it on the board. (I know there is a firmware readily available but I wanted to experiment this approach).
Kindly help me out with this.
Thanks,
LN
Hello @LakshmiNarayanan,
Which is your final target? The Arduino Nano 33 BLE?
Is that a requirement for you to use a main.cpp using the C++ library?
Otherwise, if you want to use that with the Arduino Nano 33 BLE Sense, I’d suggest to download the Arduino Library instead of the C++ library. It will contains ready to use examples that you can modify and recompile to suit your needs, such as the microphone_continuous that you can get from the examples once you imported the .zip library in Arduino IDE:
- What changes do I have to make to this main.cpp in-order to perform live inferencing with Arduino Nano 33 BLE.
You have to fill the static float input_buf[650] with the microphone values. For the Arduino Nano 33 BLE Sense, we provide examples when you download the Arduino library from the studio. But you’ll have to implement that on your side if you want to compile your main.cpp directly from the C++ library. Also, feel free to get inspiration on how to do that from the Arduino example.
- My final goal is to build the end application and its dependencies into a firmware and flash it on the board. (I know there is a firmware readily available but I wanted to experiment this approach).
Sounds great, do not hesitate to share your code and steps here with everyone, I’m sure other people could be interested.
Best regards,
Louis
Hi @louis,
Thanks you for the suggestions, I will try both using the IDE, get some inspiration from that code, and implement the continuous inferencing with the C++ library.
Will definitely share my work once its done.
Thanks,
LN