Question/Issue:
Using the Bosch bme688 kit, I have captured the odor/gas resistances, which I have processed and converted into png images. These png images of different odors are the training and testing data for my neural networks created with edge impulse.
The purpose is to distinguish odors, so I have created the different projects:
This is the ID of the edge impulse project that differentiates between air and coffee: 563841
This is the ID of the edge impulse project that differentiates between cinnamon and cocoa: 578418
This is the ID of the edge impulse project that differentiates between air, coffee, cinnamon and cocoa: 578439
The data of the neural network that differentiates air, coffee, cinnamon and cocoa is exactly the same data as the air vs coffee and cinnamon vs cocoa projects.
Although I get pretty good results with the paired odor projects (air vs coffee and cinnamon vs cocoa), when I do a project with the data from the 4 odors, the results get dramatically worse:
Air vs coffee: Accuracy training → 100%, Accuracy testing → 86.79%
Cinnamon vs cocoa: Accuracy training → 98.9%, Accuracy testing → 81.17%
Air vs coffee vs cinnamon vs cocoa: Accuracy training → 95.6%, Accuracy testing → 66.59%
The idea is to be able to differentiate 4 or more odors in real time thanks to a bme688 sensor connected to an esp32 wroom. To do this, my idea is to download the project in the deployment section as an arduino library with the model optimizations: tensorFlow Lite.
I have downloaded and included these libraries in the Arduino IDE and used the static buffer example. I have tested them with the values from the same project and the example works perfectly.
The problem I have is that the more data from different smells I try to add to my project, the worse the results are, as we can see in the accuracy values of the air vs coffee vs cinnamon vs cocoa project.
Therefore, I had the idea of making a single Arduino project with the two libraries from the coffee vs air and cinnamon vs cocoa projects. In this way I can pass my data through the two neural networks and keep the best percentage of success from the two neural networks. In this way I could differentiate between the 4 smells but with greater precision than using the air vs coffee vs cinnamon vs cocoa library directly.
The problem when trying to compile the Arduino code, from the following section, with both libraries is that both share many files and variables and therefore they overlap, the program does not compile. In the Type of errors compiling the arduino code section, you can see some of the errors that appear when compiling the arduino code (not all of them are shown because they are more than 1000 lines)
I would like to be able to use both libraries in the same arduino code. Is this possible?
Arduino code
This is the arduino code (based on the static buffer code from the examples) that should feed the features vector into both neural networks and give the results.
// Combined Edge Impulse inference example
// This code demonstrates how to run inference using two different neural networks
// generated by Edge Impulse and compare the results.
#include <Demo_air_coffee_2_png_inferencing.h> // Include first neural network
#include <Demo_canela_cacao_png_inferencing.h> // Include second neural network
static const float features[] = {
// Here are the model features.
// Copy the raw data from the ‘Live Rankings’ page.
0x0, 0x10101, 0x10101, 0x20202, 0x20202, 0x393939, 0xaaaaaa, 0xd8d8d8, 0xefefef, 0xffffff, 0x2f2f2f, 0x8b8b8b, 0xadadad, 0xbfbfbf, 0xcbcbcb, 0x242424, 0x626262, 0x787878, 0x858585, 0x8c8c8c, 0x20202, 0x70707, 0x90909, 0xa0a0a, 0xa0a0a, 0x20202, 0x70707, 0x80808, 0x90909, 0x90909, 0x20202, 0x60606, 0x80808, 0x80808, 0x80808, 0x0, 0x0, 0x10101, 0x10101, 0x10101, 0x0, 0x10101, 0x10101, 0x10101, 0x10101, 0x10101, 0x10101, 0x10101, 0x10101, 0x20202
};
// @brief Copia los datos crudos de características en out_ptr
// Función llamada por la librería de inferencia
//
// @param[in] offset El desplazamiento
// @param[in] length La longitud
// @param out_ptr El puntero de salida
//
// @return 0
//
int raw_feature_get_data(size_t offset, size_t length, float *out_ptr) {
memcpy(out_ptr, features + offset, length * sizeof(float));
return 0;
}
void print_inference_result(ei_impulse_result_t result, const char* model_name);
void setup() {
Serial.begin(115200);
while (!Serial);
Serial.println(“Edge Impulse Inferencing Demo”);
}
void loop() {
ei_printf(“Inferencing con Edge Impulse (Arduino)\n”);
if (sizeof(features) / sizeof(float) != EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE) {
ei_printf("El tamaño de tu arreglo 'features' no es correcto. Se esperaban %lu elementos, pero hay %lu\n",
EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, sizeof(features) / sizeof(float));
delay(1000);
return;
}
ei_impulse_result_t result1 = { 0 }; // Resultados de la primera red neuronal
ei_impulse_result_t result2 = { 0 }; // Resultados de la segunda red neuronal
// Configuration for features signal_t features_signal;
features_signal.total_length = sizeof(features) / sizeof(features[0]);
features_signal.get_data = &raw_feature_get_data;
// Run the inference on the first neural network
EI_IMPULSE_ERROR res1 = run_classifier(&features_signal, &result1, false);
if (res1 != EI_IMPULSE_OK) {
ei_printf("ERR: Falló la ejecución del clasificador 1 (%d)\n", res1);
return;
}
// Run the inference on the second neural network
EI_IMPULSE_ERROR res2 = run_classifier(&features_signal, &result2, false);
if (res2 != EI_IMPULSE_OK) {
ei_printf("ERR: Falló la ejecución del clasificador 2 (%d)\n", res2);
return;
}
// Print results of both neural networks
print_inference_result(result1, "Cinnamon vs Cocoa Model");
print_inference_result(result2, "Air vs Coffee Model");
delay(1000);
}
void print_inference_result(ei_impulse_result_t result, const char* model_name) {
// Print the model name and inference time
ei_printf(“%s - Tiempos: DSP %d ms, inferencia %d ms, anomalía %d ms\r\n”,
model_name,
result.timing.dsp,
result.timing.classification,
result.timing.anomaly);
// Print the sorting results
ei_printf(“%s - Predicciones:\r\n”, model_name);
for (uint16_t i = 0; i < EI_CLASSIFIER_LABEL_COUNT; i++) {
ei_printf(" %s: %.5f\r\n", ei_classifier_inferencing_categories[i], result.classification[i].value);
}
// If the model has anomaly prediction, print it
#if EI_CLASSIFIER_HAS_ANOMALY
ei_printf(“%s - Predicción de anomalía: %.3f\r\n”, model_name, result.anomaly);
#endif
#if EI_CLASSIFIER_HAS_VISUAL_ANOMALY
ei_printf(“%s - Anomalías visuales:\r\n”, model_name);
for (uint32_t i = 0; i < result.visual_ad_count; i++) {
ei_impulse_result_bounding_box_t bb = result.visual_ad_grid_cells[i];
if (bb.value == 0) {
continue;
}
ei_printf(" %s (%f) [ x: %u, y: %u, width: %u, height: %u ]\r\n",
bb.label,
bb.value,
bb.x,
bb.y,
bb.width,
bb.height);
}
#endif
}
Type of errors compiling the arduino code
c:/users/laura.barrero/appdata/local/arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/gcc8_4_0-esp-2021r2-patch3/bin/…/lib/gcc/xtensa-esp32-elf/8.4.0/…/…/…/…/xtensa-esp32-elf/bin/ld.exe: C:\Users\laura.barrero\AppData\Local\Temp\arduino\sketches\0CBBDE0BA1D8CEC5237A3A1D0CE9FF29\libraries\Demo_canela_cacao_png_inferencing\edge-impulse-sdk\tensorflow\lite\micro\memory_planner\objs.a(non_persistent_buffer_planner_shim.cpp.o): in function tflite::NonPersistentMemoryPlannerShim::NonPersistentMemoryPlannerShim(tflite::BufferPlan const*)': c:\Users\laura.barrero\OneDrive - Smarting\Documentos\Arduino\libraries\Demo_canela_cacao_png_inferencing\src\edge-impulse-sdk\tensorflow\lite\micro\memory_planner/non_persistent_buffer_planner_shim.cpp:22: multiple definition of tflite::NonPersistentMemoryPlannerShim::NonPersistentMemoryPlannerShim(tflite::BufferPlan const*)'; C:\Users\laura.barrero\AppData\Local\Temp\arduino\sketches\0CBBDE0BA1D8CEC5237A3A1D0CE9FF29\libraries\Demo_air_coffee_2_png_inferencing\edge-impulse-sdk\tensorflow\lite\micro\memory_planner\objs.a(non_persistent_buffer_planner_shim.cpp.o):c:\Users\laura.barrero\OneDrive - Smarting\Documentos\Arduino\libraries\Demo_air_coffee_2_png_inferencing\src/edge-impulse-sdk/tensorflow/lite/micro/memory_planner/non_persistent_buffer_planner_shim.cpp:22: first defined here
collect2.exe: error: ld returned 1 exit status
exit status 1
Compilation error: exit status 1
Project ID:
563841: air vs coffee
578418: cinnamon vs cocoa
578439: air vs coffee vs cinnamon vs cocoa
Environment:
- Platform: ESP32 wroom (in Arduino IDE the board is: ESP32-WROOM-DA Module)
- Build Environment Details: Arduino IDE 2.3.3, ESP32 Core for Arduino 2.0.4 (installed via ESP32 by Espressif Systems Board Manager)
- OS Version: Windows11
- Edge Impulse Version (Firmware):[Not applicable or unknown; Edge Impulse is accessed via Google Chrome and Microsoft Edge]