Question/Issue:
[I am facing an issue when integrating sensor data (MQ-2, MQ-3, MQ-135, temperature, and humidity) with image data (captured by the ESP32-CAM) in Edge Impulse.]
Project ID:
[867263]
[835630]
Context/Use case:
[ system is designed to predict the early stages of food spoilage. We are using components such as NodeMCU, ESP32-CAM with baseboard, MQ gas sensors (MQ-2, MQ-3, MQ-135), a DHT22 sensor, ADS1115, breadboard, jumper wires, and resistors. We have successfully established POST HTTP communication between the NodeMCU and ESP32-CAM. Sensor readings were collected using the Arduino IDE and saved as CSV files with proper labels, including: fresh_apple, spoiled_apple, fresh_banana, spoiled_banana, fresh_chikoo, and spoiled_chikoo. Additionally, we used Edge Impulse to collect image data of the same fruits, completed the deployment stage, and generated the ZIP file. Our current task is to integrate the sensor data (CSV files) with the image data and accurately predict food spoilage. We seek guidance on performing this integration and prediction process for our project]
Steps Taken:
- [ Collected sensor data (MQ-2, MQ-3, MQ-135, temperature, humidity) using NodeMCU and DHT22, and saved it as labeled CSV files (
fresh_apple,spoiled_apple, etc.) via Arduino IDE.] - [ Captured image data of fruits using ESP32-CAM and uploaded it to Edge Impulse for processing.]
3.[Trained and deployed a model in Edge Impulse for sensor-based classification and generated the deployment ZIP file.]
Expected Outcome:
[The system is expected to accurately detect and predict food spoilage by combining gas sensor data, temperature and humidity readings, and image data using an Edge Impulse–based AI model, enabling early spoilage detection and reduced food wastage]
Actual Outcome:
[Although sensor-based spoilage detection was achieved, integrating multimodal data (sensor and image) within Edge Impulse posed implementation challenges]
Reproducibility:
- [ ] Always
- [✓] Sometimes
- [ ] Rarely
Environment:
- Platform: [ESP32-CAM, Nodemcu.]
- Build Environment Details: [ Arduino IDE 2.3.6 ]
- OS Version: [ Windows 11]
- Edge Impulse Version (Firmware): [not used]
- To find out Edge Impulse Version:
- if you have pre-compiled firmware: run edge-impulse-run-impulse --raw and type AT+INFO. Look for Edge Impulse version in the output.
- if you have a library deployment: inside the unarchived deployment, open model-parameters/model_metadata.h and look for ei-fruit_smell_classifier-arduino-1.0.2, ei-food_vision_model-arduino-1.0.5
- Edge Impulse CLI Version: [Not used]
- Project Version: [e.g., 1.0.0]
-
Custom Blocks / Impulse Configuration: [Time-series sensor data processing using gas sensors (MQ-2, MQ-3, MQ-135) and DHT22 with a classification learning block.]
Logs/Attachments:
[ https://mail.google.com/mail/u/0?ui=2&ik=e66de27088&attid=0.1&permmsgid=msg-f:1853673608678814709&th=19b992536ed133f5&view=att&disp=safe&realattid=f_mjqw2m8h3&zw and https://mail.google.com/mail/u/0?ui=2&ik=e66de27088&attid=0.2&permmsgid=msg-f:1853673608678814709&th=19b992536ed133f5&view=att&disp=safe&realattid=f_mjqw2m862&zw ]
Additional Information:
[ We tried to integrate both sensor data and image data using Google Gemini and ChatGPT, but these approaches did not work as expected. Therefore, we created two separate projects in Edge Impulse: the image project (ID: 835630) and the sensor project (ID: 867263). We completed the deployment and generated separate ZIP files for both image( ei-food_vision_model-arduino-1.0.5 )and sensor data(ei-fruit_smell_classifier-arduino-1.0.2). These libraries were installed in the Arduino IDE, but the sensor ZIP library failed to upload—the processing bar got stuck, even though all sensors are connected to the NodeMCU. Currently, we are getting outputs separately for images and sensors. We need suggestions on how to integrate both types of data, and we are open to making any necessary changes to improve the system.]