Inconsistent data acquisition/frequency (using dataforwarder)

Hi all,

I am using an arduino nano 33 BLE SENSE together with 3 IMUs and 1 multiplexer to get 33 sensor data axis.
I was able to collect 30 min of data at 27Hz in one sitting, but the next day the detected data frequency was different (using the same script). I.e. it was 28Hz but later also 25 Hz or 26 Hz.

  1. how important is it for the ML-algorithm to have all its data at the same frequency?
  2. is there an arduino code example that can collect and serial.print data at a constant rate? (I had a hard time finding it)

I came up with this snippet:

unsigned long currentMillis = millis(); //check current time

// only print if time is more than interval
if (currentMillis - previousMillis >= interval) {
    // save the last time you printed
    previousMillis = currentMillis;
    Serial.println(output); //end of datapoint, print all the data stored in "output"
}

In which output is a string in which I stepwise append all the collected data, TAB-seperated ofcourse. (e.g. IMU.readAcceleration(ax, ay, az); //in g
output += ax * CONVERT_G_TO_MS2;
output += ‘\t’;
)
The problem with above snippit is that the data forwarder sometimes detected even 50hz, and other times 27Hz. The interval for which 27 hz sometimes was detected was hardcoded as 22 milliseconds, not the expected 37. I have the feeling above code is insufficient…

  1. In model testing I got the following (logical) error:
    WARN: failed to process 8035/testing/testing.json.1in8p5in.ingestion-fc6dbb8c8-lzxgp.json: Frequency does not match. Expected 27Hz, but got 28.00000000000001Hz

  2. If there is any way to fix the data acquisition rate, within edge impulse IDE or arduino, that would be great.

Any help is appreciated! I love the edge impulse interface, and the way it makes ML accessible, but above edge-cases are quite hard to wrap you head around on your own!

With kind regards,
Rick

1 Like

Hi @Rick1, this works for me to pin the accelerometer to 50Hz…

#include <Arduino_LSM9DS1.h>

#define CONVERT_G_TO_MS2    9.80665f
#define FREQUENCY_HZ        50
#define INTERVAL_MS         (1000 / (FREQUENCY_HZ + 1))

void setup() {
    Serial.begin(115200);
    Serial.println("Started");

    if (!IMU.begin()) {
        Serial.println("Failed to initialize IMU!");
        while (1);
    }
}

void loop() {
    static unsigned long last_interval_ms = 0;
    float x, y, z;

    if (millis() > last_interval_ms + INTERVAL_MS) {
        last_interval_ms = millis();

        IMU.readAcceleration(x, y, z);

        Serial.print(x * CONVERT_G_TO_MS2);
        Serial.print('\t');
        Serial.print(y * CONVERT_G_TO_MS2);
        Serial.print('\t');
        Serial.println(z * CONVERT_G_TO_MS2);
    }
}

I was able to collect 30 min of data at 27Hz in one sitting, but the next day the detected data frequency was different (using the same script). I.e. it was 28Hz but later also 25 Hz or 26 Hz.

Should not be too big of an issue normally. If we see a frequency detected that’s within 10Hz of the last detected frequency we keep the old one (for consistency in the dataset). I’m wondering how the 28Hz samples came into play though. As you can see in the UI it’s important to have a consistent dataset, as we cannot classify data at a different frequency.

Two solutions for your immediate issue (data in the wrong frequency):

  1. In your home directory (on Mac e.g. ~) there is a file called edge-impulse-config.json. It contains all devices configured by the data forwarder with their frequencies. You can set the desired frequency there and as long as detected frequency is within 10Hz that is used:
{
    "dataForwarderDevices": {
        "EA:CC:51:05:F2:F4:3D:C1": {
            "projectId": 8530,
            "apiKey": "ei_de...",
            "hmacKey": "6c...",
            "samplingFreq": 48,
            "sensors": [
                "accX",
                "accY",
                "accZ"
            ]
        }
    }
}
  1. I’ve added a new flag to the data forwarder (--frequency). If you run the data forwarder with this flag the frequency will be overriden. E.g.:

    edge-impulse-data-forwarder --frequency 100
    

    This is released in CLI version 1.8.3 (npm update -g edge-impulse-cli).

1 Like

Thank you for your suggestions and fixes!
I will try them out on my system asap.

Keep up the good work!

1 Like

It worked on my system, thanks for the support!

1 Like