Errors generating features

#1

Hi,

I am trying to generate feature using the attached data using Spectral features however i am seeing the below errors during the feature extraction.

what could be the issue? does edgeimpuse expects the same number of columns in all files ?

Creating job… OK (ID: 1147367)

Scheduling job in cluster…
Job started
Creating windows from 5 files…
[1/5] Creating windows from files…
WARN: failed to process 42559/training/upstairs.csv.2bffklun.ingestion-64f68c8458-wwdf7.json: Shape of data does not match earlier data. Expected 49 columns, but got 55. Does all your data have the same frequency?
WARN: failed to process 42559/training/standing.csv.2bffkhdl.ingestion-64f68c8458-6zp5w.json: Shape of data does not match earlier data. Expected 49 columns, but got 55. Does all your data have the same frequency?
WARN: failed to process 42559/training/sitting.csv.2bffk8nq.ingestion-64f68c8458-wwdf7.json: Shape of data does not match earlier data. Expected 49 columns, but got 130. Does all your data have the same frequency?
WARN: failed to process 42559/training/jogging.csv.2bffjlsm.ingestion-64f68c8458-jbkmw.json: Shape of data does not match earlier data. Expected 49 columns, but got 67. Does all your data have the same frequency?
[5/5] Creating windows from files…
Created 2155 windows: jogging: 0, sitting: 0, standing: 0, upstairs: 0, walking: 2155

Creating features
[ 1/2155] Creating features…
[ 975/2155] Creating features…
[1947/2155] Creating features…
[2155/2155] Creating features…
Created features

Job completed

Data files in CSV - https://github.com/gsreddy99/har/tree/srinivas-local-copy/EDGEIMPULSE-DATA

#2

@gsrinivas Yes, you need to have the same number of columns and the same names for the columns in all files.

#3

From what I see in your CSV files, the timestamp intervals need to be the same (sampling at the same frequency).

In real conditions, your accelerometer will record data at a given frequency to fill a buffer of a fix size and then pass this buffer to the run_classifier() function.

One trick that you could do is to “force” the timestamp of the CSV file to be as close as possible to the one you used in your data collection.

Like this for a sampling rate of 50Hz (20ms interval)

timestamp, X (mg), Y (mg), Z (mg)
0, number, number, number
20, number, number, number
40, number, number, number

Regards,

Louis

1 Like
#4

Thank you. I will ignore the timestamp from SensorTile and add an incremental value to the timestamp.

1 Like