Upload dataset in CSV file to Edge Impulse

Hi,
I want to upload a dataset on edge impulse project but the dataset is a CSV file. How to upload the data?
I tried using the API key but it responds wrong API key. Please help.

Hi @AashiDutt,

For now the best way is to convert your csv to json, you can follow the data acquisition format documentation . Regarding the API key, can you check it is correctly copied? It should be 68 characters long.

I will also add the csv import feature in our backlog.

Aurelien

Hi Aurel,
Thanks for your guidance. But i seem to run into another error.Could you suggest, whether their is some problem in conversion or other?
Error: Failed to upload 6ip6n-5u4rm.json Missing protected header

I tried with numerous json files but got the same error.

Thanks

@AashiDutt we need some metadata about the device (in the protected header). Here’s an example in Python on converting CSV:

test.csv

timestamp,accX,accY,accZ
1596181208106,9.81,0.11,0.13
1596181208122,9.71,0.14,-0.27
1596181208138,9.83,0.12,0.01

convert.py

import csv, json, math, hmac, hashlib

header = None
# keep track of the first row to know the beginning timestamp
first_row = True
begin_ts = 0
next_ts = 0
values = []

HMAC_KEY = "fed53116f20684c067774ebf9e7bcbdc"

# Parse the CSV file
with open('test.csv', newline='') as csvfile:
    rows = csv.reader(csvfile, delimiter=',')
    for row in rows:
        if (not header):
            header = row
            continue

        if not begin_ts:
            begin_ts = int(row[0])
        elif not next_ts:
            next_ts = int(row[0])

        # skip over timestamp column, and add the rest
        values.append([ float(x) for x in row[1:] ])

# empty signature (all zeros). HS256 gives 32 byte signature, and we encode in hex, so we need 64 characters here
emptySignature = ''.join(['0'] * 64)

# This is the Edge Impulse Data Acquisition Format, it has the protected header
data = {
    "protected": {
        "ver": "v1",
        "alg": "HS256",
        "iat": math.floor(begin_ts / 1000) # epoch time, seconds since 1970 (the timestamp earlier was in ms.)
    },
    "signature": emptySignature,
    "payload": {
        "device_type": "CSV_IMPORTER",
        "interval_ms": next_ts - begin_ts,
        "sensors": [ { "name": x, "units": "m/s2" } for x in header[1:] ],
        "values": values
    }
}

# encode in JSON
encoded = json.dumps(data)

# sign message
signature = hmac.new(bytes(HMAC_KEY, 'utf-8'), msg = encoded.encode('utf-8'), digestmod = hashlib.sha256).hexdigest()

# set the signature again in the message, and encode again
data['signature'] = signature
encoded = json.dumps(data)

print(encoded)

Thanks🙂, will give it a try.