Docs request: worked production example

#1

Hi team
I am picking up edge impulse as a python developer with experience of machine learning and some experience with embedded devices, principally using micro/circuitpython (e.g. here and here). My requests is for a section in the docs (or even just a repo on Github) covering deployment of the trained model, since this will provide the framework for creating my own solutions. I assume C is required, but if a solution with micropython is available, even better! I would suggest something minimal, for example using your audio detection example and exposing the classification data via a rest endpoint on the device, or sending data via MQTT. BTW The project I am planning is a baby detector, to classify the sounds coming from the nursery, or motion of the cot.
Cheers
Robin

Update July 2021 (by @janjongboom): We now have the Edge Impulse Linux SDK with full examples on classifying data locally in Python, Node.js, Go and C++: https://docs.edgeimpulse.com/docs/edge-impulse-for-linux

#2

We have some docs on deploying the model here: running your impulse locally, but yes, this is only C++ for now. There is only one function that you need to invoke there, which is the run_classifier function, e.g. via:

    ei_impulse_result_t result;

    // the features are stored into flash, and we don't want to load everything into RAM
    signal_t features_signal;
    features_signal.total_length = sizeof(features) / sizeof(features[0]);
    features_signal.get_data = &raw_feature_get_data;

    // invoke the impulse
    EI_IMPULSE_ERROR res = run_classifier(&features_signal, &result, true);

Not entirely sure how hard it would be to use e.g. micropython-wrap to make this function callable from the MicroPython environment. If someone that reads this in the future has an idea and wants to contribute that, I’d be very happy to take it!

Regarding a REST endpoint, you can (sort of) do this already, but it requires two REST calls. One to upload the data to the testing endpoint in the ingestion API, and then one to classify the new sample. Here’s an example in Python:

import json, os
import time, hmac, hashlib
import requests, numpy as np

# project ID is in the studio URL, API and HMAC keys can be obtained from Dashboard > Keys
PROJECT_ID = 1
API_KEY = "ei_0de116b6fa6099a831153825b79a08dd6bea65ef05064830966676e50a4cc269"
HMAC_KEY = "5e188e72873a76734fac5fe13e1987c3bad3c473a67354022bdaf3f995e9271f"

# array of your features
features = [ 0.3200, 3.6900, -5.4600, -3.6100, 4.7300, -2.7200, -7.4400, 4.9700, -0.7700, -8.9600, 4.4500, 0.0900, -10.3100, 4.4900, 1.1300, -10.3100, 4.4900, 1.1300, -15.5000, 5.6100, 3.5900, -19.8100, 6.0100, 6.4400, -19.9800, 5.6000, 8.2700, -19.9800, 3.9800, 9.4800, -19.9800, 2.7100, 9.9100, -19.9800, 0.9900, 9.4900, -19.9800, 0.9900, 9.4900, -19.9800, -0.5600, 8.8500, -19.9800, -1.1000, 8.6200, -19.8800, -0.6900, 8.2300, -17.6600, -1.1300, 5.9700, -16.8000, -0.9000, 5.9500, -16.7600, -0.0200, 6.2600, -16.7600, -0.0200, 6.2600, -14.1400, 0.6100, 4.8000, -10.2000, 0.2800, 2.4300, -8.4800, 1.2800, 2.2200, -7.8500, 2.1200, 2.0800, -6.6100, 2.6200, 0.4400, -3.9100, 1.6900, -3.0100, -3.9100, 1.6900, -3.0100, 0.1500, 0.6600, -5.3600, 3.3200, 1.0000, -5.5000, 6.2300, 0.4300, -6.1900, 8.9000, -1.1700, -7.1000, 10.5600, -1.7000, -5.9900, 12.9600, -2.4300, -5.5400, 12.9600, -2.4300, -5.5400, 18.0600, -4.6300, -7.2100, 19.9700, -7.2800, -8.3200, 19.9700, -6.5100, -5.9200, 19.9700, -5.5700, -3.6700, 19.9700, -6.7000, -3.7700, 19.9700, -8.5300, -5.3900, 19.9700, -8.5300, -5.3900, 19.9700, -8.4400, -5.8600, 18.6100, -6.8500, -4.8100, 14.9500, -4.9400, -3.4200, 12.0800, -3.5500, -2.2500, 10.4900, -3.7700, -3.2300, 8.4200, -3.9000, -4.3800, 8.4200, -3.9000, -4.3800, 5.2700, -2.1900, -3.6600, 2.2200, -0.5100, -1.6400, 1.3100, -0.1100, -0.9600, 2.2600, -1.2100, -1.5900, 0.7500, 0.6100, -1.2400, 0.7500, 0.6100, -1.2400, -1.5500, 2.5200, 1.2200, -2.0300, 1.9200, 0.6500, -4.5400, 0.0800, 0.2200, -7.1100, 0.5200, 0.9400, -10.6300, 1.5800, 2.0400, -18.0500, 4.0600, 5.5600, -18.0500, 4.0600, 5.5600, -19.9800, 6.1200, 10.9200, -19.9800, 4.6100, 13.2300, -19.9800, 0.9700, 12.6300, -19.9800, -1.3200, 11.2500, -19.9800, -2.8900, 9.3400, -19.9800, -3.3400, 8.7000, -19.9800, -3.3400, 8.7000, -19.9800, -2.3700, 8.2100, -18.9800, -2.7700, 6.7800, -15.0600, -3.7300, 4.7400, -15.3900, -2.1400, 5.6200, -15.2300, -0.7300, 5.3500, -12.3400, -0.7300, 3.3300, -12.3400, -0.7300, 3.3300, -8.7400, -0.7000, 0.3100, -6.8100, -0.4200, -0.1900, -6.0600, 1.1100, -0.4400, -4.3300, 2.0900, -0.7300, -1.5200, 1.5900, -2.5000, 0.5800, 1.1600, -4.4000, 0.5800, 1.1600, -4.4000, 1.9000, 2.2900, -4.2200, 8.1200, 2.3600, -7.4300, 11.2200, 0.1500, -6.7800, 12.5300, -0.6300, -6.8300, 16.6900, -1.3300, -5.7300, 19.9700, -3.4500, -7.0500, 19.9700, -3.4500, -7.0500, 19.9700, -6.9800, -9.3700, 19.9700, -7.8600, -8.3100, 19.9700, -6.3200, -6.0000, 19.9700, -5.9100, -5.5600, 19.9700, -7.3900, -6.4000, 19.1600, -6.3500, -5.7900, 19.1600, -6.3500, -5.7900, 15.0400, -5.2300, -4.7600, 12.0300, -5.0000, -4.6000, 10.3300, -5.2500, -4.3900, 9.0400, -4.4700, -4.9800, 6.1000, -3.2100, -4.5100, 2.2400, -2.1300, -3.1400, 2.2400, -2.1300, -3.1400, 0.9400, -1.4800, -2.3300, 1.2400, -1.3300, -2.4600, 0.0300, -0.1500, -1.3100, -2.7500, 0.9200, 0.8700, -4.7000, 0.3600, 1.2200, -5.4100, -1.4000, 1.0400, -5.4100, -1.4000, 1.0400, -6.6300, -0.9800, 1.6700, -10.6700, 0.8800, 4.1900, -14.3800, 0.6900, 5.8900, -15.6700, -1.5200, 6.5500, -17.2600, -2.6000, 7.6700, -17.2600, -2.6000, 7.6700, -18.8400, -2.4200, 8.3100, -19.9800, -2.5600, 9.1200, -19.9800, -3.3300, 9.8500, -19.9800, -4.6500, 8.4400, -19.9800, -5.1700, 7.7300, -19.9300, -4.5400, 7.8600, -19.9300, -4.5400, 7.8600, -18.8700, -4.2900, 7.0800, -17.7800, -3.7400, 5.9400 ]
# reshape from 1D to 3D array (not required for audio)
features = np.reshape(features, (int(len(features) / 3), 3))

def upload_file():
    # empty signature (all zeros). HS256 gives 32 byte signature, and we encode in hex, so we need 64 characters here
    emptySignature = ''.join(['0'] * 64)

    data = {
        "protected": {
            "ver": "v1",
            "alg": "HS256",
            "iat": time.time() # epoch time, seconds since 1970
        },
        "signature": emptySignature,
        "payload": {
            "device_type": "DISCO-L475VG-IOT01A",
            "interval_ms": 16,
            "sensors": [
                { "name": "accX", "units": "m/s2" },
                { "name": "accY", "units": "m/s2" },
                { "name": "accZ", "units": "m/s2" }
            ],
            "values": features.tolist()
        }
    }

    # encode in JSON
    encoded = json.dumps(data)

    # sign message
    signature = hmac.new(bytes(HMAC_KEY, 'utf-8'), msg = encoded.encode('utf-8'), digestmod = hashlib.sha256).hexdigest()

    # set the signature again in the message, and encode again
    data['signature'] = signature
    encoded = json.dumps(data)

    # and upload the file
    res = requests.post(url='https://ingestion.edgeimpulse.com/api/testing/data',
                        data=encoded,
                        headers={
                            'Content-Type': 'application/json',
                            'x-file-name': 'testing.01',
                            'x-api-key': API_KEY
                        })
    if (res.status_code == 200):
        print('Uploaded file to Edge Impulse', res.status_code, res.content)
        return res.text
    else:
        raise Exception('Failed to upload file to Edge Impulse', res.status_code, res.content)

def get_sample_id_from_name(file_name):
    filter = '?category=testing&filename=' + os.path.splitext(file_name)[0]

    # query all samples in the testing category
    res = requests.get(url='https://studio.edgeimpulse.com/v1/api/' + str(PROJECT_ID) + '/raw-data' + filter,
                        headers={
                            'Content-Type': 'application/json',
                            'x-api-key': API_KEY
                        })

    # see if the API request succeeded
    ret = res.json()
    if (ret['success'] == False):
        raise Exception(ret['error'])

    # and return the id of the sample
    return ret['samples'][0]['id']

def classify_sample(id):
    res = requests.get(url='https://studio.edgeimpulse.com/v1/api/' + str(PROJECT_ID) + '/classify/' + str(id),
                    headers={
                        'Content-Type': 'application/json',
                        'x-api-key': API_KEY
                    })
    ret = res.json()
    if (ret['success'] == False):
        raise Exception(ret['error'])
    return ret['classifications']

file_name = upload_file()
print('file name is', file_name)

sample_id = get_sample_id_from_name(file_name)
print('sample_id', sample_id)

classifications = classify_sample(sample_id)
print('classifications', classifications)

Which returns the classifications per block:

[{
    'learnBlock': {
        'id': 50,
        'type': 'keras',
        'name': 'NN Classifier',
        'dsp': [41],
        'title': 'Neural Network (Keras)'
    },
    'result': [{
        'idle': 0.000745,
        'snake': 3.5e-05,
        'updown': 0.001455,
        'wave': 0.997766
    }],
    'minimumConfidenceRating': 0.8
}, {
    'learnBlock': {
        'id': 53,
        'type': 'anomaly',
        'name': 'Anomaly detection',
        'dsp': [41],
        'title': 'K-means Anomaly Detection'
    },
    'result': [{
        'anomaly': 0.2667196043261167
    }],
    'minimumConfidenceRating': 0.3
}]

I’ve added an item to the backlog to add a single API call for this, but for now the above should work (maybe add a call to delete the sample afterwards).

1 Like
#3

Thanks for the C example, I have used Arduino in the past so can probably hack something together.

RE rest, the inferencing is actually being done on your servers in this case, or on the board?

RE micropython-wrap that looks promising, but appears it would require someone with C++ chops to implement. What the micropython ecosystem would really benefit from is a scikit-learn type library with a nice predict method, and loading models from binaries. Probably out of scope for discussion on this forum but I am interested to follow this up somewhere

#4

If you’re using the C++ export the inferencing runs on the board itself. The REST interface runs it on our servers.

#5

That is interesting, is there a ‘fair usage’ policy on inferencing on your servers? The intention of my original question was around exposing live inferencing feed from the board itself however

#6

There is a compute time limit per account (see your dashboard), but it’s not enforced at the moment.

exposing live inferencing feed from the board itself however

Ah, clear. Yeah, this is so usecase specific that we don’t have a concrete example for this. E.g. in the sheep activity tracker we send the inferencing result back over LoRaWAN and show it on the big screen (we get the results back from The Things Network over MQTT). An easy way would be to post the data over HTTP to a service you control and show it there, the example firmware for the ST IoT Discovery Kit has the mbed-http library already.

1 Like
#7

OK perhaps a very generic example would be triggering an AWS lambda function, I will checkout the docs

#8

@robmarkcole OK, some pointers to build something like this would be:

  1. Take https://github.com/edgeimpulse/example-standalone-inferencing-mbed
  2. Add the mbed-http and wifi-ism43362 libraries.
  3. Copy mbed_app.json from here to the project (this sets up the WiFi driver).

And then write some code to send the inferencing result over HTTP or HTTPS to trigger a lambda (example on how to do HTTP(S) calls here).

Don’t have a ready-made example, but this should get you underway.

1 Like
#9

Hello,
I’m trying to get my head around wrapping run_classifier in micropython but I do not understand in the code you mentioned where features & features[0] are coming from.
If you could point me to where to look it would be very much appreciated !
Thanks,
Nico

#10

Hi @Keja, so features is an array of floats (of size EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE - declared in model_metadata.h) that is the raw sensor data. Docs on the format of the array is here: https://docs.edgeimpulse.com/docs/running-your-impulse-locally-1#input-to-the-run_classifier-function

#11

Hi, this code seems to be no longer working (due to change in the api results formats?)

  • first, the classification request is GET and not POST, apparently
  • then, the return value of res.text is of the form “somename.2alfqvkh.json” and does not match the ‘coldstorageFilename’ format which is of the form: “somename.2alfqvkh.ingestion-6cf86c74c-2dl6h.json”, so the search with “endswith” fails, unless we replace in with a search with “in”, after removing the “.json” extension.
    With those changes, i could make it work
#12

Hi @fleurda, thanks for the feedback I’ve updated the script. Note that Edge Impulse Linux makes it trivial to run this locally in Python now: https://docs.edgeimpulse.com/docs/edge-impulse-for-linux

1 Like