So, I have these two files, one is a wave file that is correctly formatted, and another one is the same file with the header stripped out.
I’m attempting to upload this data to the ingestion service for my project. I’ve also read on the docs that by appending “Ref-Binary-i16” (or another type depending) to the values field, I can send the file as a binary payload. But, every time I POST this info to the ingestion endpoint, it returns an error stating :
Non-compliant values: Ref-BINARY-i16
I went ahead and copied the example byte array from the docs into CBOR playground and my message is formatted exactly the same.
Has anyone come across this before? Or know of a way to fix this issue?
For anyone attempting to assist, I’m using C++ for everything.
I’ve answered in your other thread, but if that fails could you write your buffer to a file and email it to jan@edgeimpulse.com? Easier to look at that way 
FYI here’s a valid WAV file (base64 encoded) with binary payload:
o2lwcm90ZWN0ZWSiY3ZlcmJ2MWNhbGdlSFMyNTZpc2lnbmF0dXJleEAxMmM1NGU4ZjUxMGQ5OWZjYTY5NmVjYmE3YjYyMWI2NTgyMDdkMjMwMTAyZDgzNmExNGIyOTY0MDZkMzQ2Mzk3Z3BheWxvYWSla2RldmljZV9uYW1lcUM0OjdGOjUxOjhEOjEzOkFFa2RldmljZV90eXBlc0RJU0NPX0w0NzVWR19JT1QwMUFraW50ZXJ2YWxfbXP5LABnc2Vuc29yc4GiZG5hbWVlYXVkaW9ldW5pdHNjd2F2ZnZhbHVlc59uUmVmLUJJTkFSWS1pMTb/lR3zHb4dOh2XHBQctRtfGwEb5Br0Gg8bERvrGscahBoxGuMZ1BlRGjcb+Rt1HMwcAB30HKMcRhwsHEcchBzVHCMdQx1bHS4dxxwRHE4b3hqZGnAaNBolGk8aihp9GnEaYhpxGqsaGxuqG3wcCh1oHZsduh2HHS0dvRxlHDocJBxOHIUcbBziGyIbVxq1GUcZJRlXGeMZixpGG/AbTBx6HHwcgByOHNIcNR2DHdkdAx7vHYod7xxfHPYbzBuvG8Eb9BsrHCwcMBwXHL4bjhucG8UbDRxWHKYctxzBHKkcbxwzHBwcCxwYHAYc/xsDHAwcDhwEHOsb4xvnGwIcLBw9HEQcQxw2HEccfBysHK4crhzcHA4dLh0qHQEd6hzLHLMcxBwMHUQdSB00HUQdRh0DHb0ckxyAHJEcrxx3HJobTxp7GWwZ9BlOGm4aUhprGsMaSBuxG78btxsFHHgc7RxdHdgd9R3LHVgdxRxCHNUboRt8G0sbFBviGqAaRRr0Gb4ZkBlaGZIZIRr5GrobKByAHK4cvxyhHHgcVBxDHGscqBzbHAUd4hx3HMwbFxuiGkIa/RnlGd0Z0BnYGQMaExrxGb4ZsBnUGVIaGRv4G4cc/Rw5HTEd8RyOHDQc8xvVG+IbFRweHPkbiRu7GpAZfhjgF+kXURjvGLYZkRpjG/AbNBw4HDUcLxxoHAIdvB07HnIePB6aHcgc+xtKG9MathrwGl0bvRvYG70bhhs+G+4a4hoQG5AbUhzgHE0dXB0lHekcgxwVHO0b/xv/G+wb5xvmG+sbyBt/G18bQxtsG60b0BvvG90b2hvdG+Ib9RvwG+Mb9RswHJIcyBzbHM4cvBywHKccxxzqHO0c/RxAHUIdFB3aHK8clByPHJ4crRy3HHQcqBtiGjwZwxj2GHQZuhmiGd4ZYxrqGjsbRBs+G2gbzRt+HDYdwh0kHgwegB3mHHkcKBzPG24bMhscG/4asBpXGuwZoBlPGRMZExmGGVYaIBu2GwEcRBxWHCsc0hubG7kbGRyAHOscRx15HUod0RwbHHMb+hqnGmoaMxoSGhkaJRoiGg4a0xmsGaIZ/xmgGk8b+Rt7HJscoxyXHGscKBz3GwAcChwvHGMceRxFHL8bFhuCGgYaihlCGVEZjhk3GuwaghveGxQcRhx4HJwc5BxTHbkd8h0IHtodWB2aHP8bqhtqG0cbSxtZG2MbVxtoG2cbaRt6G6gb2RsLHFQcnBzfHP8cCh31HKYcRxz5G8cblBtTG1obkRuwG7QbzBvwGxEcLRxQHE0cFxw5HGwcrxzcHOkc7BzMHLocrByMHIkclRyHHIMcjxy8HNUctxyGHIEcrRzKHKwcgBxrHH8cqxztHBsdFB3fHFkcShsLGk0ZLhl1GegZEBpFGqwaHBtqG2sbHhskG5MbMxzXHGEdvR3bHaodUB3mHGoc1xt8GzEb+xrsGsUajhoxGuIZrhl4GWYZtRlbGigb2RtaHJccxBzWHLkcpRymHMgcBh1LHXkdox2EHUAduBwlHJkbRBvrGpoacRpYGjAaNxpLGjsaEhoVGmUa0hpWG9MbSRygHOsc9hzhHLgcnRyFHHAcfByJHHocUhzeGzsbeBr5GbEZlxl4GX0Z2RltGvAacxvKG/AbGRxYHJ0c6BxfHbEdyx23HaUdVR3nHJAcURw+HDscIBwFHBocNBxNHE4cHhz3GxYcWxyVHOQcHB1CHUsdXx1EHQgdrRx4HHcccBxdHFscdxx5HI0crhzbHPIcAh3sHNwc1BzMHP8cHR0IHeYczxy2HIAcgRyZHJAcoByuHK8c0Bz5HP4c4BzZHPQcFB1HHYMdxR3qHccdxh3KHdEdyh3VHcMdWh1jHA4bBRqPGa8ZDBoUGvkZLhqvGi8bWBs0Gy0bjBsXHM4ciB0lHm0eTx7bHW0dGh2FHNgbcBs/Gy8bKhsaG+EaphplGgAapBmjGQIaiBouG60b+hs4HGscVRwbHPsbPRycHAQdah2fHcwdrh1EHdMcRByzGzob6BqvGqQazhr2GgUb2xqfGn0aeBqnGgcbiRsOHLAcLh13HXcdRh3kHJAcZRyHHOccNB1JHSMdxRxQHKob9xpUGqMZRhlbGdsZjxorG4QbkhuiG8Yb3BsrHIsc7hxdHcEdDx4nHv4dhx3hHHwcQxwiHCgcMRw/HC0c/hvOG5obdhuHG5AbihulG9wbDBwnHDwcJhz8G78boxubG7Eb3xs9HIsc0BzgHOEc3hzxHOccxxyiHJUcfxxpHFYcNxwWHP0b3xvFG8kb0BvbGxMcOxxLHGsclhy0HLoc3hwcHV8deh2VHaod1h3YHbwdlB2fHZ0diB1OHZ4cVRsEGicZ5BgxGV8ZihmZGf0ZjBrdGtka5BomG50bRRzzHHIduB2eHVkd9Bx9HA8cqRtqG0EbOxtbG3cbXhsbG7oaSxrLGZcZnRntGWEa+hpIG2MbeBtvG0YbGxtAG4Eb8Bt8HOYc9RzRHKYcWxwUHMIbchs/GyEb9BrHGtkayBqCGkwaQRoxGlIarRooG3oboRvyGzIcQhxJHFEcVhxSHGgcpRzfHOsc8xzCHG4cIRzJGz0bfRrfGZUZmhm9Ge4ZJRpMGpcaBRt7G8Eb8Rs9HIEcuhwGHWMdgh1ZHRAd3ByIHCoc6RvUG9MbzhvOG9gbxhuqG3gbQBsgGxIbYBuqG9QbBBw8HE8caxxZHC8cChwAHCQccRy5HOwcGB03HUkdJx0JHdocpRxwHGIcXBxiHFQcMRzsG4sbWBtRG3MbqBvnGxwcbxzhHCsdWR1zHW8dcx1yHY0dxh32HR8eMh4pHvgd1h24HZwdVx3+HFgcZRtqGq4ZYRltGYgZmRnSGUgaxhovG2wbmxurG70b7htSHMwcGh0tHSEd/xy0HDgcrxtzG3YbgxuDG3EbXxs8GwIbsRpnGkAaUhqKGroa/xpfG7cbxhuvG5Ubght/G4cbtBsQHHwcyRwlHUgdFB3SHKIccBwtHOUbzhvCG74bxButG3QbKBvWGr0aAhtQG5gbyRv0Gx8cOhw1HCEcEBwIHBYcORxtHIkclBxwHCQc1BuVG1EbCRvlGtoayBqRGnoaexqeGsMa7RoVG0sbgRvEGyAcYxyTHLEcvByTHGccahx+HHwcghyhHMsc3By/HHccMhz5G9Ib0RvcG7obuxvTGwQc/hsBHO8bvxuiG50bnButG8obFRw2HEMcWxyKHLkcuhyhHHccYxx2HH8chBx4HGscRhwFHL8brRumG58bnhu4G9ob+BstHD8cVRxZHHwcxBwEHSIdNB1fHXodhh2SHaEdhh10HXgdfx2THXsdYh1dHU8dNx36HJQcCRx7Gwsb2Bq6GrAaxxrsGvQa/hrrGuoa7BoZG20b1BsiHFwcgRyhHI4cVRzRG1obKhs6G1UbaBuGG5EbfRtBG/EalxoyGvEZ7RklGnQa3RosG10bhRuYG4obiBujG/AbQRy2HAcdLB05HUcdMB3/HMwctxy8HM0c2hzXHLAcfBwuHMobchsnGyMbRhtzG6Yb+hswHEscKxwQHPwb7RsNHDMcaBy6HPIcCx34HLAcPxzlG6EbexuUG7kb+xvmG8cboRtoG0cbQRtNG3IboxvsGzccVRxhHI8cyRzsHAYdER0uHUkdSh1FHTEd8xy4HKocmRx/HGkcRhwTHNwb1RvsG/8bChwnHCkcIhwcHBYcDBwYHCwcMhwkHCgcNBxIHFAcXBxtHHscgxyFHJkcsRyqHJ0ciBxwHG4cURwsHBIc+BvwG9wb1BvwGykcSxxeHIQclhymHKccqxy9HMgc5BwEHR0dKx0WHesc0hy2HK8cjhx5HHwcXRwyHBQc9xvUG6kbiRtbG1IbXBtOG04bVhuEG8sb4RvRG7IbrhutG7kb8hsZHAsc5xuxG5YbeRtRGw8b4hrhGuwaCBstGyYbDBsHGwUbCBv5Gt0aDRsxG2sbphvIG+kb+Bv8G9sb7xsTHCUcLhxMHFYcUhwwHPEbnRtaG0QbQRtuG5MbqxvPG+8bBBwGHPUb6hvxGw0cKxxfHHccdRxTHDEcEBzYG6wbkxuMG6IbshuqG64btBu4G58bmxu5G8AbzBvxG/Ub6RvSG8Ub4hsFHBIcDRwSHBgcCBz2G/0b8hvnG9obwxukG6AbuRvVG70bphudG4QbZRtkG38bghtzG2sbaBtcG1YbShtBGz0bJxsJGxYbJBs9G2AbjBvIG/8bGRwdHCYcJBwUHAgcDhwXHCEcMBwYHAQc7xu/G68bwhvsG/0bBRw=
@janjongboom Thanks for the response! Okay, well from the docs:
Then, an example was provided where:
a36970726f746563746564a26376657262763163616c67654853323536697369676e6174757265784038346432316234343665323831383561356336303761396437656134636336333735356137363261666164643036343334613136353766623433386237656630677061796c6f6164a56b6465766963655f6e616d657143343a37463a35313a38443a31383a34416b6465766963655f7479706573444953434f5f4c34373556475f494f543031416b696e74657276616c5f6d73f92c006773656e736f727381a2646e616d6565617564696f65756e697473637761766676616c7565739f6e5265662d42494e4152592d693136fff918fa18e718de18e218dd18e118e218f218f018f918f418e918ee18f618f918fc180919101915191119fe18f6180b190e190a19021912191419fc18f218f718
Gives us:
Which is where this confusion came from. But either way, I’m actually getting a new error message this time around, saying that it could not decode the message (Failed to parse)! Oof!
I just emailed you the entire cbor file that I send, with the appended binary payload. This file is structured such that it attempts what you mentioned in the other thread, set:
“values” : “Ref-BINARY-i16” instead of a string within an array!
Ugh, sorry, you’re completely correct. It should be an array. However, afterwards I can upload data as long as you set the content type to application/octet-stream. Here’s an example where I patch the string out for the array in your file:
const crypto = require('crypto');
const fs = require('fs');
let buffer = fs.readFileSync('/Users/janjongboom/Downloads/encoded.cbor');
const request = require('request');
// patch buffer from string => array
let valuesSearchFor = Buffer.from('6E5265662D42494E4152592D693136', 'hex');
let valuesSectionIx = buffer.indexOf(valuesSearchFor);
console.log('index', valuesSectionIx);
buffer = Buffer.concat([
buffer.slice(0, valuesSectionIx),
Buffer.from('9F6E5265662D42494E4152592D693136FF', 'hex'),
buffer.slice(valuesSectionIx + valuesSearchFor.length)
]);
// fs.writeFileSync('/Users/janjongboom/Downloads/encoded.fixed.cbor', buffer);
// now upload the buffer to Edge Impulse
request.post('https://ingestion.edgeimpulse.com/api/training/data', {
headers: {
'x-api-key': 'ei_...',
'x-file-name': 'test01',
'x-label': 'test',
'Content-Type': 'application/octet-stream'
},
body: buffer,
encoding: 'binary'
}, function (err, response, body) {
if (err) return console.error('Request failed', err);
console.log('Uploaded file to Edge Impulse', response.statusCode, body);
});
One thing that might go wrong here is that we expect the values array to be an indefinite length array (so we can abuse the 0xFF termination of the CBOR header). That might have gone amiss in your code. I’ll try and document this better (I’m pretty sure you’re the first external person to implement this part of the protocol). (edit: I see that that part was already in the docs, but I’ve added the Content-Type section).
1 Like
Hey, don’t stress it! I got it to work the other day by appending the values in the array instead, as a workaround. What went wrong when I tried this the first time was that interestingly enough, I’m encoding my audio from Android as:
AudioRecord.PCM_ENCODING_16BIT
So when parsing the data out of the generated file, I was using int16_t. But, the audio was slowed, so I tested with int32_t and it worked fine. I’m assuming this is just something I’m ignorant about, not really sure. But, that’s why I’m here, I was hoping for a user error on my end that we could flesh out, but I was curious because I couldn’t find anything related on the forums so I had a feeling.
Btw, I love this stuff it’s fantastic and @janjongboom you are a beast sir. You are all over the forums, helping, solving and fixing. You’ve been super on point with responses for me and pretty patient with my ignorance with this stuff, but I’m learning!
I’ll give this a shot soon.
No problem! If you’re on Android and have enough processing power that’s a safe choice anyway!
