Too much audio input data?

akashkumar · September 3, 2022, 2:41pm

Question/Issue:
Getting the following errors: MaxListenersExceededWarning: Possible EventEmitter memory leak detected.

Project ID:
https://studio.edgeimpulse.com/studio/121737

Context/Use case:
Hi, I was able to create models about a week ago. But now I only get errors either trying to use MFE or MFCC methods. I would like to reduce the window increase to 100ms because higher window increases errors out on ESP32 (memory errors). I am suspecting that this happened after augmenting it with more water and noise samples on the Edge Impulse studio? Please help me figure this out. Thanks.

Scheduling job in cluster...
Job started
Initializing job pod...
(node:18) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 change listeners added to [StatWatcher]. Use emitter.setMaxListeners() to increase limit
    at _addListener (node:events:465:17)
    at StatWatcher.addListener (node:events:487:10)
    at Object.watchFile (node:fs:2319:8)
    at new PgDB (/home/node/studio/build/server/shared/db/pg_db.js:71:26)
    at Object.<anonymous> (/home/node/studio/build/server/server/init-server.js:139:14)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
    at Module.Hook._require.Module.require (/home/node/common/node_modules/dd-trace/packages/dd-trace/src/ritm.js:66:39)
Initializing job pod OK

Scheduling job in cluster...
Job started
Creating windows from files...
[  0/142] Creating windows from files...
[  0/142] Creating windows from files...
[  0/142] Creating windows from files...
[  0/142] Creating windows from files...
[  0/142] Creating windows from files...
[  0/142] Creating windows from files...

<--- Last few GCs --->

[8:0x6514340]   141281 ms: Mark-sweep 4059.3 (4128.2) -> 4039.2 (4123.8) MB, 3633.9 / 0.0 ms  (average mu = 0.119, current mu = 0.004) allocation failure scavenge might not succeed
[8:0x6514340]   144866 ms: Mark-sweep 4061.9 (4130.9) -> 4041.5 (4126.1) MB, 3572.9 / 0.0 ms  (average mu = 0.065, current mu = 0.003) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb00d90 node::Abort() [node]
 2: 0xa1823b node::FatalError(char const*, char const*) [node]
 3: 0xcedbce v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xcedf47 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xea6105  [node]
 6: 0xea6be6  [node]
 7: 0xeb4b1e  [node]
 8: 0xeb5560 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xeb84de v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
10: 0xe7990a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
11: 0x11f2f06 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
12: 0x15e7819  [node]

Attached to job 3875519...
/home/create_features.sh: line 3:     8 Aborted                 (core dumped) node "/app/node/windowing/build/window-time-series.js" "/home/input.json" "/home/output.json"
Application exited with code 134

Job failed (see above)

Creating job... OK (ID: 3875484)

Scheduling job in cluster...
Job started

Created window size should be under 4096 MiB but was 15362 MiB. Set a larger window increase to reduce the number of windows.

Alternatively, the enterprise version of Edge Impulse has no limits, see https://www.edgeimpulse.com/pricing for more information.

Job failed (see above)

shawn_edgeimpulse · September 5, 2022, 2:13pm

Hi @akashkumar,

It looks like you are running into a limitation with the free version of Edge Impulse. The developer version is limited to 4 GB of data memory. You likely do not need all of the data samples in your training set to get a basic prototype working (it might not be 99.999% accurate, but you should be able to get it close enough to 95-99% for running water detection). Here are some possible tips to help out: Lower compute time - Edge Impulse Documentation

If you are interested in an enterprise license, I invite you to contact our sales team here: Pricing

akashkumar · September 5, 2022, 2:38pm

Thank you. We will work on it. Appreciate your response.