Creating job... OK (ID: 2086869)
Scheduling job in cluster...
Creating windows from 13 files...
[2/3] Pre-caching files...
[3/3] Pre-caching files...
Pre-caching files OK
[ 1/13] Creating windows from files...
[13/13] Creating windows from files...
Traceback (most recent call last):
File "/app/input-block-preprocessing/downsample.py", line 86, in <module>
window = sn.resample_poly(window, target_freq, sample_freq, axis=ds_axis)
File "/usr/local/lib/python3.7/dist-packages/scipy/signal/signaltools.py", line 3089, in resample_poly
h = firwin(2 * half_len + 1, f_c, window=window)
File "/usr/local/lib/python3.7/dist-packages/scipy/signal/fir_filter_design.py", line 457, in firwin
m = np.arange(0, numtaps) - alpha
ValueError: Maximum allowed size exceeded
Error windowing Downsample error code was not 0, but was 1
Downsample error code was not 0, but was 1
Application exited with code 1 (Error)
Job failed (see above)
I keep running into the above error. I reduced my sample sizes but I had no luck getting the generate features to work.
Oh! So that means that I want to overlap the windows? Your drawing was quite clear!
I changed the sample period - I also noticed that the sample frequency reported by the data ingestion service is determined by the timestamp difference between the first two samples instead of the average sample rate in the dataset.
I’m still running into this error:
MemoryError: Unable to allocate array with shape (57471264367816104,) and data type int64
The frequency should be constant in all your data samples (but I think you figured it out, just checked your project again and you disabled the samples that were not using the 16.666…Hz frequency .
You may want to have a look at this thread too (maybe you did already): Error windowing Downsample error code was not 0, but was 1
@janjongboom & @louis Thanks for your quick replies! The issue is that my sampling rate has a 1 ~ 2ms jitter when sampling, so I’m not always able to hit the same interval between samples. Is there a way to downsample to the lowest frequency sample?
How would this affect the subsequent inference when I port the code over to my device?