Feature generation crashes when number of windows > 2000

I am assuming there is a limit of 2000 windows max that has been set in the feature generator but if that is the case indeed, it would be nice to have a better error message than the current one:

[1500/2066] Creating features...
[1600/2066] Creating features...
[1700/2066] Creating features...
[1800/2066] Creating features...
[1900/2066] Creating features...
[2000/2066] Creating features...
Traceback (most recent call last):
File "/home/create_features.py", line 50, in <module>
    sample_file[sample_file_total] = f['features']
File "/usr/local/lib/python3.7/site-packages/numpy/core/memmap.py", line 331, in __getitem__
    res = super(memmap, self).__getitem__(index)
IndexError: index 2000 is out of bounds for axis 0 with size 2000

ERR: {"active":1,"failed":1,"startTime":"2020-02-04T06:17:40.000Z"}

Job failed (see above)

Hi @kartben2, it’s not actually - we have projects with over 400K windows(!). But we slice 2,000 of these windows into a separate file for the feature explorer, and we are writing out of bounds of this array. Quickfix would be to change the windowing parameters a bit so you have more windows (I think there’s a race-condition where we miscalculate the step between the windows for the feature explorer), and we’ll deploy a fix as part of today’s release.

@kartben2 This fix has now been deployed to production, please let me know if the problem persists.