Uploading data csv's - An item with this hash already exists

Report bugs
#1

Hi. When I try and upload csv data files via the website I get lots of errors see below;

Have tried a few times now, all the filenames are different.
…

[462/468] Failed to upload 1.461acc.csv: An item with this hash already exists (ids: 176392673, 176392672)
[463/468] Failed to upload 1.462acc.csv: An item with this hash already exists (ids: 176392673, 176392672)
[464/468] Failed to upload 1.463acc.csv: An item with this hash already exists (ids: 176392673, 176392672)
[465/468] Failed to upload 1.464acc.csv: An item with this hash already exists (ids: 176392673, 176392672)
[466/468] Failed to upload 1.465acc.csv: An item with this hash already exists (ids: 176392673, 176392672)
[467/468] Failed to upload 1.466acc.csv: An item with this hash already exists (ids: 176392673, 176392672)
[468/468] Failed to upload 1.467acc.csv: An item with this hash already exists (ids: 176392673, 176392672)

Done. Files uploaded successful: 0. Files that failed to upload: 468.

#2

Hello @pmax,

Can I make sure those items are not already present in your test dataset (under the data acquisition page, on the upper tabs, go to Test Data).

Let me know if that’s the case, if not, I’ll have a deeper look.

Best,

Louis

#3

Hi Louis.

Yes only 5 of the csv files uploaded.

upload
upload1879×848 66.4 KB

Regards
Phil

#4

Cam you share your project ID so I can have a look please?

Best,

Louis

#5

Here it is - 70058

Thanks
Phil

#6

Hello @pmax,

Sorry for my late reply, I was traveling for work and struggled to catch up with the forum.
This behavior is strange considering the files in your project.

Could you try to create a new project and see if it fixes it?
Alternatively, you can also try @Clinton suggestion: Data acquisition Error - #12 by sinameshksar

Best,

Louis

#7

Hi Louis.

Hmm tried that same issue. 13 of the files upload the rest have ‘item with this hash exists’ error.

  • Done. Files uploaded successful: 13. Files that failed to upload: 455.

I tried then uploading 1 on its own which didn’t work either.

Uploading 1 file…

[1/1] Failed to upload 1.467acc.csv: An item with this hash already exists (ids: 180961632, 180961631, 180961630, 180961628, 180961627, 180961626, 180961625, 180961624, 180961623)

Done. Files uploaded successful: 0. Files that failed to upload: 1.

Job completed

#8

Hi @pmax,

Could you send me your dataset so I can try on my side?
Either sharing a link here is your dataset is not confidential or by email (louis@edgeimpulse.com).

Best,

Louis

#9

Thanks, I have emailed it over. Have a good weekend,

Regards
Phil

#10

Hello @pmax,

I can reproduce your issue even with a new project.
I’m investigating… I’ll let you know asap!

Best,

Louis

#11

Hello @pmax,

I managed to upload your data using the CLI Uploader with the flag --allow-duplicates:

In your files directory, you can run the following command:

edge-impulse-uploader --clean *.csv --allow-duplicates

We are still investigating why we have the “An item with the hash already exists” error. I created an internal ticket so our core engineers can investigate further.

Best,

Louis

#12

Also @pmax,

I noticed that many of your files contain the exact same values…