Bulk Upload Data (Images) Python Snippet

#1

Anyone have a small snippet of Python that would allow me to upload images in bulk via the Ingestion Service?

ChatGPT says this should work, but, I am not convinced:

Can anyone give me a correct snippet, please and thanks in advance!

import os
import requests

# Set the directory to parse
directory = 'my/data/directory'

# Iterate through the subdirectories in the given directory
for subdir in os.listdir(directory):
    subdir_path = os.path.join(directory, subdir)
    if os.path.isdir(subdir_path):
        label = subdir

        # Iterate through the files in the subdirectory
        for file in os.listdir(subdir_path):
            file_path = os.path.join(subdir_path, file)
            if os.path.isfile(file_path):
            	with open(file, 'r') as file:
            	 res = requests.post(url='https://ingestion.edgeimpulse.com/api/training/data',
            	 data=file,
            	 headers={
            	 'Content-Type': 'image/jpeg',
            	 'x-file-name': file,
            	 'x-label': label,
            	 'x-api-key': 'ei_xxxxxxxxx'
            	})

            if (res.status_code == 200):
             print('Uploaded file to Edge Impulse', res.status_code, res.content)
            else:
             print('Failed to upload file to Edge Impulse', res.status_code, res.content)
#2

@davidtischler_edgeim

See New ingestion API - Edge Impulse API ← does it in bulk already

1 Like
#3

No doubt Jan is correct but one must manually create the files array (based on the examples at the link given):

files = [
    ('data', open('one.png', 'rb')),
    ('data', open('two.png', 'rb')),
]

The original post code in question goes and gets all files pointed to without any manual array manipulation. A use case would be files automatically added to a post production release training set for example when an inference gave a very low prediction rate.

1 Like