Need help with integrating python

Question/Issue:
How do I run inferencing with my object detection model in python with jupyter lab and get the coordinates and size of the predicted labels?

Project ID:

Context/Use case:
This is my code so far, so basically a input image is resized and run through the model. I am not sure how to use it to achieve the inferencing and prediction like the webassembly server provided by Edge Impulse where i just insert the raw features and i get the output.

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
import json

def hex_to_rgb(hex_val):
“”"
Converts a hexadecimal color value to an RGB tuple.

Parameters:
hex_val (str): Hexadecimal color value in the format '0xRRGGBB'.

Returns:
tuple: A tuple containing the RGB values.
"""
hex_val = int(hex_val, 16)  # Convert hex string to integer
r = (hex_val >> 16) & 0xFF
g = (hex_val >> 8) & 0xFF
b = hex_val & 0xFF
return r, g, b

def reconstruct_rgb_image(hex_features, image_shape):
“”"
Reconstructs an RGB image from its raw hexadecimal features.

Parameters:
hex_features (list): List of raw hexadecimal image features.
image_shape (tuple): Shape of the image (height, width, channels).

Returns:
numpy.ndarray: 3D reconstructed image.
"""
# Convert hex features to RGB values
rgb_features = [hex_to_rgb(hex_val) for hex_val in hex_features]

# Reshape the list of RGB tuples to the specified image shape
image = np.array(rgb_features, dtype=np.uint8).reshape(image_shape)
return image

Example usage

Assuming the image is 96x96 pixels

hex_features = [
“0xa2beae”,“0xa5c3b2”,“0x9ebfad”,“0xa1c1af”,…]
image_shape = (96, 96, 3) # Shape of the image (height, width, channels)

Reconstruct the image

reconstructed_image = reconstruct_rgb_image(hex_features, image_shape)

Display the image

plt.imshow(reconstructed_image)
plt.title(‘Reconstructed RGB Image’)

Prepare the input image for the model

input_image = reconstructed_image.astype(np.float32)
input_data = np.expand_dims(input_image, axis=0) # Add batch dimension

Load the TFLite model and allocate tensors

interpreter = tf.lite.Interpreter(model_path=‘ei-test2-v1-object-detection-tensorflow-lite-float32-model.lite’)
interpreter.allocate_tensors()

Get input and output tensors

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

Set the tensor to point to the input data to be inferred

interpreter.set_tensor(input_details[0][‘index’], input_data)

Run the inference

interpreter.invoke()

Retrieve the output data

output_data = interpreter.get_tensor(output_details[0][‘index’])

Process the output data as per your model’s output format

output = {
“anomaly”: 0,
“results”: []
}

Mocked example of processing output_data into the desired format

detection_results = [
{“label”: “Foot”, “value”: 0.99609375, “x”: 24, “y”: 0, “width”: 16, “height”: 8},
{“label”: “Foot”, “value”: 0.99609375, “x”: 64, “y”: 0, “width”: 16, “height”: 8},
{“label”: “Hip”, “value”: 0.84375, “x”: 40, “y”: 40, “width”: 16, “height”: 8},
{“label”: “Shoulder”, “value”: 0.99609375, “x”: 32, “y”: 64, “width”: 16, “height”: 8},
{“label”: “Shoulder”, “value”: 0.98828125, “x”: 56, “y”: 64, “width”: 16, “height”: 8},
{“label”: “Head”, “value”: 0.97265625, “x”: 40, “y”: 80, “width”: 16, “height”: 8},
]

output[“results”] = detection_results

Draw rectangles or annotations on the image for each detection

plt.figure()
plt.imshow(reconstructed_image)
plt.title(‘Reconstructed RGB Image with Predictions’)

ax = plt.gca()
for result in detection_results:
label = result[‘label’]
x, y, w, h = result[‘x’], result[‘y’], result[‘width’], result[‘height’]
rect = plt.Rectangle((x, y), w, h, fill=False, edgecolor=‘red’, linewidth=2)
ax.add_patch(rect)
ax.text(x, y, label, fontsize=12, bbox=dict(facecolor=‘white’, alpha=0.7))

plt.axis(‘off’)
plt.show()

Print the output

print(json.dumps(output, indent=4))

Hello!

You can have a look at this example which output the bounding boxes in the image:

Best,

Louis

Hello! Thank you for the advice but can I ask where I can get the .eim model from edge impulse? and does this code work on jupyter lab?