Inference Model when Playing Scene in Isaac Sim & Import model from local drive into Omniverse

Feature requests
#1

Feature Description:
Feature 1
Inference Edge Impulse Object Detection model live when playing the scene in Isaac Sim

Feature 2
Import model stored locally into Isaac Sim rather than via calling API key over internet

Context/Use case:

  1. As model inference is computationally affordable, it would be quite useful to test the model when playing the scene in Isaac Sim and have viewport images pipe into the model continuously with bouding box appearing either in the same viewport or a new one.

At the moment, I can only inference the model by clicking Classify button.

  1. If user doesn’t want to transfer model to Omniverse over internet, it would be handy to be able to:
    a) Build WebAssembly pack and store locally (already do-able now)
    b) Import model (on local drive ) in Omniverse Extension window and start inferencing

Thank you for viewing the post.

#2

Great suggestion @aaronzzz! let me note that to the tech team.

Do you have any more detail of your project or git gists / repos to share with us? You must have some interesting observations for the rest of us starting out if you are getting so proficient to notice such features.

@jenny fyi!

Best

Eoin