Wasm standalone inference targeting wasmtime runtime

adiibi · March 17, 2025, 4:39am

Hi

I’m trying to integrate edgeimpulse webassembly models into a wasmtime runtime ( elixir wasmex ).

so far I only get:

wasmtime run ~/Downloads/tutorial_-continuous-motion-recognition-wasm-v59/browser/edge-impulse-standalone.wasm
Error: failed to run main module /Users/adrianibanez/Downloads/tutorial_-continuous-motion-recognition-wasm-v59/browser/edge-impulse-standalone.wasm

Caused by:
0: failed to instantiate “/Users/adrianibanez/Downloads/tutorial_-continuous-motion-recognition-wasm-v59/browser/edge-impulse-standalone.wasm”
1: unknown import: env::abort has not been defined

I found this repo but I’m not really sure how to use it.

Could I get a hint in the right direction?
Has anyone used wasmtime to run inference? Or other runtimes on ios / android?

adiibi · March 17, 2025, 7:01am

In the meantime I made some progress. But the wasm interface looks quite lowlevel Is the output of all webassembly deployments structured in the very same way? Eg. could I do one integration loop and always rely on that interface? Or is wasm + javascript generated dynamically according to the edge impulse project setup?

Any way there could be a higher level abstraction on the wasm file with less emphasis on the javascript layer? I have no plans / desire of enabling javascript in the wasmtime/wasmex/elixir runtime. The target platform are mobiles and smaller “embedded” ( nerves project ) devices.

adiibi · March 17, 2025, 10:09am

Could compiling the C++ source code to wasm be a more feasible approach to achieve compatibility with wasmtime? Resp does that make sense at all. Haven’t thought too much about how to deploy edge impulse models onto mobile targets ios / android. But assumed that wasm could be a good strategy with good multiplatform characteristics.

I saw some information about GPU accelerated android edge impulse inference. does that apply to ios too? Will have a look at the available mobile app.

Any recommendations / expertise would be very helpful

adiibi · March 17, 2025, 4:09pm

Have you considered providing a rust version of the model deployments? The ecosystem regarding wasm support seems quite good as far as I understand.
And I’m specifically asking because I already use rust resp. rustler to interface with elixir and provide multiplatform support for things like BLE sensors.
Having rust code would be quite interesting in all those applications. eg. as wasm and integrated in elixir or as native library on different platforms

adiibi · March 17, 2025, 6:32pm

Having looked at the options of compiling to wasm target with C++ I would like to suggest a feature of building a wasi wasm deployment deliverable based on GitHub - WebAssembly/wasi-sdk: WASI-enabled WebAssembly C/C++ toolchain They provide docker builder images.

Would be nice to also have a function exposing the model_parameters so that downstream wasm runtimes like wasmex resp endclients have an entrypoint for selecting / implementing encoders without much friction.

what is your opinion on GitHub - WebAssembly/wasi-nn: Neural Network proposal for WASI? any plans to support that at some point?

adiibi · March 18, 2025, 12:14am

Wasm Artifact: https://github.com/adiibanez/wasi_edge_impulse/actions/runs/13912366627/artifacts/2768927220

There still are a number of issues but I can instantiate the wasm in wasmex and at least lookup exports / imports. The run_ functions and also get_model_parameters_json are exported but there is a also a lot of mangled functions. Would be nice to have a little less noise in terms of exports; not sure if that is possible with wasm.

gist.github.com

https://gist.github.com/adiibanez/45f4efec7bb4ed4229ebd531469dcced

gistfile1.txt

%{module: "wasi_snapshot_preview1", name: "sched_yield", type: "import", details: {:fn, [], [:i32]}}
%{module: "wasi_snapshot_preview1", name: "path_create_directory", type: "import", details: {:fn, [:i32, :i32, :i32], [:i32]}}
%{module: "wasi_snapshot_preview1", name: "path_unlink_file", type: "import", details: {:fn, [:i32, :i32, :i32], [:i32]}}
%{module: "wasi_snapshot_preview1", name: "fd_write", type: "import", details: {:fn, [:i32, :i32, :i32, :i32], [:i32]}}
%{module: "wasi_snapshot_preview1", name: "fd_allocate", type: "import", details: {:fn, [:i32, :i64, :i64], [:i32]}}
%{module: "wasi_snapshot_preview1", name: "fd_sync", type: "import", details: {:fn, [:i32], [:i32]}}
%{module: "wasi_snapshot_preview1", name: "fd_prestat_get", type: "import", details: {:fn, [:i32, :i32], [:i32]}}
%{module: "wasi_snapshot_preview1", name: "fd_fdstat_set_rights", type: "import", details: {:fn, [:i32, :i64, :i64], [:i32]}}
%{module: "wasi_snapshot_preview1", name: "path_open", type: "import", details: {:fn, [:i32, :i32, :i32, :i32, :i32, :i64, :i64, :i32, :i32], [:i32]}}
%{module: "wasi_snapshot_preview1", name: "args_get", type: "import", details: {:fn, [:i32, :i32], [:i32]}}

This file has been truncated. show original

When I try to call get_model_parameters_json I get following exception:

(MatchError) no match of right hand side value: {:error, “unknown import: wasi_snapshot_preview1::args_get has not been defined”}

Maybe someone a bit less “rust-y” with Make / C++ could have a look? I’m also not too familiar with the wasi api. Took me quite a few LLM queries and head scratchers to implement the proof of concept.

As far as I understand the file should run on any of the wasm runtimes. wasmtime, wasmedge, … I tried only wasmtime via wasmex elixir binding.

adiibi · March 20, 2025, 11:26am

Any chance I could get some feedback from the team regarding wasm wasi, wasi-nn, wasi components, …

Are there any plans or at least the interest to support those APIs in any form at some point in the future?