[WIP] Add asynchronous tensor I/O custom module sample

[Copilot]

Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress.

Original prompt

Asynchronous tensor I/O custom module sample

This sample expects that you've already produced a working version of the basic sample (including compiler installation and CMake setup).

This sample demonstrates adding custom modules callable from compiler-produced programs that take and return tensor types. Both the calls into the compiled program and the custom call made from the compiled program are made asynchronously using HAL fences for ordering work. This allows the entire invocation - including the custom user call - to be scheduled without blocking and enables pipelining and overlapping invocations. When embedded into a larger user-level framework this lets IREE invocations be interleaved with other user work.
Instructions

Compile the example module to a .vmfb file:

iree-compile
--iree-execution-model=async-external
--iree-hal-target-device=local
--iree-hal-local-target-device-backends==llvm-cpu
samples/custom_module/async/test/example.mlir
-o=/tmp/example.vmfb

Build the iree_samples_custom_module_async_run CMake target :

cmake -B ../iree-build/ -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo .
-DCMAKE_C_FLAGS=-DIREE_VM_EXECUTION_TRACING_FORCE_ENABLE=1
cmake --build ../iree-build/ --target iree_samples_custom_module_async_run

(here we force runtime execution tracing for demonstration purposes)

See here for general instructions on building using CMake.

Run the example program to call the main function:

../iree-build/samples/custom_module/async/custom-module-async-run
/tmp/example.vmfb example.main

TBD

Expose a way to tie call arguments and results for in-place operations.
Expose a way to specify the lifetime of the I/O to allow for transient memory.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.