Merge pull request #1909 from pguyot/w42/pico-embedded-jit

Embedded JIT mode for Pico

Continuation of:
- #1829
- #1838
- #1881
- #1891
- #1893
- #1900
- #1901
- #1903
- #1907

Add the ability to perform just in time compilation directly on Raspberry Pi
Pico. Erlang bytecode is compiled to armv6m on the first run and directly
executed on subsequent runs. In this PR, the native code compilation does not
take advantage of being done on the device, it still generates position
independent code.

If Erlang modules include `Type` chunk, the same optimizations that happen on
the desktop with precompilation could be done on the Pico, however current
release (0.7.5) of [`packbeam`](https://github.com/atomvm/atomvm_packbeam)
prunes the `Type` chunk.

Just in time compilation on the Pico is achieved by:
- Adding a flush call after JIT is performed, which is noop on all platforms
except on armv6m where we flush the literal pool (normally a noop, though, as
the pool should have been flushed earlier).
- Updating avm API to fetch the last section named end.
- Adding a `jit_stream_flash` for RP2 platform and adding a cache native code
mechanism that rely on the avm end section on RP2 to find out where to flash
jit code (after the last end) and detect if avm were updated.

These changes are made under both the "Apache 2.0" and the "GNU Lesser General
Public License 2.1 or later" license terms (dual license).

SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later

Author: bettio

.github/workflows/build-and-test.yaml

@@ -535,6 +535,19 @@ jobs:
         ulimit -c unlimited
         ./tests/test-heap
 
+    - name: "Test: test-jit_stream_flash with valgrind"
+      if: matrix.library-arch == ''
+      working-directory: build
+      run: |
+        ulimit -c unlimited
+        valgrind --error-exitcode=1 ./tests/test-jit_stream_flash
+
+    - name: "Test: test-jit_stream_flash"
+      working-directory: build
+      run: |
+        ulimit -c unlimited
+        ./tests/test-jit_stream_flash
+
     - name: "Test: test-mailbox with valgrind"
       if: matrix.library-arch == ''
       working-directory: build

doc/src/atomvm-internals.md

@@ -137,7 +137,7 @@ Following BEAM, there are two flavors of the emulator: jit and emu, but eventual
 - Native: the VM only runs native code and all code must be precompiled on the desktop using the JIT compiler (which effectively is a AOT or Ahead-of-Time compiler). In this mode, it is not necessary to bundle the jit compiler on the embedded target.
 - Hybrid: the VM can run native code as well as emulated BEAM code and some code is precompiled on the desktop.
 
-JIT is available on some platforms (currently only x86_64 and aarch64) and compiles Erlang bytecode at runtime. Erlang bytecode is never interpreted. EMU is available on all platforms and Erlang bytecode is interpreted.
+JIT is available on some platforms (currently only x86_64, aarch64 and armv6m) and compiles Erlang bytecode at runtime. Erlang bytecode is never interpreted. EMU is available on all platforms and Erlang bytecode is interpreted.
 
 Modules can include precompiled code in a dedicated beam chunk with name 'avmN'. The chunk can contain native code for several architectures, however it may only contain native code for a given version of the native interface. Current version is 1. This native code is executed by the jit-flavor of the emulator as well as the emu flavor if execution of precompiled is enabled.
 
@@ -158,6 +158,27 @@ A backend implementation is required for each architecture. The backend is calle
 
 A stream implementation is responsible for streaming the machine code, especially in the context of low memory. Two implementations currently exist: `jit_stream_binary` that streams assembly code to an Erlang binary, suitable for tests and precompilation on the desktop, and `jit_stream_mmap` that streams assembly code in an `mmap(2)` allocated page, suitable for JIT compilation on Unix.
 
+### Embedded JIT and Native
+
+On embedded devices, Native mode means the code is precompiled on the desktop and executed natively on the device. This currently works on all ARMv6M devices (Pico and STM32).
+
+The default partition scheme on all platforms is optimized for the Emulated VM which is larger than the JIT or Native VM, and for the Emulated atomvmlib (with no native code for estdlib and no jit library) which is smaller than the JIT atomvmlib (that includes native code for estdlib and jit library).
+
+JIT mode means the Erlang bytecode is compiled to native code directly on the device. This actually is possible on Raspberry Pi Pico by using the flash to store the native code. The first time the code is executed, it is compiled and streamed to flash, and for next runs (including at a future boot), the native code is directly executed.
+
+To achive embedded JIT, it is required to flash the device with the JIT compiler for armv6m which is part of the jit library. This library is quite large, so for Pico boards that come with 2MB of flash, it is required to remove jit modules for other backends. It is also required to change the way code is partitioned.
+
+For example, it is possible to have the following offsets defined in `src/platforms/rp2/src/main.c`:
+
+```
+#define LIB_AVM ((void *) 0x10060000)
+#define MAIN_AVM ((void *) 0x101B0000)
+```
+
+To fit in the lib partition, all networking modules should also be removed (the Pico doesn't have any networking capacity).
+
+After the first run, compiled modules in flash are used unless there is a version mismatch or the application avm or the library avm have been updated on the device. AVM packages end with a section called "end" (0x656E64). When the JIT compiler flashes native code, it changes this name to "END" (0x454E44), by effectively clearing 3 bits in the flash, which is possible without erasing any flash block. Any rewrite of these avm packages will overwrite the section names to "end".
+
 ## The Scheduler
 
 In SMP builds, AtomVM runs one scheduler thread per core.  Scheduler threads are actually started on demand.  The number of scheduler threads can be queried with [`erlang:system_info/1`](./apidocs/erlang/estdlib/erlang.md#system_info1) and be modified with [`erlang:system_flag/2`](./apidocs/erlang/estdlib/erlang.md#system_flag2).  All scheduler threads are considered equal and there is no notion of main thread except when shutting down (main thread is shut down last).

libs/jit/src/CMakeLists.txt

@@ -26,6 +26,7 @@ set(ERLANG_MODULES
     jit
     jit_precompile
     jit_stream_binary
+    jit_stream_flash
     jit_stream_mmap
     jit_aarch64
     jit_aarch64_asm