Promote tier 3 riscv32 ESP-IDF targets to tier 2

[SergioGasquez]

Proposal

I would like to propose promoting the RISC-V 32bit ESP-IDF targets,
riscv32imc-esp-espidf, riscv32imac-esp-espidf, & riscv32imafc-esp-espidf to tier 2.

Tier 2 target requirements

A tier 2 target must have value to people other than its maintainers. (It may still be a niche target, but it must not be exclusively useful for an closed group.)

In general, RISC-V is an open specification, used and accessible to anyone, including individuals.

The *-espidf targets are specific for the Espressif line of MCUs and their ESP-IDF operating system (which is largely POSIX compatible).
With that said, the ESP IDF is:
Open source
Having a high popularity (the “de facto” SDK for developing C-based firmware on top of the Espressif MCUs)

…so we believe it is not the case at all that it is used by a closed group.

A tier 2 target must have a designated team of developers (the "target maintainers") available to consult on target-specific build-breaking issues, or if necessary to develop target-specific language or library implementation details. This team must have at least 2 developers.

@MabezDev, @ivmarkov, and I (@SergioGasquez) will maintain the targets.

The target must not place undue burden on Rust developers not specifically concerned with that target. Rust developers are expected to not gratuitously break a tier 2 target, but are not expected to become experts in every tier 2 target, and are not expected to provide target-specific implementations for every tier 2 target.

I don't foresee this being an issue.
The maintainers will ensure we avoid undue burden for the general Rust community.

The target must provide documentation for the Rust community explaining how to build for the target using cross-compilation, and explaining how to run tests for the target. If at all possible, this documentation should show how to run Rust programs and tests for the target using emulation, to allow anyone to do so. If the target cannot be feasibly emulated, the documentation should explain how to obtain and work with physical hardware, cloud systems, or equivalent.

There is a QEMU fork of Espressif that supports ESP32-C3. Also there is an experimental feature for running ESP-IDF applications on Linux, which is available for all the RISC-V targets, Wokwi simulation could also be an option as its supports all the targets.

The target must document its baseline expectations for the features or versions of CPUs, operating systems, libraries, runtime environments, and similar.

We already have some documentation, see *-espidf chapter of the The rustc book

If introducing a new tier 2 or higher target that is identical to an existing Rust target except for the baseline expectations for the features or versions of CPUs, operating systems, libraries, runtime environments, and similar, then the proposed target must document to the satisfaction of the approving teams why the specific difference in baseline expectations provides sufficient value to justify a separate target.

The riscv32XXX-esp-espidf targets are indeed very similar to their bare-metal counterparts (riscv32XXX-unknown-core-elf). Where they do differ (but this is key) is in having the following additional properties set on the -espidf targets:

• Family (“unix”)
• target_os (= “espidf”)
• target_env (= “newlib”)
• target_vendor (= “espressif”)

This allows the STD code to branch where necessary for these targets, as done by all others that do support the Rust Standard Library.

Tier 2 targets must not leave any significant portions of core or the standard library unimplemented or stubbed out, unless they cannot possibly be supported on the target.

• core, alloc, and panic_abortare fully implemented.
• std is fully implemented as well, except for support for processes and signals, which are stubbed out.

We do understand that not implementing processes (and therefore signals) is a bit controversial, but with that said, an MCU target cannot feasibly implement those, as often there is just no notion of “multiple processes” there (as is the case with ESP-IDF).

So we do believe we cannot possibly support processes and signals indeed, and thus we fall in the same category as other Tier 2 targets, like wasm, where neither processes, nor even other core aspects of the Standard Library, like multi-threading, BSD sockets, or even mutexes are implemented.

The code generation backend for the target should not have deficiencies that invalidate Rust safety properties, as evaluated by the Rust compiler team. (This requirement does not apply to arbitrary security enhancements or mitigations provided by code generation backends, only to those properties needed to ensure safe Rust code cannot cause undefined behavior or other unsoundness.) If this requirement does not hold, the target must clearly and prominently document any such limitations as part of the target's entry in the target tier list, and ideally also via a failing test in the testsuite. The Rust compiler team must be satisfied with the balance between these limitations and the difficulty of implementing the necessary features.

RISC-V is a well-established and well-maintained LLVM backend. To the best of my knowledge, the backend won't cause the generated code to have undefined behavior.

If the target supports C code, and the target has an interoperable calling convention for C code, the Rust target must support that C calling convention for the platform via extern "C". The C calling convention does not need to be the default Rust calling convention for the target, however.

The C calling convention is supported.

The target must build reliably in CI, for all components that Rust's CI considers mandatory.

This should not be a problem at all. The only dependency the *-espidf targets need is on libc which is already a listed dependency in the Standard Library and which does support ESP-IDF.

There is no need for any native libraries to be present in the CI, and as such building the Standard Library for ESP-IDF only requires the Rust compiler and nothing else.

The approving teams may additionally require that a subset of tests pass in CI, such as enough to build a functional "hello world" program, ./x.py test --no-run, or equivalent "smoke tests". In particular, this requirement may apply if the target builds host tools, or if the tests in question provide substantial value via early detection of critical problems.

The targets will not build host tools. With that said:

• If there is a need to have a build that ends up with an Espressif MCU ELF binary (as opposed to just building the Rust Standard Library), Espressif does have an infrastructure to do this by downloading all dependencies automatically during the build (including checking out the ESP IDF repository which is built from source code, a bit like the Linux kernel)
• The above dependencies can also be installed in a “vendored” way if necessary

Building the target in CI must not take substantially longer than the current slowest target in CI, and should not substantially raise the maintenance burden of the CI infrastructure. This requirement is subjective, to be evaluated by the infrastructure team, and will take the community importance of the target into account.

To the best of my knowledge, this will not induce a burden on the current CI infra.

• Building the Standard Library itself for ESP-IDF is very fast
• Building an Espressif MCU ELF binary is also fast (in the order of a couple of minutes), yet there is extra time necessary (also in the order of a few minutes) to download the Espressif-specific C toolchains. If this is a concern and if building an ELF binary is actually even necessary, we can consider vendoring the toolchains and the ESP-IDF itself to cut on internet traffic

Tier 2 targets should, if at all possible, support cross-compiling. Tier 2 targets should not require using the target as the host for builds, even if the target supports host tools.

Cross-compilation is supported and documented in the platform support document. In fact, cross-compilation is the only way to build for these targets.

In addition to the legal requirements for all targets (specified in the tier 3 requirements), because a tier 2 target typically involves the Rust project building and supplying various compiled binaries, incorporating the target and redistributing any resulting compiled binaries (e.g. built libraries, host tools if any) must not impose any onerous license requirements on any members of the Rust project, including infrastructure team members and those operating CI systems. This is a subjective requirement, to be evaluated by the approving teams.

There are no additional license issues to worry about.

Tier 2 targets must not impose burden on the authors of pull requests, or other developers in the community, to ensure that tests pass for the target. In particular, do not post comments (automated or manual) on a PR that derail or suggest a block on the PR based on tests failing for the target. Do not send automated messages or notifications (via any medium, including via @) to a PR author or others involved with a PR regarding the PR breaking tests on a tier 2 target, unless they have opted into such messages.

Agree.

The target maintainers should regularly run the testsuite for the target, and should fix any test failures in a reasonably timely fashion.

Agree.

All requirements for tier 3 apply.

They apply, they currently are tier 3 targets.

Tier 2 with host tools

Given that the target runs on MCUs, it will not have any host tools.

Mentors or Reviewers

We asked @davidtwco and @Amanieu, but no response from their side yet. Anyone is welcome, but they do have some context already.

Process

The main points of the Major Change Process are as follows:

• File an issue describing the proposal.
• A compiler team member or contributor who is knowledgeable in the area can second by writing @rustbot second.
  • Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a -C flag, then full team check-off is required.
  • Compiler team members can initiate a check-off via @rfcbot fcp merge on either the MCP or the PR.
• Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.

You can read more about Major Change Proposals on forge.

Note

Concerns (0 active)

• ask libs consensus on limited std support resolved in this comment
• fd duplication resolved in this comment

Managed by @rustbot—see help for details.

[rustbot]

Important

This issue is not meant to be used for technical discussion. There is a Zulip stream for that.
Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

Concerns or objections can formally be registered here by adding a comment.

@rfcbot concern reason-for-concern
<description of the concern>

Concerns can be lifted with:

@rfcbot resolve reason-for-concern

See documentation at https://forge.rust-lang.org

cc @rust-lang/compiler

[jieyouxu]

@rustbot concern ask libs consensus on limited std support

We should ask lib team consensus on promoting RISC-V 32bit ESP-IDF targets,
riscv32imc-esp-espidf, riscv32imac-esp-espidf, and riscv32imafc-esp-espidf to tier 2 regarding limited std support re.:

• core, alloc, and panic_abort are fully implemented.
• std is fully implemented as well, except for support for processes and signals, which are stubbed out.

We do understand that not implementing processes (and therefore signals) is a bit controversial, but with that said, an MCU target cannot feasibly implement those, as often there is just no notion of “multiple processes” there (as is the case with ESP-IDF).

So we do believe we cannot possibly support processes and signals indeed, and thus we fall in the same category as other Tier 2 targets, like wasm, where neither processes, nor even other core aspects of the Standard Library, like multi-threading, BSD sockets, or even mutexes are implemented.

EDIT: library team nomination at rust-lang/rust#140042.

[riking]

@rustbot concern fd duplication

The idea that an OwnedFd can be duplicated is fundamental in several Unix-like file handling algorithms and Rust in general (BorrowedFd as ToOwned). It's the exact kind of stdlib blind spot we're trying to prevent.

Upstream issue: espressif/esp-idf#7412

[ivmarkov]

@rfcbot concern fd-duplication

The idea that an OwnedFd can be duplicated is fundamental in several Unix-like file handling algorithms

Would you elaborate on those fundamental Unix-like file handling algorithms?

and Rust in general (BorrowedFd as ToOwned). It's the exact kind of stdlib blind spot we're trying to prevent.

If you mean BorrowedFd::try_clone_to_owned, this API is:
(a) Fallible
(b) The documentation is pretty clear that - in its own words - it is not supported on all platforms, i.e. "To obtain an OwnedFd, you can use BorrowedFd::try_clone_to_owned instead, but this is not supported on all platforms.".

Am I missing something?

My point of view is that (a) cloning file handles with dup / dup2 might also fail for other reasons anyway, even if these syscalls are supported (as in, you are running out of file descriptors). So higher-level APIs should guard for these failures anyway; (b) originally I thought not supporting duplication of file handles is going to be a big deal, but I've yet to find serious use cases, where this is a show stopper. Hence the request above to provide a few concrete examples.

[jieyouxu]

@rustbot resolve ask libs consensus on limited std support

Library team has some more questions (see MCP thread itself), but resolving the initial question part.

[rustbot]

Only organization members can add or resolve concerns.

[Urgau]

@rustbot concern fd duplication

From #864 (comment)

The idea that an OwnedFd can be duplicated is fundamental in several Unix-like file handling algorithms and Rust in general (BorrowedFd as ToOwned). It's the exact kind of stdlib blind spot we're trying to prevent.

Upstream issue: espressif/esp-idf#7412