Description
The compiler team would like to ask the library team for consensus regarding whether or not library team is willing to accept a promotion of RISC-V 32bit ESP-IDF targets, riscv32imc-esp-espidf
, riscv32imac-esp-espidf
, and riscv32imafc-esp-espidf
to tier 2 with limited std support:
core
,alloc
, andpanic_abort
are fully implemented.std
is fully implemented as well, except for support for processes and signals, which are stubbed out.We do understand that not implementing processes (and therefore signals) is a bit controversial, but with that said, an MCU target cannot feasibly implement those, as often there is just no notion of “multiple processes” there (as is the case with ESP-IDF).
So we do believe we cannot possibly support processes and signals indeed, and thus we fall in the same category as other Tier 2 targets, like wasm, where neither processes, nor even other core aspects of the Standard Library, like multi-threading, BSD sockets, or even mutexes are implemented.
Prior related library team decisions
- One example e.g.
armv7a-vex-v5
in Addarmv7a-vex-v5
tier three target #131530 where that target
armv7a-vex-v5
implements as much of the standard library as is possible using only public VEX SDK functions.
This includes:
std::time
, not includingSystemTime
as the SDK does not provide absolute time information.std::io
, includingstd::io::stdin()
andstd::io::stdout
, but notstd::stderr()
, as stderr does not exist on this platform.std::fs
, with the exception of directory reading and file deletion, due to public SDK limitations.- modules which do not need to interact with the OS beyond allocation,
such asstd::collections
,std::hash
,std::future
,std::sync
, etc.Notable modules which are not implemented include:
std::process
std::thread
std::net
There, the library team consensus was
We discussed this in the libs meeting today. While this is indeed missing a lot of what would normally be present in a
std
target, it does at least have the basics down like support forInstant
(which is one of the biggest issues with wasm32-unknown-unknown). We're happy for this to be accepted at tier 3 but would likely reject it for higher tiers.
- Another related consensus for the
riscv32im-risc0-zkvm-elf
target: Status of the riscv32im-risc0-zkvm-elf target #135376 (comment)
From the target tier policy document:
Tier 3:
std for targets with an operating system or equivalent layer of system-provided functionality), but may leave some code unimplemented (either unavailable or stubbed out as appropriate), whether because the target makes it impossible to implement or challenging to implement.
Tier 2:
Tier 2 targets must not leave any significant portions of core or the standard library unimplemented or stubbed out, unless they cannot possibly be supported on the target.
The libs team is happy to have as a tier 3 target but would block it from any higher tier.
More context
- MCP: Promote tier 3 riscv32 ESP-IDF targets to tier 2 compiler-team#864
- MCP thread: #t-compiler/major changes > Promote tier 3 riscv32 ESP-IDF targets to… compiler-team#864
@rustbot label: I-libs-nominated T-libs
Activity
[-]Library team consensus on promoting tier 3 riscv32 ESP-IDF targets to tier 2 with std that has processes and signals supported stubbed out[/-][+]Library team consensus on promoting tier 3 riscv32 ESP-IDF targets to tier 2 with std that has processes and signals support stubbed out[/+]joshtriplett commentedon Apr 23, 2025
We talked about this at length in today's @rust-lang/libs-api meeting.
Among the things that came up: we'd love to get some better documentation on the platform support page (https://doc.rust-lang.org/nightly/rustc/platform-support/esp-idf.html), which gives the high-level summary of what functionality the platform intentionally doesn't have: no processes, no signals, no current working directory, etc.
We also noticed that ESP-IDF presents itself as
cfg(unix)
, but then stubs out a fair bit of UNIX functionality. Is it correct for it to present itself as UNIX? The right time to make that decision would be before promoting it to tier 2. (We're not making a judgement here on whether it should or shouldn't be; we wanted to raise the question for consideration and get someone to make a clear and deliberate case one way or the other.)Finally: from a quick check through the standard library, it looks like ESP-IDF just stubs out functionality (e.g. getcwd returning
"/"
) or always returns an error from functions that returnResult
, but we didn't see any cases where it panics in functions that aren't suppose to fail. Can you confirm if there are any cases where ESP-IDF panics because of missing functionality, in functions that don't have a way to do a proper error return?jieyouxu commentedon Apr 23, 2025
Forwarded your comment on the MCP thread, thanks!
cc @SergioGasquez
jieyouxu commentedon Apr 24, 2025
Waiting to hear back from author.
@rustbot label: -I-libs-nominated
ivmarkov commentedon Apr 25, 2025
(Apologies in advance if I should've addressed this only in the MCP thread.)
Thanks for taking the time to do a thorough review.
That’s a fair request and also provides us the opportunity to actually advertise not just what is not there, but also what is in there, as well as potential pitfalls and “gotchas”.
We’ll include this information in the PR shortly.
Great question - I think
cfg(unix)
really has two orthogonal aspects:(1)
cfg(unix)
is a bit of an advertisement to the actual users - including 3rd party crates - as to the fact that what is in there looks like a Posix system.Our subjective opinion here is that perhaps the glass is more of a “half full” than “half empty” so to say, in that the supported functionality is similar or out-numbers what is not supported. In particular (I'm surely missing a couple in both categories though):
std::thread
impl based on Posix threadsstd::io
impl based onnewlib
, including the standard streams, including non-blocking modestd::fs
- ditto. (Espressif chips can use FATFS, Spiffs and LittleFS either directly on their internal NOR-Flash storage, or via an SD-Card)std::net
- TCP and UDP sockets based on the BSD-API-compatible sockets exposed viaLwIP
- though that's shared withcfg(windows)
which also has "BSD sockets"Not supported:
std::process
dup
syscall)Moreover, perhaps the stronger argument is that even though this is a subset of Posix, what is in there is really Posix and not something else masquerading as such, as no syscalls into non-Posix ESP-IDF APIs (= non-
libc
) are actually made as far as I remember.With regards to 3rd party crates -
cfg(unix)
makes our lives easier in supporting those crates which do distinguish between e.g.cfg(unix)
andcfg(windows)
in that ESP-IDF is automatically treated as UNIX/Posix and as such the 3rd party crate either compiles cleanly on top of ESP-IDF with its already existing Posix-handling code, or requires minimal annotations to stub out functionality which does not exist on ESP-IDF.socket2
andrustix
come to mind as two examples that use thecfg(unix)
annotation.(2) The other aspect of
cfg(unix)
is the ability to end up with a small and easy to maintain STD implementation of ESP-IDF.Looking back at the original Tier 3 contribution from 2021 it was ~ 630 LOCs and hasn’t grown considerably ever since. The primary reason for this minimal implementation is precisely
cfg(unix)
. Flagging this up allowed us to basically tap into the existing UNIX PAL code already available in the Standard Library and sure - here and there - put an extracfg(target_os = “espidf”)
extra conditionals to handle ESP-IDF special-casing.The
cfg(target_os = “espidf”)
special-casing is very often not really just ESP-IDF specific, in that we noticed that besides ESP-IDF there were a bunch of other OSes - (illumos, redox, vitta, recently nuttx) - which fall in the “lowest common Posix denominator” case as well and for which the necessary code-path was already present so we just had to put an extra, cfg(target_os = “espidf”)
annotation to the already existing code.This has aged well. Living as part of the common
cfg(unix)
PAL umbrella as a Tier 3 target does mean that from time to time (on average - once per 3 to 4 months) we do get an occasional breakage (i.e. here, here, here, here) from newly-introduced functionality that did not account for ESP-IDF (as it was not built as part of CI - which is precisely what this Tier 2 promotion would address).But almost all of those breakages required a one-liner fix.
What would be the alternative?
It might very well the case, that if we are required to isolate the ESP-IDF portion in its own “non-Posix” Platform Access Layer, we might actually pull back on the Tier 2 promotion ourselves, as I’m not 100% convinced the benefits of such a motion would outweigh the disadvantages, at least in terms of maintenance effort.
A middle ground would be to stop advertising ESP-IDF as
cfg(unix)
but internally still hit the sys/unix PAL forcfg(target_os = “espidf”)
- as in modifying this line - though I wonder what the benefit of that would be? And then again, 3rd party crates…Off the top of my head, I can’t recall any such cases. The thing is, the Standard Library (or at least its
unix
code-path) is built on top of an API which is fallible anyway, in that all (most?) Posix functions are fallible, including withENOSYS
(i.e. “API not available”) and we make use of that.But we’ll double check.
joshtriplett commentedon Apr 25, 2025
Thank you for the detailed analysis!
Speaking for myself (not the rest of the team), the case for
cfg(unix)
sounds fairly reasonable.I do think it's worth emphasizing that the decision of whether to advertise
cfg(unix)
and the decision of whether to reuse the UNIX code in the standard library can be made independently.I also think it's worth considering, in the future, introducing one level of abstraction in the features of the standard library implementation, so that instead of putting espidf conditionals in various parts of the UNIX implementation, we can instead have
have_processes
andhave_threads
andhave_cwd
andhave_unix_sockets
annotations. Then we can have one central place that defines which of those conditionals each target meets (and sometimes introduce new ones). This would be similar to the Linux kernel's Kconfig abstractions likeARCH_HAS_
/ARCH_HAVE_
. But to be clear, that is not a blocker for promoting ESP-IDF; it's an alternate solution to the problem of having target-specific conditionals in more places than we might prefer.workingjubilee commentedon May 31, 2025
It's not clear to me if this is how users understand
target_family = "unix"
, as the concern in actuality extends beyond std: if I am any of the 100,000 crates on crates.io that is not maintained by the ESPIDF maintainers, then how do Icfg
on "targets with signals" without simply enumerating every singletarget_os
? The common understanding of "unix" does indeed seem to be that you support signals.