Skip to content

Using Option to get undefined behaviour in a const #95332

@arnaudgolfouse

Description

@arnaudgolfouse
Contributor

I tried this code:

use std::num::NonZeroU8;

pub const UNDEFINED: Option<NonZeroU8> = Some(unsafe { NonZeroU8::new_unchecked(0) });

And assumed the compiler would reject it, but there was no compilation errors.

Note that the following also compiled:

use std::num::NonZeroU8;

pub const UNDEFINED: Option<NonZeroU8> = Some(unsafe { NonZeroU8::new_unchecked(0) });

// Sometime using the undefined constant triggers an error: not here...
pub use_undefined() -> Option<NonZeroU8> { UNDEFINED }

// This is not limited to Option !
pub enum MyOption {
    Some(NonZeroU8),
    None
}

pub const UNDEFINED2: MyOption = MyOption::Some(unsafe { NonZeroU8::new_unchecked(0) });

pub use_undefined2() -> MyOption { UNDEFINED2 }

I am assuming this is caused by layout optimizations ?

Meta

rustc --version --verbose:

rustc 1.59.0 (9d1b2106e 2022-02-23)
binary: rustc
commit-hash: 9d1b2106e23b1abd32fce1f17267604a5102f57a
commit-date: 2022-02-23
host: x86_64-unknown-linux-gnu
release: 1.59.0
LLVM version: 13.0.0

rustc +nightly --version --verbose:

rustc 1.61.0-nightly (68369a041 2022-02-22)
binary: rustc
commit-hash: 68369a041cea809a87e5bd80701da90e0e0a4799
commit-date: 2022-02-22
host: x86_64-unknown-linux-gnu
release: 1.61.0-nightly
LLVM version: 14.0.0

Activity

RalfJung

RalfJung commented on Mar 26, 2022

@RalfJung
Member

And assumed the compiler would reject it, but there was no compilation errors.

We do not guarantee that all Undefined Behavior is detected, not at run-time and not at compile-time. When you use unsafe, it is up to you to make sure you follow the rules.

We do detect some UB, e.g. here:

pub const UNDEFINED: NonZeroU8 = unsafe { NonZeroU8::new_unchecked(0) };

But exhaustively checking for all UB is super expensive at best. Even Miri, which does a lot more work than CTFE, does not detect all UB.

So I consider this not a bug.
Cc @rust-lang/wg-const-eval

saethlin

saethlin commented on Mar 26, 2022

@saethlin
Member

Miri detects this if the code is shifted to normal runtime eval. Is it worth considering moving this check into CTFE or does that just open up too many questions?

Lokathor

Lokathor commented on Mar 26, 2022

@Lokathor
Contributor

on the one hand i would like CTFE to catch it, on the other I'm worried if people might think that it'll then catch the runtime version as well (which it won't).

saethlin

saethlin commented on Mar 26, 2022

@saethlin
Member

I'm not so sure about that. C++ has set a precedent that const-eval has extra UB checking powers that normal runtime eval does not have.

oli-obk

oli-obk commented on Mar 26, 2022

@oli-obk
Contributor

As Ralf noted, this can be very expensive. We can benchmark it to see how expensive and make a decision based on that.

We can also consider using a lint for these extra checks and only turning them on if the lint is warn/deny (and set it to allow by default for speed).

Oh, even funnier idea: clippy can overwrite the const eval query to make it run with validation. This way cargo clippy will catch more UB than cargo check

ChrisDenton

ChrisDenton commented on Mar 26, 2022

@ChrisDenton
Member

Specifically in the narrow case of NonZero* integer types, would it be quicker for Rust to warn against any use of new_unchecked while in a const context?

saethlin

saethlin commented on Mar 26, 2022

@saethlin
Member

I am suspicious of claims that this check is expensive. Miri is poorly-benchmarked, does CTFE have any benchmarks?

oli-obk

oli-obk commented on Mar 26, 2022

@oli-obk
Contributor

Yes, CTFE is benchmarked with the rustc benchmarks. Trying it out is as simple as opening a PR removing

fn enforce_validity(_ecx: &InterpCx<$mir, $tcx, Self>) -> bool {
and manually adding it to the two use sites of the macro. Then flip the bool for the const_eval case.

fee1-dead

fee1-dead commented on Mar 27, 2022

@fee1-dead
Member

It feels like it can just have debug_assert! inside the unchecked functions, or would it not be ideal?

saethlin

saethlin commented on Mar 27, 2022

@saethlin
Member

@fee1-dead I've had a PR up for the past few months that does just that :) #92686

RalfJung

RalfJung commented on Mar 27, 2022

@RalfJung
Member

That wouldn't make any difference here though, since the stdlib we ship is compiled without debug assertions.

saethlin

saethlin commented on Mar 27, 2022

@saethlin
Member

sigh of course the bigger problem is that all those checks are disabled in const eval. And I did that because I assumed while writing it that const eval would catch all these things. Around and around we go...

@oli-obk What's wrong with just doing this? I found out what is wrong with this (MIR transforms panic if they do validation).

diff --git a/compiler/rustc_const_eval/src/interpret/machine.rs b/compiler/rustc_const_eval/src/interpret/machine.rs
index 7d75c84d108..121121cdab2 100644
--- a/compiler/rustc_const_eval/src/interpret/machine.rs
+++ b/compiler/rustc_const_eval/src/interpret/machine.rs
@@ -422,7 +422,7 @@ fn force_int_for_alignment_check(_memory_extra: &Self::MemoryExtra) -> bool {
 
     #[inline(always)]
     fn enforce_validity(_ecx: &InterpCx<$mir, $tcx, Self>) -> bool {
-        false // for now, we don't enforce validity
+        true
     }
 
     #[inline(always)]
fee1-dead

fee1-dead commented on Mar 27, 2022

@fee1-dead
Member

That wouldn't make any difference here though, since the stdlib we ship is compiled without debug assertions.

Then an unconditional check using const_eval_select would suffice?

RalfJung

RalfJung commented on Mar 27, 2022

@RalfJung
Member

Then an unconditional check using const_eval_select would suffice?

Yes that would be an option.

10 remaining items

RalfJung

RalfJung commented on Apr 3, 2022

@RalfJung
Member

Simply putting the whole benchmark in an unsafe block or making every fn an unsafe fn does not produce anything measurably worse than the all-safe overhead (which I think I just drove down to 0.8%), because this check is sloppy enough that it basically doesn't validate anything.

So the check interacts badly with macros, or so? Because that should still all be unsafe code then, right?

saethlin

saethlin commented on Apr 3, 2022

@saethlin
Member

I suspect the check interacts badly with functions. It's extremely local and sloppy, in the name of doing little work when there is no unsafe code. It could definitely be tightened up if you or Oli have suggestions on a more principled but 🤔 appropriately thrifty way to do this.

Whenever we call load_mir, the HIR for the loaded Instance is searched for unsafe blocks. If one is found, checking is on. If one isn't found, checking is turned off.

If we call enforce_validity without seeing a call to load_mir and we have a stack with one frame, we check it for unsafe. If it has no unsafe blocks in it, checking is off. If it does contain an unsafe block, or there is more than one stack frame, checking is on. Subsequent calls to enforce_validity will not not change whether or not checking is turned on. Only a call to load_mir can do that.

We also defensively assume that MIR from outside the current crate contains unsafe.

added
A-const-evalArea: Constant evaluation, covers all const contexts (static, const fn, ...)
C-discussionCategory: Discussion or questions that doesn't represent real issues.
T-opsemRelevant to the opsem team
T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
and removed
C-bugCategory: This is a bug.
on Jan 24, 2024
clarfonthey

clarfonthey commented on Jun 30, 2025

@clarfonthey
Contributor

Just poking around on this: it appears that the compiler does properly check for UB in cases like this, but only if the constants are actually used. Check this example:

#[repr(u8)]
enum Test {
    False = 0,
    True = 0xFF,
}
impl Test {
    const BAD: Test = unsafe { core::mem::transmute::<u8, Test>(0x12) };
}

This will report no error, but the below will:

#[repr(u8)]
enum Test {
    False = 0,
    True = 0xFF,
}
impl Test {
    const BAD: Test = unsafe { core::mem::transmute::<u8, Test>(0x12) };
}
#[test]
fn test() {
    let _ = Test::BAD;
}

With the error:

error[E0080]: it is undefined behavior to use this value
 --> src/lib.rs:7:5
  |
7 |     const BAD: Test = unsafe { core::mem::transmute::<u8, Test>(0x12) };
  |     ^^^^^^^^^^^^^^^ constructing invalid value at .<enum-tag>: encountered 0x12, but expected a valid enum tag
  |
  = note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
  = note: the raw bytes of the constant (size: 1, align: 1) {
              12                                              │ .
          }

It also appears to apply to underscore constants:

#[repr(u8)]
enum Test {
    False = 0,
    True = 0xFF,
}
const _: Test = unsafe { core::mem::transmute::<u8, Test>(0x12) };

But it only works if the underscore constant is directly evaluated; the below still accepts the code:

#[repr(u8)]
enum Test {
    False = 0,
    True = 0xFF,
}
const _: () = {
    let _ = unsafe { core::mem::transmute::<u8, Test>(0x12) };
};
saethlin

saethlin commented on Jun 30, 2025

@saethlin
Member

I think you are just observing the details of what is/isn't considered a "mentioned" item, nothing to do with UB checking. We should have a meta issue somewhere for people being confused by what is and isn't considered mentioned.

RalfJung

RalfJung commented on Jun 30, 2025

@RalfJung
Member

Indeed, the first effect you are observing is orthogonal to this issue; that is just a matter of which constants even get evaluated.

The 2nd effect is related to the issue: we only UB-check the final value of a const in detail, not all the intermediate values.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-const-evalArea: Constant evaluation, covers all const contexts (static, const fn, ...)C-discussionCategory: Discussion or questions that doesn't represent real issues.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.T-opsemRelevant to the opsem team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @RalfJung@oli-obk@ChrisDenton@Lokathor@saethlin

        Issue actions

          Using `Option` to get undefined behaviour in a `const` · Issue #95332 · rust-lang/rust