Skip to content

Validate transmute in CTFE #144030

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
+539 −1,042
Open

Validate transmute in CTFE #144030

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fa3976f
Add regression test
oli-obk Jul 16, 2025
1aa2951
Validate transmutes in CTFE
oli-obk Jul 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion compiler/rustc_const_eval/src/const_eval/eval_queries.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ fn eval_body_using_ecx<'tcx, R: InterpretationResult<'tcx>>(
// Since evaluation had no errors, validate the resulting constant.
const_validate_mplace(ecx, &ret, cid)?;

// Only report this after validation, as validaiton produces much better diagnostics.
// Only report this after validation, as validation produces much better diagnostics.
// FIXME: ensure validation always reports this and stop making interning care about it.

match intern_result {
Expand Down
9 changes: 9 additions & 0 deletions compiler/rustc_const_eval/src/interpret/cast.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,15 @@ impl<'tcx, M: Machine<'tcx>> InterpCx<'tcx, M> {
}

self.copy_op_allow_transmute(src, dest)?;

// Even if general validation is disabled, transmutes should always check their result.
if !M::enforce_validity(self, dest.layout) {
Comment on lines +145 to +146
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Miri has a flag for disabling validity checking -- this code makes it not work properly any more.

So this still needs to be somehow controlled by the machine, with a way for the machine to entirely disable all validity checking. Maybe enforce_validity should return a 3-state enum?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... I was just gonna add a const book for extra validation checks, but that may work, too

self.validate_operand(
&dest,
M::enforce_validity_recursively(self, dest.layout),
/*reset_provenance_and_padding*/ true,
)?;
}
}
}
interp_ok(())
Expand Down
312 changes: 76 additions & 236 deletions tests/ui/consts/const-eval/raw-bytes.32bit.stderr
View file
Open in desktop

Large diffs are not rendered by default.

312 changes: 76 additions & 236 deletions tests/ui/consts/const-eval/raw-bytes.64bit.stderr
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion tests/ui/consts/const-eval/raw-bytes.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,10 @@ const BAD_RANGE2: RestrictedRange2 = unsafe { RestrictedRange2(20) };
//~^ ERROR constructing invalid value

const NULL_FAT_PTR: NonNull<dyn Send> = unsafe {
//~^ ERROR constructing invalid value
let x: &dyn Send = &42;
let meta = std::ptr::metadata(x);
mem::transmute((0_usize, meta))
//~^ ERROR constructing invalid value
};

const UNALIGNED: &u16 = unsafe { mem::transmute(&[0u8; 4]) };
Expand Down
9 changes: 2 additions & 7 deletions tests/ui/consts/const-eval/transmute-const.stderr
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,8 @@
error[E0080]: constructing invalid value: encountered 0x03, but expected a boolean
--> $DIR/transmute-const.rs:4:1
--> $DIR/transmute-const.rs:4:29
|
LL | static FOO: bool = unsafe { mem::transmute(3u8) };
| ^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: 1, align: 1) {
03 │ .
}
| ^^^^^^^^^^^^^^^^^^^ evaluation of `FOO` failed here

error: aborting due to 1 previous error

Expand Down
10 changes: 5 additions & 5 deletions tests/ui/consts/const-eval/ub-enum.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,10 @@ const BAD_ENUM: Enum = unsafe { mem::transmute(1usize) };
//~^ ERROR expected a valid enum tag

const BAD_ENUM_PTR: Enum = unsafe { mem::transmute(&1) };
//~^ ERROR unable to turn pointer into integer
//~^ ERROR encountered a pointer, but expected an integer

const BAD_ENUM_WRAPPED: Wrap<Enum> = unsafe { mem::transmute(&1) };
//~^ ERROR unable to turn pointer into integer
//~^ ERROR encountered a pointer, but expected an integer

// # simple enum with discriminant 2

Expand All @@ -47,10 +47,10 @@ enum Enum2 {
const BAD_ENUM2: Enum2 = unsafe { mem::transmute(0usize) };
//~^ ERROR expected a valid enum tag
const BAD_ENUM2_PTR: Enum2 = unsafe { mem::transmute(&0) };
//~^ ERROR unable to turn pointer into integer
//~^ ERROR encountered a pointer, but expected an integer
// something wrapping the enum so that we test layout first, not enum
const BAD_ENUM2_WRAPPED: Wrap<Enum2> = unsafe { mem::transmute(&0) };
//~^ ERROR unable to turn pointer into integer
//~^ ERROR encountered a pointer, but expected an integer

// Undef enum discriminant.
#[repr(C)]
Expand All @@ -63,7 +63,7 @@ const BAD_ENUM2_UNDEF: Enum2 = unsafe { MaybeUninit { uninit: () }.init };

// Pointer value in an enum with a niche that is not just 0.
const BAD_ENUM2_OPTION_PTR: Option<Enum2> = unsafe { mem::transmute(&0) };
//~^ ERROR unable to turn pointer into integer
//~^ ERROR encountered a pointer, but expected an integer

// # valid discriminant for uninhabited variant

Expand Down
77 changes: 26 additions & 51 deletions tests/ui/consts/const-eval/ub-enum.stderr
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,57 +1,47 @@
error[E0080]: constructing invalid value at .<enum-tag>: encountered 0x01, but expected a valid enum tag
--> $DIR/ub-enum.rs:29:1
--> $DIR/ub-enum.rs:29:33
|
LL | const BAD_ENUM: Enum = unsafe { mem::transmute(1usize) };
| ^^^^^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: $SIZE, align: $ALIGN) {
HEX_DUMP
}
| ^^^^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM` failed here

error[E0080]: unable to turn pointer into integer
--> $DIR/ub-enum.rs:32:1
error[E0080]: constructing invalid value at .<enum-tag>: encountered a pointer, but expected an integer
--> $DIR/ub-enum.rs:32:37
|
LL | const BAD_ENUM_PTR: Enum = unsafe { mem::transmute(&1) };
| ^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM_PTR` failed here
| ^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM_PTR` failed here
|
= help: this code performed an operation that depends on the underlying bytes representing a pointer
= help: the absolute address of a pointer is not known at compile-time, so such operations are not supported

error[E0080]: unable to turn pointer into integer
--> $DIR/ub-enum.rs:35:1
error[E0080]: constructing invalid value at .0.<enum-tag>: encountered a pointer, but expected an integer
--> $DIR/ub-enum.rs:35:47
|
LL | const BAD_ENUM_WRAPPED: Wrap<Enum> = unsafe { mem::transmute(&1) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM_WRAPPED` failed here
| ^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM_WRAPPED` failed here
|
= help: this code performed an operation that depends on the underlying bytes representing a pointer
= help: the absolute address of a pointer is not known at compile-time, so such operations are not supported

error[E0080]: constructing invalid value at .<enum-tag>: encountered 0x0, but expected a valid enum tag
--> $DIR/ub-enum.rs:47:1
--> $DIR/ub-enum.rs:47:35
|
LL | const BAD_ENUM2: Enum2 = unsafe { mem::transmute(0usize) };
| ^^^^^^^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: $SIZE, align: $ALIGN) {
HEX_DUMP
}
| ^^^^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM2` failed here

error[E0080]: unable to turn pointer into integer
--> $DIR/ub-enum.rs:49:1
error[E0080]: constructing invalid value at .<enum-tag>: encountered a pointer, but expected an integer
--> $DIR/ub-enum.rs:49:39
|
LL | const BAD_ENUM2_PTR: Enum2 = unsafe { mem::transmute(&0) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM2_PTR` failed here
| ^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM2_PTR` failed here
|
= help: this code performed an operation that depends on the underlying bytes representing a pointer
= help: the absolute address of a pointer is not known at compile-time, so such operations are not supported

error[E0080]: unable to turn pointer into integer
--> $DIR/ub-enum.rs:52:1
error[E0080]: constructing invalid value at .0.<enum-tag>: encountered a pointer, but expected an integer
--> $DIR/ub-enum.rs:52:49
|
LL | const BAD_ENUM2_WRAPPED: Wrap<Enum2> = unsafe { mem::transmute(&0) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM2_WRAPPED` failed here
| ^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM2_WRAPPED` failed here
|
= help: this code performed an operation that depends on the underlying bytes representing a pointer
= help: the absolute address of a pointer is not known at compile-time, so such operations are not supported
Expand All @@ -62,47 +52,32 @@ error[E0080]: using uninitialized data, but this operation requires initialized
LL | const BAD_ENUM2_UNDEF: Enum2 = unsafe { MaybeUninit { uninit: () }.init };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM2_UNDEF` failed here

error[E0080]: unable to turn pointer into integer
--> $DIR/ub-enum.rs:65:1
error[E0080]: constructing invalid value at .<enum-tag>: encountered a pointer, but expected an integer
--> $DIR/ub-enum.rs:65:54
|
LL | const BAD_ENUM2_OPTION_PTR: Option<Enum2> = unsafe { mem::transmute(&0) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM2_OPTION_PTR` failed here
| ^^^^^^^^^^^^^^^^^^ evaluation of `BAD_ENUM2_OPTION_PTR` failed here
|
= help: this code performed an operation that depends on the underlying bytes representing a pointer
= help: the absolute address of a pointer is not known at compile-time, so such operations are not supported

error[E0080]: constructing invalid value at .<enum-tag>: encountered an uninhabited enum variant
--> $DIR/ub-enum.rs:82:1
--> $DIR/ub-enum.rs:82:62
|
LL | const BAD_UNINHABITED_VARIANT1: UninhDiscriminant = unsafe { mem::transmute(1u8) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: $SIZE, align: $ALIGN) {
HEX_DUMP
}
| ^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_UNINHABITED_VARIANT1` failed here

error[E0080]: constructing invalid value at .<enum-tag>: encountered an uninhabited enum variant
--> $DIR/ub-enum.rs:84:1
--> $DIR/ub-enum.rs:84:62
|
LL | const BAD_UNINHABITED_VARIANT2: UninhDiscriminant = unsafe { mem::transmute(3u8) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: $SIZE, align: $ALIGN) {
HEX_DUMP
}
| ^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_UNINHABITED_VARIANT2` failed here

error[E0080]: constructing invalid value at .<enum-variant(Some)>.0.1: encountered 0xffffffff, but expected a valid unicode scalar value (in `0..=0x10FFFF` but not in `0xD800..=0xDFFF`)
--> $DIR/ub-enum.rs:92:1
error[E0080]: constructing invalid value: encountered 0xffffffff, but expected a valid unicode scalar value (in `0..=0x10FFFF` but not in `0xD800..=0xDFFF`)
--> $DIR/ub-enum.rs:92:67
|
LL | const BAD_OPTION_CHAR: Option<(char, char)> = Some(('x', unsafe { mem::transmute(!0u32) }));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: $SIZE, align: $ALIGN) {
HEX_DUMP
}
| ^^^^^^^^^^^^^^^^^^^^^ evaluation of `BAD_OPTION_CHAR` failed here

error[E0080]: constructing invalid value at .<enum-tag>: encountered an uninhabited enum variant
--> $DIR/ub-enum.rs:97:77
Expand Down
65 changes: 20 additions & 45 deletions tests/ui/consts/const-eval/ub-incorrect-vtable.32bit.stderr
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,57 +1,32 @@
error[E0080]: constructing invalid value: encountered ALLOC1<imm>, but expected a vtable pointer
--> $DIR/ub-incorrect-vtable.rs:18:1
error[E0080]: constructing invalid value: encountered ALLOC2<imm>, but expected a vtable pointer
--> $DIR/ub-incorrect-vtable.rs:19:14
|
LL | const INVALID_VTABLE_ALIGNMENT: &dyn Trait =
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: 8, align: 4) {
╾ALLOC0<imm>╼ ╾ALLOC1<imm>╼ │ ╾──╼╾──╼
}
LL | unsafe { std::mem::transmute((&92u8, &[0usize, 1usize, 1000usize])) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `INVALID_VTABLE_ALIGNMENT` failed here

error[E0080]: constructing invalid value: encountered ALLOC3<imm>, but expected a vtable pointer
--> $DIR/ub-incorrect-vtable.rs:22:1
|
LL | const INVALID_VTABLE_SIZE: &dyn Trait =
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
--> $DIR/ub-incorrect-vtable.rs:23:14
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: 8, align: 4) {
╾ALLOC2<imm>╼ ╾ALLOC3<imm>╼ │ ╾──╼╾──╼
}
LL | unsafe { std::mem::transmute((&92u8, &[1usize, usize::MAX, 1usize])) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `INVALID_VTABLE_SIZE` failed here

error[E0080]: constructing invalid value at .0: encountered ALLOC5<imm>, but expected a vtable pointer
--> $DIR/ub-incorrect-vtable.rs:31:1
error[E0080]: constructing invalid value at .0: encountered ALLOC4<imm>, but expected a vtable pointer
--> $DIR/ub-incorrect-vtable.rs:32:14
|
LL | const INVALID_VTABLE_ALIGNMENT_UB: W<&dyn Trait> =
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: 8, align: 4) {
╾ALLOC4<imm>╼ ╾ALLOC5<imm>╼ │ ╾──╼╾──╼
}
LL | unsafe { std::mem::transmute((&92u8, &(drop_me as fn(*mut usize), 1usize, 1000usize))) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `INVALID_VTABLE_ALIGNMENT_UB` failed here

error[E0080]: constructing invalid value at .0: encountered ALLOC7<imm>, but expected a vtable pointer
--> $DIR/ub-incorrect-vtable.rs:35:1
|
LL | const INVALID_VTABLE_SIZE_UB: W<&dyn Trait> =
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
error[E0080]: constructing invalid value at .0: encountered ALLOC5<imm>, but expected a vtable pointer
--> $DIR/ub-incorrect-vtable.rs:36:14
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: 8, align: 4) {
╾ALLOC6<imm>╼ ╾ALLOC7<imm>╼ │ ╾──╼╾──╼
}
LL | unsafe { std::mem::transmute((&92u8, &(drop_me as fn(*mut usize), usize::MAX, 1usize))) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `INVALID_VTABLE_SIZE_UB` failed here

error[E0080]: constructing invalid value at .0: encountered ALLOC9<imm>, but expected a vtable pointer
--> $DIR/ub-incorrect-vtable.rs:40:1
|
LL | const INVALID_VTABLE_UB: W<&dyn Trait> =
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ it is undefined behavior to use this value
error[E0080]: constructing invalid value at .0: encountered ALLOC6<imm>, but expected a vtable pointer
--> $DIR/ub-incorrect-vtable.rs:41:14
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: 8, align: 4) {
╾ALLOC8<imm>╼ ╾ALLOC9<imm>╼ │ ╾──╼╾──╼
}
LL | unsafe { std::mem::transmute((&92u8, &(drop_me as fn(*mut usize), 1usize, 1usize))) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `INVALID_VTABLE_UB` failed here

error[E0080]: constructing invalid value at .1: encountered a dangling reference (going beyond the bounds of its allocation)
--> $DIR/ub-incorrect-vtable.rs:86:1
Expand All @@ -61,7 +36,7 @@ LL | const G: Wide = unsafe { Transmute { t: FOO }.u };
|
= note: The rules on what exactly is undefined behavior aren't clear, so this check might be overzealous. Please open an issue on the rustc repository if you believe it should not be considered undefined behavior.
= note: the raw bytes of the constant (size: 8, align: 4) {
╾ALLOC10<imm>╼ ╾ALLOC11╼ │ ╾──╼╾──╼
╾ALLOC0<imm>╼ ╾ALLOC1╼ │ ╾──╼╾──╼
}

error: aborting due to 6 previous errors
Expand Down
Loading
Loading