Skip to content

use openat when encountering ENAMETOOLONG #88731

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions library/std/src/fs/tests.rs
Original file line number Diff line number Diff line change
@@ -1707,3 +1707,26 @@ fn test_file_times() {
assert_eq!(metadata.created().unwrap(), created);
}
}

#[test]
#[cfg(any(target_os = "linux", target_os = "android"))]
fn deep_traversal() -> crate::io::Result<()> {
use crate::fs::{create_dir_all, metadata, remove_dir_all, write};
use crate::iter::repeat;

let tmpdir = tmpdir();

let segment = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";

let mut dir = tmpdir.join(segment);
repeat(segment).take(100).for_each(|name| dir.push(name));
assert!(dir.as_os_str().len() > libc::PATH_MAX as usize);
let file = dir.join("b");

create_dir_all(&dir).expect("deep create tailed");
write(&file, "foo").expect("deep write failed");
metadata(&file).expect("deep stat failed");
remove_dir_all(&dir).expect("deep remove failed");

Ok(())
}
1 change: 1 addition & 0 deletions library/std/src/lib.rs
Original file line number Diff line number Diff line change
@@ -296,6 +296,7 @@
#![feature(int_roundings)]
#![feature(ip)]
#![feature(ip_in_core)]
#![feature(iter_advance_by)]
#![feature(maybe_uninit_slice)]
#![feature(maybe_uninit_uninit_array)]
#![feature(maybe_uninit_write_slice)]
7 changes: 3 additions & 4 deletions library/std/src/sys/common/small_c_string.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
use crate::ffi::{CStr, CString};
use crate::ffi::{CStr, CString, OsStr};
use crate::mem::MaybeUninit;
use crate::path::Path;
use crate::slice;
use crate::{io, ptr};

@@ -15,11 +14,11 @@ const NUL_ERR: io::Error =
io::const_io_error!(io::ErrorKind::InvalidInput, "file name contained an unexpected NUL byte");

#[inline]
pub fn run_path_with_cstr<T, F>(path: &Path, f: F) -> io::Result<T>
pub fn run_path_with_cstr<T, F>(path: &(impl AsRef<OsStr> + ?Sized), f: F) -> io::Result<T>
where
F: FnOnce(&CStr) -> io::Result<T>,
{
run_with_cstr(path.as_os_str().as_os_str_bytes(), f)
run_with_cstr(path.as_ref().as_os_str_bytes(), f)
}

#[inline]
232 changes: 187 additions & 45 deletions library/std/src/sys/unix/fs.rs
Original file line number Diff line number Diff line change
@@ -43,6 +43,8 @@ use libc::c_char;
use libc::dirfd;
#[cfg(any(target_os = "linux", target_os = "emscripten"))]
use libc::fstatat64;
#[cfg(all(miri, any(target_os = "linux")))]
use libc::open64;
Comment on lines +46 to +47
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit surprised that this import is only present for Miri -- that seems strange?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two code-paths, one that uses openat and one that uses open. Miri only supports the latter, so the entire machinery around -at syscalls is configured-out under miri and the alternative is configured-in.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, and there are some other import for open64 below for other targets. The entire set of cfgs is just tricky to have an overview of. Makes sense.

#[cfg(any(
target_os = "android",
target_os = "solaris",
@@ -86,7 +88,11 @@ use libc::{
lstat as lstat64, off_t as off64_t, open as open64, stat as stat64,
};
#[cfg(any(target_os = "linux", target_os = "emscripten", target_os = "l4re"))]
use libc::{dirent64, fstat64, ftruncate64, lseek64, lstat64, off64_t, open64, stat64};
use libc::{dirent64, fstat64, ftruncate64, lseek64, lstat64, off64_t, stat64};

// FIXME: port this to other unices that support *at syscalls
#[cfg(all(not(miri), any(target_os = "linux", target_os = "android")))]
mod dir_fd;

pub use crate::sys_common::fs::try_exists;

@@ -141,7 +147,8 @@ cfg_has_statx! {{
// Default `stat64` contains no creation time and may have 32-bit `time_t`.
unsafe fn try_statx(
fd: c_int,
path: *const c_char,
raw_path: *const c_char,
path: &Path,
flags: i32,
mask: u32,
) -> Option<io::Result<FileAttr>> {
@@ -169,19 +176,17 @@ cfg_has_statx! {{
}

let mut buf: libc::statx = mem::zeroed();
if let Err(err) = cvt(statx(fd, path, flags, mask, &mut buf)) {
if STATX_SAVED_STATE.load(Ordering::Relaxed) == STATX_STATE::Present as u8 {
return Some(Err(err));
}
let result = match cvt(statx(fd, raw_path, flags, mask, &mut buf)) {
o @ Ok(_) => o,
e @ Err(_) if STATX_SAVED_STATE.load(Ordering::Relaxed) == STATX_STATE::Present as u8 => e,

// Availability not checked yet.
//
// First try the cheap way.
if err.raw_os_error() == Some(libc::ENOSYS) {
Err(err) if err.raw_os_error() == Some(libc::ENOSYS) => {
STATX_SAVED_STATE.store(STATX_STATE::Unavailable as u8, Ordering::Relaxed);
return None;
}

return None
},
// Error other than `ENOSYS` is not a good enough indicator -- it is
// known that `EPERM` can be returned as a result of using seccomp to
// block the syscall.
@@ -192,16 +197,27 @@ cfg_has_statx! {{
// previous iteration of the code checked it for all errors and for now
// this is retained.
// FIXME what about transient conditions like `ENOMEM`?
let err2 = cvt(statx(0, ptr::null(), 0, libc::STATX_ALL, ptr::null_mut()))
.err()
.and_then(|e| e.raw_os_error());
if err2 == Some(libc::EFAULT) {
STATX_SAVED_STATE.store(STATX_STATE::Present as u8, Ordering::Relaxed);
return Some(Err(err));
} else {
STATX_SAVED_STATE.store(STATX_STATE::Unavailable as u8, Ordering::Relaxed);
return None;
e @ Err(_) => {
let err2 = cvt(statx(0, ptr::null(), 0, libc::STATX_ALL, ptr::null_mut()))
.err()
.and_then(|e| e.raw_os_error());
if err2 == Some(libc::EFAULT) {
STATX_SAVED_STATE.store(STATX_STATE::Present as u8, Ordering::Relaxed);
e
} else {
STATX_SAVED_STATE.store(STATX_STATE::Unavailable as u8, Ordering::Relaxed);
return None
}
}
};

let result = long_filename_fallback!(path, result, |dirfd, file_name| {
// FIXME: use libc::AT_EMPTY_PATH
cvt(statx(dirfd.as_raw_fd(), file_name.as_ptr(), flags, mask, &mut buf))
});

if let Err(err) = result {
return Some(Err(err));
}

// We cannot fill `stat64` exhaustively because of private padding fields.
@@ -259,9 +275,25 @@ pub struct ReadDir {
}

impl ReadDir {
#[cfg(not(any(target_os = "android", target_os = "linux")))]
fn new(inner: InnerReadDir) -> Self {
Self { inner: Arc::new(inner), end_of_stream: false }
}

#[cfg(any(target_os = "android", target_os = "linux"))]
fn from_dirp(ptr: *mut libc::DIR, root: PathBuf) -> ReadDir {
let inner = InnerReadDir { dirp: Dir(ptr), root };
ReadDir {
inner: Arc::new(inner),
#[cfg(not(any(
target_os = "solaris",
target_os = "illumos",
target_os = "fuchsia",
target_os = "redox",
)))]
end_of_stream: false,
}
}
}

struct Dir(*mut libc::DIR);
@@ -820,6 +852,7 @@ impl DirEntry {
if let Some(ret) = unsafe { try_statx(
fd,
name,
self.file_name_os_str().as_ref(),
libc::AT_SYMLINK_NOFOLLOW | libc::AT_STATX_SYNC_AS_STAT,
libc::STATX_ALL,
) } {
@@ -1052,19 +1085,43 @@ impl OpenOptions {

impl File {
pub fn open(path: &Path, opts: &OpenOptions) -> io::Result<File> {
run_path_with_cstr(path, |path| File::open_c(path, opts))
let result = run_path_with_cstr(path, |path| File::open_c(None, &path, opts));

#[cfg(all(not(miri), any(target_os = "linux", target_os = "android")))]
let result = {
use crate::io::ErrorKind;
match result {
Ok(file) => Ok(file),
Err(e) if e.kind() == ErrorKind::InvalidFilename => {
dir_fd::open_deep(None, path, opts)
}
Err(e) => Err(e),
}
};

result
}

pub fn open_c(path: &CStr, opts: &OpenOptions) -> io::Result<File> {
#[cfg(any(miri, not(any(target_os = "linux", target_os = "android"))))]
pub fn open_c(
dirfd: Option<BorrowedFd<'_>>,
path: &CStr,
opts: &OpenOptions,
) -> io::Result<File> {
let flags = libc::O_CLOEXEC
| opts.get_access_mode()?
| opts.get_creation_mode()?
| (opts.custom_flags as c_int & !libc::O_ACCMODE);
// The third argument of `open64` is documented to have type `mode_t`. On
// some platforms (like macOS, where `open64` is actually `open`), `mode_t` is `u16`.
// However, since this is a variadic function, C integer promotion rules mean that on
// the ABI level, this still gets passed as `c_int` (aka `u32` on Unix platforms).
let fd = cvt_r(|| unsafe { open64(path.as_ptr(), flags, opts.mode as c_int) })?;
let fd = match dirfd {
None => {
// The third argument of `open64` is documented to have type `mode_t`. On
// some platforms (like macOS, where `open64` is actually `open`), `mode_t` is `u16`.
// However, since this is a variadic function, C integer promotion rules mean that on
// the ABI level, this still gets passed as `c_int` (aka `u32` on Unix platforms).
cvt_r(|| unsafe { open64(path.as_ptr(), flags, opts.mode as c_int) })?
}
Some(dirfd) => return super::unsupported::unsupported(),
};
Ok(File(unsafe { FileDesc::from_raw_fd(fd) }))
}

@@ -1075,6 +1132,7 @@ impl File {
if let Some(ret) = unsafe { try_statx(
fd,
b"\0" as *const _ as *const c_char,
"".as_ref(),
libc::AT_EMPTY_PATH | libc::AT_STATX_SYNC_AS_STAT,
libc::STATX_ALL,
) } {
@@ -1322,7 +1380,13 @@ impl DirBuilder {
}

pub fn mkdir(&self, p: &Path) -> io::Result<()> {
run_path_with_cstr(p, |p| cvt(unsafe { libc::mkdir(p.as_ptr(), self.mode) }).map(|_| ()))
let result = run_path_with_cstr(p, |p| cvt(unsafe { libc::mkdir(p.as_ptr(), self.mode) }));

let result = long_filename_fallback!(p, result, |dirfd, file_name| {
cvt(unsafe { libc::mkdirat(dirfd.as_raw_fd(), file_name.as_ptr(), self.mode) })
});

result.map(|_| ())
}

pub fn set_mode(&mut self, mode: u32) {
@@ -1499,19 +1563,49 @@ impl fmt::Debug for File {
}
}

pub fn readdir(path: &Path) -> io::Result<ReadDir> {
let ptr = run_path_with_cstr(path, |p| unsafe { Ok(libc::opendir(p.as_ptr())) })?;
fn cvt_p<T>(ptr: *mut T) -> io::Result<*mut T> {
if ptr.is_null() {
Err(Error::last_os_error())
} else {
let root = path.to_path_buf();
let inner = InnerReadDir { dirp: Dir(ptr), root };
Ok(ReadDir::new(inner))
return Err(Error::last_os_error());
}
Ok(ptr)
}

pub fn unlink(p: &Path) -> io::Result<()> {
run_path_with_cstr(p, |p| cvt(unsafe { libc::unlink(p.as_ptr()) }).map(|_| ()))
pub fn readdir(path: &Path) -> io::Result<ReadDir> {
let root = path.to_path_buf();
let ptr = cvt_p(run_path_with_cstr(path, |p| unsafe { Ok(libc::opendir(p.as_ptr())) })?);

let ptr = match ptr {
#[cfg(any(target_os = "linux", target_os = "android"))]
Err(e) if e.kind() == crate::io::ErrorKind::InvalidFilename => {
let mut opts = OpenOptions::new();
opts.read(true);
opts.custom_flags(libc::O_DIRECTORY);
let fd = File::open(path, &opts)?.into_raw_fd();
cvt_p(unsafe { libc::fdopendir(fd) })
}
other @ _ => other,
}?;

let inner = InnerReadDir { dirp: Dir(ptr), root };
Ok(ReadDir {
inner: Arc::new(inner),
#[cfg(not(any(
target_os = "solaris",
target_os = "illumos",
target_os = "fuchsia",
target_os = "redox",
)))]
end_of_stream: false,
})
}

pub fn unlink(path: &Path) -> io::Result<()> {
let result = run_path_with_cstr(path, |p| cvt(unsafe { libc::unlink(p.as_ptr()) }));
let result = long_filename_fallback!(path, result, |dirfd, file_name| {
cvt(unsafe { libc::unlinkat(dirfd.as_raw_fd(), file_name.as_ptr(), 0) })
});

result.map(|_| ())
}

pub fn rename(old: &Path, new: &Path) -> io::Result<()> {
@@ -1526,8 +1620,15 @@ pub fn set_perm(p: &Path, perm: FilePermissions) -> io::Result<()> {
run_path_with_cstr(p, |p| cvt_r(|| unsafe { libc::chmod(p.as_ptr(), perm.mode) }).map(|_| ()))
}

pub fn rmdir(p: &Path) -> io::Result<()> {
run_path_with_cstr(p, |p| cvt(unsafe { libc::rmdir(p.as_ptr()) }).map(|_| ()))
pub fn rmdir(path: &Path) -> io::Result<()> {
let result = run_path_with_cstr(path, |p| cvt(unsafe { libc::rmdir(p.as_ptr()) }));

#[cfg(any(target_os = "linux", target_os = "android"))]
let result = long_filename_fallback!(path, result, |dirfd, file_name| {
cvt(unsafe { libc::unlinkat(dirfd.as_raw_fd(), file_name.as_ptr(), libc::AT_REMOVEDIR) })
});

result.map(|_| ())
}

pub fn readlink(p: &Path) -> io::Result<PathBuf> {
@@ -1602,12 +1703,20 @@ pub fn link(original: &Path, link: &Path) -> io::Result<()> {
})
}

pub fn stat(p: &Path) -> io::Result<FileAttr> {
run_path_with_cstr(p, |p| {
// On linux this is the default behavior for lstat and stat but it has to be set explicitly for fstatat
#[cfg(any(target_os = "linux", target_os = "android"))]
const DEFAULT_STATAT_FLAGS: c_int = libc::AT_NO_AUTOMOUNT;

#[cfg(not(any(target_os = "linux", target_os = "android")))]
const DEFAULT_STATAT_FLAGS: c_int = 0;

pub fn stat(path: &Path) -> io::Result<FileAttr> {
run_path_with_cstr(path, |p| {
cfg_has_statx! {
if let Some(ret) = unsafe { try_statx(
libc::AT_FDCWD,
p.as_ptr(),
path,
libc::AT_STATX_SYNC_AS_STAT,
libc::STATX_ALL,
) } {
@@ -1616,17 +1725,23 @@ pub fn stat(p: &Path) -> io::Result<FileAttr> {
}

let mut stat: stat64 = unsafe { mem::zeroed() };
cvt(unsafe { stat64(p.as_ptr(), &mut stat) })?;
let result = cvt(unsafe { stat64(p.as_ptr(), &mut stat) });
long_filename_fallback!(path, result, |dirfd, file_name| {
cvt(unsafe {
fstatat64(dirfd.as_raw_fd(), file_name.as_ptr(), &mut stat, DEFAULT_STATAT_FLAGS)
})
})?;
Ok(FileAttr::from_stat64(stat))
})
}

pub fn lstat(p: &Path) -> io::Result<FileAttr> {
run_path_with_cstr(p, |p| {
pub fn lstat(path: &Path) -> io::Result<FileAttr> {
run_path_with_cstr(path, |p| {
cfg_has_statx! {
if let Some(ret) = unsafe { try_statx(
libc::AT_FDCWD,
p.as_ptr(),
path,
libc::AT_SYMLINK_NOFOLLOW | libc::AT_STATX_SYNC_AS_STAT,
libc::STATX_ALL,
) } {
@@ -1635,7 +1750,17 @@ pub fn lstat(p: &Path) -> io::Result<FileAttr> {
}

let mut stat: stat64 = unsafe { mem::zeroed() };
cvt(unsafe { lstat64(p.as_ptr(), &mut stat) })?;
let result = cvt(unsafe { lstat64(p.as_ptr(), &mut stat) });
long_filename_fallback!(path, result, |dirfd, file_name| {
cvt(unsafe {
fstatat64(
dirfd.as_raw_fd(),
file_name.as_ptr(),
&mut stat,
DEFAULT_STATAT_FLAGS | libc::AT_SYMLINK_NOFOLLOW,
)
})
})?;
Ok(FileAttr::from_stat64(stat))
})
}
@@ -1654,6 +1779,17 @@ pub fn canonicalize(p: &Path) -> io::Result<PathBuf> {
})))
}

macro long_filename_fallback($path:expr, $result:expr, $fallback:expr) {{
cfg_if::cfg_if! {
// miri doesn't support the *at syscalls
if #[cfg(all(not(miri), any(target_os = "linux", target_os = "android")))] {
dir_fd::long_filename_fallback($result, $path, $fallback)
} else {
$result
}
}
}}

fn open_from(from: &Path) -> io::Result<(crate::fs::File, crate::fs::Metadata)> {
use crate::fs::File;
use crate::sys_common::fs::NOT_FILE_ERROR;
@@ -1683,7 +1819,6 @@ fn open_to_and_set_permissions(
reader_metadata: crate::fs::Metadata,
) -> io::Result<(crate::fs::File, crate::fs::Metadata)> {
use crate::fs::OpenOptions;
use crate::os::unix::fs::{OpenOptionsExt, PermissionsExt};

let perm = reader_metadata.permissions();
let writer = OpenOptions::new()
@@ -1888,13 +2023,20 @@ mod remove_dir_impl {
pub use crate::sys_common::fs::remove_dir_all;
}

#[cfg(all(not(miri), any(target_os = "linux", target_os = "android")))]
mod remove_dir_impl {
pub use super::dir_fd::remove_dir_all;
}

// Modern implementation using openat(), unlinkat() and fdopendir()
#[cfg(not(any(
target_os = "redox",
target_os = "espidf",
target_os = "horizon",
target_os = "vita",
target_os = "nto",
target_os = "linux",
target_os = "android",
miri
)))]
mod remove_dir_impl {
464 changes: 464 additions & 0 deletions library/std/src/sys/unix/fs/dir_fd.rs

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion library/std/src/sys/unix/mod.rs
Original file line number Diff line number Diff line change
@@ -420,7 +420,7 @@ cfg_if::cfg_if! {
}
}

#[cfg(any(target_os = "espidf", target_os = "horizon", target_os = "vita"))]
#[cfg(any(miri, not(any(target_os = "linux", target_os = "android"))))]
mod unsupported {
use crate::io;

2 changes: 1 addition & 1 deletion library/std/src/sys/unix/process/process_common.rs
Original file line number Diff line number Diff line change
@@ -475,7 +475,7 @@ impl Stdio {
opts.read(readable);
opts.write(!readable);
let path = unsafe { CStr::from_ptr(DEV_NULL.as_ptr() as *const _) };
let fd = File::open_c(&path, &opts)?;
let fd = File::open_c(None, &path, &opts)?;
Ok((ChildStdio::Owned(fd.into_inner()), None))
}

6 changes: 3 additions & 3 deletions src/tools/miri/tests/fail/shims/fs/isolated_file.stderr
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
error: unsupported operation: `open` not available when isolation is enabled
--> RUSTLIB/std/src/sys/PLATFORM/fs.rs:LL:CC
|
LL | let fd = cvt_r(|| unsafe { open64(path.as_ptr(), flags, opts.mode as c_int) })?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `open` not available when isolation is enabled
LL | cvt_r(|| unsafe { open64(path.as_ptr(), flags, opts.mode as c_int) })?
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `open` not available when isolation is enabled
|
= help: pass the flag `-Zmiri-disable-isolation` to disable isolation;
= help: or pass `-Zmiri-isolation-error=warn` to configure Miri to return an error code from isolated operations (if supported for that operation) and continue with a warning
@@ -12,7 +12,7 @@ LL | let fd = cvt_r(|| unsafe { open64(path.as_ptr(), flags, opts.mode a
= note: inside `std::sys::PLATFORM::fs::File::open_c` at RUSTLIB/std/src/sys/PLATFORM/fs.rs:LL:CC
= note: inside closure at RUSTLIB/std/src/sys/PLATFORM/fs.rs:LL:CC
= note: inside `std::sys::PLATFORM::small_c_string::run_with_cstr::<std::sys::PLATFORM::fs::File, [closure@std::sys::PLATFORM::fs::File::open::{closure#0}]>` at RUSTLIB/std/src/sys/PLATFORM/small_c_string.rs:LL:CC
= note: inside `std::sys::PLATFORM::small_c_string::run_path_with_cstr::<std::sys::PLATFORM::fs::File, [closure@std::sys::PLATFORM::fs::File::open::{closure#0}]>` at RUSTLIB/std/src/sys/PLATFORM/small_c_string.rs:LL:CC
= note: inside `std::sys::PLATFORM::small_c_string::run_path_with_cstr::<std::sys::PLATFORM::fs::File, [closure@std::sys::PLATFORM::fs::File::open::{closure#0}], std::path::Path>` at RUSTLIB/std/src/sys/PLATFORM/small_c_string.rs:LL:CC
= note: inside `std::sys::PLATFORM::fs::File::open` at RUSTLIB/std/src/sys/PLATFORM/fs.rs:LL:CC
= note: inside `std::fs::OpenOptions::_open` at RUSTLIB/std/src/fs.rs:LL:CC
= note: inside `std::fs::OpenOptions::open::<&std::path::Path>` at RUSTLIB/std/src/fs.rs:LL:CC