Tracking issue for the GlobalAlloc trait and related APIs

[SimonSapin]

PR #49669 adds a GlobalAlloc trait, separate from Alloc. This issue track the stabilization of this trait and some related APIs, to provide the ability to change the global allocator, as well as allocating memory without abusing Vec::with_capacity + mem::forget.

Defined in or reexported from the std::alloc module:

Update: below is the API proposed when this issue was first opened. The one being stabilized is at #49668 (comment).

/// #[global_allocator] can only be applied to a `static` item that implements this trait
pub unsafe trait GlobalAlloc {
    unsafe fn alloc(&self, layout: Layout) -> *mut Opaque;
    unsafe fn dealloc(&self, ptr: *mut Opaque, layout: Layout);

    unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut Opaque { 
        // Default impl: self.alloc() and ptr::write_bytes()
    }
    unsafe fn realloc(&self, ptr: *mut Opaque, layout: Layout, new_size: usize) -> *mut Opaque {
        // Default impl: self.alloc() and ptr::copy_nonoverlapping() and self.dealloc()
    }
    
    // More methods with default impls may be added in the future
}

extern {
    pub type Opaque;
}

#[derive(Debug, Copy, Clone, PartialEq, Eq)]
pub struct Layout { /* private */ }

impl Layout {
    pub fn from_size_align(size: usize: align: usize) -> Result<Self, LayoutError> {…}
    pub unsafe fn from_size_align_unchecked(size: usize: align: usize) -> Self {…}
    pub fn size(&self) -> usize {…}
    pub fn align(&self) -> usize {…}
    pub fn new<T>() -> Self {…}
    pub fn for_value<T: ?Sized>(t: &T) -> Self {…}
}

#[derive(Copy, Clone, Debug)]
pub struct LayoutError { /* private */ }

/// Forwards method calls to the `static` item registered
/// with `#[global_allocator]` if any, or the operating system’s default.
pub struct Global;

/// The operating system’s default allocator.
pub struct System;

impl GlobalAlloc for Global {…}
impl GlobalAlloc for System {…}

CC @rust-lang/libs, @glandium

Unresolved questions or otherwise blocking

• Wait for extern types Tracking issue for RFC 1861: Extern types #43467 to be stable in the language before stabilazing one in the standard library.
• Name of the Global type. GlobalAllocator?
• Name of the Void type. Lower-case void would match C (our *mut void is very close to C’s void* type), but violates the convension of camel-case for non-primitive types. But Void in camel-case is already the name of an empty enum (not an extern type like here) in a popular crate. (An extern type is desirable so that <*mut _>::offset cannot be called without first casting to another pointer type.) Maybe call it Opaque? Unknown? Renamed to Opaque.
  • Rename again to something else?
• Taking Layout by reference, or making it Copy Alloc methods should take layout by reference #48458. Copy: Implement Copy for std::alloc::Layout #50109
• GlobalAlloc::owns: Alloc: Add owns method #44302 proposes making it required for allocators to be able to tell if it “owns” a given pointer. Not to be required by this trait since glibc and Windows do not support this.
• Settle semantics of layout "fit" and other safety conditions. Without an usable_size method (for now), the layout passed to dealloc must be the same as was passed to alloc. (Same as with Alloc::usable_size’s default impl returning (layout.size(), layout.size()).)
• An oom method is part of GlobalAlloc so that implementation-specific printing to stderr does not need to be part of liballoc and instead goes through #[global_allocator], but this does not really belong in the GlobalAlloc trait. Should another mechanism like #[global_allocator] be added to have pluggable OOM handling? Replace {Alloc,GlobalAlloc}::oom with a lang item. #50144

Not proposed (yet) for stabilization

• The Alloc trait
• The rest of Layout methods
• The AllocErr type

We’re not ready to settle on a design for collections generic over the allocation type. Discussion of this use case should continue on the tracking issue for the Alloc trait: #32838.

[euclio]

I was initially confused by the name Void. I confused it with the aforementioned empty enum. Bikeshed: I wonder if it could be called Allocation or Memory?

[glandium]

Relevant discussions pre-PR:

• about Void:
  0497b5e#r28405553
• about Global:
  c32a3de#commitcomment-28405600

[glandium]

But I don’t know if any platform’s system allocator (like libc’s malloc) supports this, let alone all of them.

The OSX system allocator supports that. Jemalloc does too, optionally. Glibc malloc and Windows heap allocator don't.

[Amanieu]

Taking Layout by reference, or making it Copy #48458

The fact that realloc now takes the new size as usize rather than a Layout strongly points towards just making Layout implement Copy and treating it as a simple (length, alignment) pair.

[alexreg]

Has the Heap type been renamed to Global?

[SimonSapin]

@alexreg Yes, as described in the PR message of #49669.

[RalfJung]

Naming bikesheds:

Given that the attribute is global_allocator, should the trait be called GlobalAllocator instead? alloc vs. allocator is inconsistent.

Also, I am not very happy with Void. void is probably the most misunderstood type in C. It is often described as "empty", but it actually is a unit type (), not the empty never type !. void* is NOT just a pointer to () though, but an independent concept. I'd rather we do not step into the middle of this confusion.
From the alternatives proposed above, I prefer Unknown. Some more random suggestions: Blob, Extern, Data. I think from all of these, my personal favorite is Blob, which I have often seen used as a term for "not further specified binary data". But really, any of these is strictly better than Void IMHO.

[SimonSapin]

If GlobalAlloc is renamed to GlobalAllocator, should Alloc be Allocator? And the alloc method allocate? (And similarly other methods.)

Agreed that Void or void is probably best avoided, though I’m not sure what to pick instead.