lio_listio API needs a rethink

[asomers]

The current API for lio_listio isn't very good. It needs a major rethink. In fact, of the three different use cases shown in the docs, I think that perhaps two different APIs should be used. The current API's problems are:

• It requires an allocation. (It actually doesn't allocate right now, but that's wrong. See aio_suspend passes invalid pointers to libc #1980).
• It requires the arguments to be supplied as an array, rather than an iterator. If it's going to require an allocation, then it shouldn't insist that the arguments already be in an array.
• It leaves EIO, EINTR, and EAGAIN error handling up to the user, and those error types can be quite difficult to handle.

The best thing about the current API is that it allows the user to freely mix write, read, writev, and readv operations. It will be difficult to preserve that property while also eliminating the need for allocation. Ideally, the API should:

• Allow mixing read, readv, write, and writev operations.
• Not internally allocate, if possible.
• Accept an iterator, if allocation is unavoidable.
• Help the user deal with EINTR, EIO, and EAGAIN errors, if it can be done without additional runtime cost.

Here's one idea: Create an LioListio trait with a submit method, and implement it on different types:

trait LioListio {
    // submit all contained operations with lio_listio
    fn submit(&mut self, mode: LioMode, sigev_notify: SigevNotify) -> Result<()>;
}
// When operating on a homogenous array, no internal allocation will be necessary
// Also implement for AioWritev, AioRead, etc.
impl LioListio for &mut [Pin<Box<AioWrite>>] {...}
// Also, implement for unboxed types like this:
impl LioListio for &mut [Pin<&mut AioWrite>] {...}
// Maybe even for pinned vecs, using the `pinvec` crate:
impl LioListio for &mut [pinvec::vec::Pinvec<AioWrite>] { ...  }

// For heterogenous types, allocation will be necessary.  Here's one option:
struct LioList(Vec<...>);
impl LioList {
    fn with_capacity() -> Self {...}
    fn add<T: AsMut<Pin<libc::aiocb>>>(&mut self, t: &mut T) {...}
}
impl LioListio for LioList {...}