Add support for completely non-blocking I/O submission

[hmwill]

Even though meant to be non-blocking. io_submit can block in some cases. io_submit supports a flag RWF_NOWAIT that can be used to require the call to be non-blocking. Specifically, this option means:
Don't wait if the I/O will block for operations such as file block allocations, dirty page flush, mutex locks, or a congested block device inside the kernel.

When RMF_NOWAIT is set, I/O requests may fail immediately with EAGAIN.

This issue is about investigating if RWF_NOWAIT can be meaningfully integrated into this API.