Improvement on KVIKIO_COMPAT_MODE

[kingcrimsontianyu]

Currently, there are 3 possibilities with the environment variable KVIKIO_COMPAT_MODE:

• If it is unset, KvikIO performs a system config check [*] for GDS. If (1) the check passes and (2) if the file can be opened with O_DIRECT flag, GDS is used; otherwise, POSIX I/O is used.
• If it is set to true, KvikIO compatibility mode is enabled, which enforces POSIX I/O, and avoids the system config check.
• It it is set to false, KvikIO enforces GDS I/O without the system config check, and errors out on GDS operations on an incompatible config. The only exception is that if the file cannot be opened with the O_DIRECT flag, KvikIO compatibility mode will be enabled.

[*] The system config check consists of 3 parts:

• Whether cuFile library can be correctly loaded
• Whether /run/udev is readable
• Whether the OS is not an WSL

Changes to the above design may be needed given the fact that:

• CUDA 12.2 (GDS version 1.7) introduces support for non O_DIRECT. Therefore for CUDA 12.2+, when KVIKIO_COMPAT_MODE=OFF, GDS I/O path should still be taken even if the file cannot be opened with O_DIRECT. Likewise, when the env var is unset, a failure with O_DIRECT should not result in the POSIX I/O fallback.
• KVIKIO_COMPAT_MODE=OFF expresses the users intention to choose GDS. So the O_DIRECT check can be removed.
• It is necessary to have a separate option equivalent to the case where the env var is unset. It has been proposed that this option be named "ALLOW".