Add Initial Java Support for GDS to KvikIO #396

aslobodaNV · 2024-06-25T20:41:43Z

This PR is intended to add initial support for Java binding to GDS as part of the KvikIO library. In this PR are the minimal set of bindings required to support synchronous read and write IO operations via GDS as well as a single example to demonstrate how the bindings can be used alongside other CUDA libraries, such as JCuda. Full support for the GDS CuFile API, including batch and asynchronous IO, has not yet been implemented and more sophisticated error/exception handling is not yet in place. There is a README located within kvikio/java detailing how this new functionality can be compiled and built locally, along with detailed instructions on how to run the included usage example.

copy-pr-bot · 2024-06-25T20:41:47Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

…xample

wence- · 2024-06-26T09:03:58Z

java/README.md

+Java Bindings
+
+Summary
+These Java KvikIO bindings for GDS currently support only synchronous read and write IO operations using the underlying CuFile API. Support for batch IO and asynchronous operations are not yet supported.
+
+Dependencies
+The Java KvikIO bindings have been developed to work on Linux based systems and require CUDA to be installed and for GDS to be properly enabled. Instructions for how to install and enable GDS can be found on NVIDIA's website. To compile the shared library it is also necessary to have a JDK installed. To run the included example, it is also necessary to install JCuda as it is used to handle memory allocations and the transfer of data between host and GPU memory. JCuda jar files supporting CUDA 12.x can be found here:
+https://repo1.maven.org/maven2/org/jcuda/jcuda/12.0.0/jcuda-12.0.0.jar
+https://repo1.maven.org/maven2/org/jcuda/jcuda-natives/12.0.0/jcuda-natives-12.0.0.jar
+
+Compilation
+To recompile the .so file for your local system run the following command. Note: Update the command to reflect the directory where you have installed CUDA and your JDK.
+


Let's use markdown formatting here, so that this will look a bit better when rendered.

Suggested change

Java Bindings

Summary

These Java KvikIO bindings for GDS currently support only synchronous read and write IO operations using the underlying CuFile API. Support for batch IO and asynchronous operations are not yet supported.

Dependencies

The Java KvikIO bindings have been developed to work on Linux based systems and require CUDA to be installed and for GDS to be properly enabled. Instructions for how to install and enable GDS can be found on NVIDIA's website. To compile the shared library it is also necessary to have a JDK installed. To run the included example, it is also necessary to install JCuda as it is used to handle memory allocations and the transfer of data between host and GPU memory. JCuda jar files supporting CUDA 12.x can be found here:

https://repo1.maven.org/maven2/org/jcuda/jcuda/12.0.0/jcuda-12.0.0.jar

https://repo1.maven.org/maven2/org/jcuda/jcuda-natives/12.0.0/jcuda-natives-12.0.0.jar

Compilation

To recompile the .so file for your local system run the following command. Note: Update the command to reflect the directory where you have installed CUDA and your JDK.

# Java Bindings

## Summary

These Java KvikIO bindings for GDS currently support only synchronous read and write IO operations using the underlying CuFile API. Support for batch IO and asynchronous operations are not yet supported.

## Dependencies

The Java KvikIO bindings have been developed to work on Linux based systems and require CUDA to be installed and for GDS to be properly enabled. Instructions for how to install and enable GDS can be found on NVIDIA's website. To compile the shared library it is also necessary to have a JDK installed. To run the included example, it is also necessary to install JCuda as it is used to handle memory allocations and the transfer of data between host and GPU memory. JCuda jar files supporting CUDA 12.x can be found here:

https://repo1.maven.org/maven2/org/jcuda/jcuda/12.0.0/jcuda-12.0.0.jar

https://repo1.maven.org/maven2/org/jcuda/jcuda-natives/12.0.0/jcuda-natives-12.0.0.jar

## Compilation

To recompile the .so file for your local system run the following command. Note: Update the command to reflect the directory where you have installed CUDA and your JDK.

(See https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax for further syntax).

wence- · 2024-06-26T09:04:59Z

java/README.md

+https://repo1.maven.org/maven2/org/jcuda/jcuda/12.0.0/jcuda-12.0.0.jar
+https://repo1.maven.org/maven2/org/jcuda/jcuda-natives/12.0.0/jcuda-natives-12.0.0.jar


suggestion: Prefer not to link to explicit files since this can easily go out of date without us remembering to update things. For preference, can we link to an appropriate installation document for JCuda?

Thanks for the suggestion, I can link to the installation instructions as well but found that the installation instructions themselves are a bit unclear/potentially out of date. I may keep these links but just add a note to confirm they are still up to date and reference the installation instructions.

wence- · 2024-06-26T09:10:49Z

java/src/main/native/src/CuFileJni.cpp

I am a bit confused by these bindings. They look to be bindings to the cufile API, and not the kvikio API. Is that deliberate? It seems like if so one will have to replicate a lot of the implementation that already exists in kvikio here.

You are correct, these are directly bound to the underlying cufile API. In the initial discussions with the rest of the GDS team there was some concern about potential performance overhead of Java bindings -> C++ bindings -> C API. I have not investigated the overhead yet, but definitely see the benefit you are suggesting to sharing one underlying set of bindings. If you don't mind, I will add an issue to my backlog to investigate the potential performance differences and, if they are minimal, update these to point to the C++ bindings at a later date.

Sounds good, let's continue with raw java bindings for now.

Side note, I would be very surprised if the basic C++ API introduce any significant overhead. E.g. the overhead of calling read() vs. calling cuFileRead() is tiny.

madsbk · 2024-06-27T06:58:04Z

/ok to test

bdice

A few high level comments before digging too deep into this code:

I have a weak preference to wrap the C++ bindings rather than re-implement kvikio in Java. This will probably be easier than reimplementing the code as new features are introduced to kvikio, and gives more confidence in testing (bindings are probably less bug-prone than a full reimplementation).
This will need to pass style checks. See the cuDF guide on Code Formatting. The same guidance applies to using pre-commit with kvikio development.
This will need a build system, probably Maven. Long commands in the README that explain how to supply all the dependencies are not a good solution if those dependencies can be fetched and built automatically. See how cuDF's Java bindings use Maven, for instance. https://github.com/rapidsai/cudf/tree/branch-24.08/java
- The end result should be that commands like mvn clean install work properly.
This will need CI builds and tests. Review the following resources on cuDF's Java CI and try to duplicate them:
- GitHub Action Job for PRs
- GitHub Action Job for Nightly Tests
- Java build/test script
  - This uses conda to supply the libcudf package, and builds the Java bindings from that. If you use direct bindings and do not depend on the kvikio C++ library, then you can do something simpler that only requires installing cuFile from conda.
- Java dependencies.yaml file key
  - This file key gives a list of what dependency lists to include. This file key is referenced by the CI build/test script. This file key should include the dependency list cuda, so you get the cufile package. See the rapids-dependency-file-generator README for more information.
- Java dependencies.yaml list
  - This dependency list should probably include maven and openjdk. Not sure what else.

Give this a start and please feel free to schedule time with me to discuss anything that is unclear!

jakirkham · 2024-07-24T20:15:32Z

Is this still planned for 24.08 or should we move to 24.10?

aslobodaNV · 2024-07-24T20:20:48Z

I will not likely finish the updates here for a couple weeks due to other priorities, so this should be moved to 24.10

jakirkham · 2024-07-24T20:28:01Z

Ok thanks Alex! 🙏

Have moved to 24.10

…runs

bdice

Quick first pass of review -- mostly focused on packaging and CI. Give my suggestions a try and I'll comment /ok to test to run your next commits on CI.

.github/workflows/build.yaml

.github/workflows/pr.yaml

.github/workflows/test.yaml

bdice · 2024-09-10T16:51:45Z

java/README.md

+#### Run example
+
+    cd kvikio/java/
+    java -cp target/cufile-24.10.0-SNAPSHOT.jar:$HOME/.m2/repository/org/jcuda/jcuda/12.0.0/jcuda-12.0.0.jar:$HOME/.m2/repository/org/jcuda/jcuda-natives/12.0.0/jcuda-natives-12.0.0.jar -Djava.library.path=./target bindings.kvikio.example.Main


cufile-24.10.0-SNAPSHOT.jar

cuFile is not versioned like RAPIDS. Is this correct?

Regarding the version, the jar that is generated is not aware of what version of libcufile was used to generate it... that just depends on what version of the cuda toolkit the host machine has installed. It appears I can't make the jar not have a version number at all, so I'm inclined to have the version represent the version of kvikio. If you think there's a way I can inject the version of libcufile into maven at runtime I am open to that, but I have not found any information on how that would be done so far.

java/src/main/java/bindings/kvikio/cufile/CuFileDriver.java

bdice · 2024-10-01T22:36:36Z

/ok to test

bdice · 2024-10-01T22:48:36Z

/ok to test

bdice · 2024-10-01T23:06:31Z

java/pom.xml

+            <configuration>
+              <target>
+                <!-- Compile native code using nvcc -->
+                <exec executable="/usr/local/cuda/bin/nvcc">


@aslobodaNV The first real failure: this path is not the correct path to nvcc in the CI environment, which installed cuda-nvcc with conda. Can you just call nvcc?

Suggested change

<exec executable="/usr/local/cuda/bin/nvcc">

<exec executable="nvcc">

Error log here:

Error: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:3.0.0:run (compile-native-code) on project cufile: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "/usr/local/cuda/bin/nvcc" (in directory "/__w/kvikio/kvikio/java"): error=2, No such file or directory Error: around Ant part ...<exec executable="/usr/local/cuda/bin/nvcc">... @ 4:49 in /__w/kvikio/kvikio/java/target/antrun/build-main.xml

Similarly, the include paths should be located in the conda environment. The CUDA packages use a special "targets" layout. For example, libcufile-dev will install cufile.h into ${CONDA_PREFIX}/targets/x86_64-linux/include/cufile.h.

The openjdk package installs some headers into ${CONDA_PREFIX}/include and some into ${CONDA_PREFIX}/include/linux.

CMake knows how to detect and parse this layout of the CUDA Toolkit but I do not know how Maven or other build systems are supposed to consume it. I would ask the Spark team, perhaps they know: @revans2, do you have insight here?

Totally fine with committing the update to just call nvcc instead of the path, I just didn't happen to have nvcc on my path when I wrote that. Let me look into the include paths and reach out to Bobby if I get stuck.

bdice · 2024-10-01T23:13:57Z

dependencies.yaml

+      - output_types: conda
+        packages:
+          - maven
+          - openjdk=8.*


Is there a reason to use openjdk=8.*? It appears that the latest is openjdk=22.*.

No particular reason, I just happened to have 8 installed and used that. I will update to 22

JDK8 is a very much a least common denominator. It is no longer supported without paying a lot of money to oracle though.

https://www.oracle.com/java/technologies/java-se-support-roadmap.html

and oracle is trying very hard to push people away from older versions of java.

java 11, 17, and 21 are long term support versions that are currently GA.

Java generally has really good backwards compatibility so almost anything compiled with JDK8 can run on anything newer. But newer code is not backwards compatible with older runtimes. But you can ask newer compilers to output binary code that is compatible with older JDK versions. That is what we do with the Spark plugin.

We also support whatever Spark supports for their default binary release. So for older versions of Spark it is JDK8. For some of the latest versions that is JDK17, but because of how we ask the compiler to output older binary versions it really becomes a requirement that they have at least that version of java installed.

I wouldn't update to jdk22 unless you also decide which version of java you want to be minimally compatible with and ask the compiler to output compatible .class files.

Also just another point to think about jcuda still uses openjdk8. This is probably to have as much compatibility as possible. https://github.com/jcuda/jcuda-main/blob/master/BUILDING.md

Thanks for that context, that makes a lot of sense. I have not been following the broader Java versioning ecosystem so was unaware of much of this. I think it is likely we will want to pick one of the versions with long term support, I would guess 17 or 21, but will think on it some more before making an update.

bdice · 2024-10-01T23:17:40Z

java/pom.xml

+            <configuration>
+              <target>
+                <!-- Compile native code using nvcc -->
+                <exec executable="/usr/local/cuda/bin/nvcc">


Similarly, the include paths should be located in the conda environment. The CUDA packages use a special "targets" layout. For example, libcufile-dev will install cufile.h into ${CONDA_PREFIX}/targets/x86_64-linux/include/cufile.h.

The openjdk package installs some headers into ${CONDA_PREFIX}/include and some into ${CONDA_PREFIX}/include/linux.

CMake knows how to detect and parse this layout of the CUDA Toolkit but I do not know how Maven or other build systems are supposed to consume it. I would ask the Spark team, perhaps they know: @revans2, do you have insight here?

Co-authored-by: Bradley Dice <[email protected]>

revans2

Overall it looks good. Just a few nits. Mostly around java convention. I am curious about your plans to distribute binaries. The java code itself is going to require the native library to work, but the native library is something that the user is expected to compile themselves. This is not a great user experience, but it is a real pain to try and distribute native binaries too. If you want/need some help with this there are some people on my team that have spent a lot of time to make this work well, especially for x86 linux.

java/pom.xml

java/src/main/java/bindings/kvikio/cufile/CuFile.java

revans2 · 2024-10-07T15:03:11Z

java/src/main/java/bindings/kvikio/cufile/CuFile.java

+                    driver.close();
+                }));
+                initialized = true;
+            } catch (Throwable t) {


nit: catching Throwable is not ideal. It can include all kinds of unrecoverable errors. At a minimum you probably want to print out why this could not be initialized in the error message. That way you can debug that a dependency was missing/etc.

Would you be able to point me to an example of best practices in this area? Happy to change it further from my latest update.

I marked it as a nit because it is probably fine in this case. In fact when I went back to look at the CUDF code, I did the exact same thing 5 years ago.

https://github.com/rapidsai/cudf/blob/4dbb8a354a9d4f0b4d82a5bf9747409c6304358f/java/src/main/java/ai/rapids/cudf/NativeDepsLoader.java#L74-L83

It is just generally not good practice to catch a Throwable, because Throwable includes all Errors and Errors are generally considered to not be recoverable. But really the main thing here is giving the user enough information that they can debug why it is failing. You are printing the error message, but the full stack trace would be better. That is the main thing that I would suggest you do.

revans2 · 2024-10-07T15:04:23Z

java/src/main/java/bindings/kvikio/cufile/CuFileHandle.java

+
+package bindings.kvikio.cufile;
+
+abstract class CuFileHandle implements AutoCloseable {


It might be nice to have some javadocs here.

I will be sure to get these javadocs that you have requested in before we merge, just wanted to tackle the build issues we were seeing first. Thank you for all your feedback.

revans2 · 2024-10-07T15:05:37Z

java/src/main/java/bindings/kvikio/cufile/CuFileReadHandle.java

+
+package bindings.kvikio.cufile;
+
+public final class CuFileReadHandle extends CuFileHandle {


javadocs would be very nice for this class as it is intended to be used by developers.

revans2 · 2024-10-07T15:06:26Z

java/src/main/java/bindings/kvikio/cufile/CuFileWriteHandle.java

+
+package bindings.kvikio.cufile;
+
+public final class CuFileWriteHandle extends CuFileHandle {


Same here javadocs would be very nice for the developer experience.

revans2 · 2024-10-07T15:14:36Z

I forgot to add that the APIs to take a raw pointer is far from ideal. It might be nice to have a separate library that is compiled against jcuda and the code kvikio jar so that you can hide the scary part of pulling the native address out of a Pointer and provide a cleaner/safer API.

…_java_support

aslobodaNV · 2024-10-11T18:06:39Z

@madsbk or @bdice would one of you be able to try and run this through the build tests? I have updated it to try and make it more agnostic to the build location. It works on my machine, but need to know if there are more build errors remaining to be fixed for the CI/CD machines.

revans2

Generally it looks really good. I am not going to approve it because I don't know enough about cufile to be able to say that this matches all of the requirements. But from a java perspective it looks decent.

revans2 · 2024-10-11T18:16:11Z

dependencies.yaml

+      - output_types: conda
+        packages:
+          - maven
+          - openjdk=8.*


JDK8 is a very much a least common denominator. It is no longer supported without paying a lot of money to oracle though.

https://www.oracle.com/java/technologies/java-se-support-roadmap.html

and oracle is trying very hard to push people away from older versions of java.

java 11, 17, and 21 are long term support versions that are currently GA.

Java generally has really good backwards compatibility so almost anything compiled with JDK8 can run on anything newer. But newer code is not backwards compatible with older runtimes. But you can ask newer compilers to output binary code that is compatible with older JDK versions. That is what we do with the Spark plugin.

We also support whatever Spark supports for their default binary release. So for older versions of Spark it is JDK8. For some of the latest versions that is JDK17, but because of how we ask the compiler to output older binary versions it really becomes a requirement that they have at least that version of java installed.

I wouldn't update to jdk22 unless you also decide which version of java you want to be minimally compatible with and ask the compiler to output compatible .class files.

Also just another point to think about jcuda still uses openjdk8. This is probably to have as much compatibility as possible. https://github.com/jcuda/jcuda-main/blob/master/BUILDING.md

revans2 · 2024-10-11T18:21:58Z

java/pom.xml

+          <version>3.8.0</version>
+          <configuration>
+                <source>21</source>
+                <target>21</target>


This is where you can set what the output target is that you want.

bdice · 2024-10-18T17:49:34Z

java/src/main/native/CMakeLists.txt

@@ -0,0 +1,21 @@
+cmake_minimum_required(VERSION 3.10)


RAPIDS requires a minimum of cmake 3.26.4 right now, I think. Can we set a higher minimum for these bindings? I don't think we will ever test with a CMake version as old as 3.10.

bdice · 2024-10-18T17:49:53Z

/ok to test

bdice · 2024-10-18T18:13:39Z

/ok to test

bdice · 2024-10-18T18:28:52Z

/ok to test

bdice · 2024-10-18T20:31:09Z

/ok to test

revans2

From the java side of things it looks good. I am not an expert on the APIs or all of the use cases, but the plumbing looks decent.

bdice · 2024-10-22T00:27:42Z

@aslobodaNV Looks like there are CI build failures. I'm not sure how to diagnose this.

Error:  Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project cufile: Fatal error compiling: error: invalid target release: 21 -> [Help 1]

java/pom.xml

…_java_support

bdice · 2024-10-22T02:18:56Z

/ok to test

bdice · 2024-11-13T20:25:28Z

/ok to test

Initial commit

83044ff

aslobodaNV marked this pull request as draft June 25, 2024 20:48

aslobodaNV added 2 commits June 25, 2024 22:29

Update documentation to better flash out how to compile and run the e…

62649e6

…xample

code touchups and Readme update

161b260

madsbk added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Jun 26, 2024

wence- reviewed Jun 26, 2024

View reviewed changes

Update README with markdown formatting. Improve instructions and linkage

30aa3dc

bdice requested changes Jun 27, 2024

View reviewed changes

aslobodaNV added 2 commits July 12, 2024 20:33

Add initial maven setup, needs to be debugged

299bdb6

Update maven build to properly generate shared library

1162efc

jakirkham changed the base branch from branch-24.08 to branch-24.10 July 24, 2024 20:27

Merge branch 'branch-24.10' into add_initial_java_support

3f29546

aslobodaNV and others added 5 commits September 4, 2024 10:37

Merge branch 'rapidsai:branch-24.10' into add_initial_java_support

b3dd38f

Fix pre-commit issues

fe91427

move example to be a test, update pom and dependencies to support CI …

4ad08c6

…runs

pre-commit fixes

83875b6

add github workflow items

4120e4c

aslobodaNV marked this pull request as ready for review September 4, 2024 21:21

aslobodaNV requested review from a team as code owners September 4, 2024 21:21

aslobodaNV requested a review from raydouglass September 4, 2024 21:21

bdice reviewed Sep 10, 2024

View reviewed changes

aslobodaNV and others added 2 commits September 18, 2024 19:02

Updating workflows

888d6a7

Merge branch 'branch-24.10' into add_initial_java_support

d337a16

bdice reviewed Oct 1, 2024

View reviewed changes

Update java/pom.xml

fa749a3

Co-authored-by: Bradley Dice <[email protected]>

revans2 reviewed Oct 7, 2024

View reviewed changes

aslobodaNV added 3 commits October 11, 2024 16:17

Update package for java bindings.

222426c

Use cmake instead of explicit nvcc command

c704108

Merge remote-tracking branch 'upstream/branch-24.12' into add_initial…

69b6d39

…_java_support

revans2 reviewed Oct 11, 2024

View reviewed changes

Merge branch 'branch-24.12' into add_initial_java_support

40a588f

bdice reviewed Oct 18, 2024

View reviewed changes

Fix style issues and missing license

fabd3c1

Fix CMakeLists style issues

c29ac7b

aslobodaNV added 2 commits October 18, 2024 19:42

Update dependencies to try and fix container build

157867f

Just add make not ninja

acc6777

revans2 approved these changes Oct 21, 2024

View reviewed changes

bdice reviewed Oct 22, 2024

View reviewed changes

java/pom.xml Outdated Show resolved Hide resolved

aslobodaNV added 2 commits October 22, 2024 01:21

Fix some of the build and container issues, linking issues remain

cfa183a

Merge remote-tracking branch 'upstream/branch-24.12' into add_initial…

2d41f10

…_java_support

Merge branch 'branch-24.12' into add_initial_java_support

72f79ff

		https://repo1.maven.org/maven2/org/jcuda/jcuda/12.0.0/jcuda-12.0.0.jar
		https://repo1.maven.org/maven2/org/jcuda/jcuda-natives/12.0.0/jcuda-natives-12.0.0.jar

	<exec executable="/usr/local/cuda/bin/nvcc">
	<exec executable="nvcc">


		package bindings.kvikio.cufile;

		abstract class CuFileHandle implements AutoCloseable {


		package bindings.kvikio.cufile;

		public final class CuFileReadHandle extends CuFileHandle {


		package bindings.kvikio.cufile;

		public final class CuFileWriteHandle extends CuFileHandle {

Add Initial Java Support for GDS to KvikIO #396

Are you sure you want to change the base?

Add Initial Java Support for GDS to KvikIO #396

Conversation

aslobodaNV commented Jun 25, 2024 • edited Loading

copy-pr-bot bot commented Jun 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

madsbk Jun 27, 2024 • edited Loading

Choose a reason for hiding this comment

madsbk commented Jun 27, 2024

bdice left a comment • edited Loading

Choose a reason for hiding this comment

jakirkham commented Jul 24, 2024

aslobodaNV commented Jul 24, 2024

jakirkham commented Jul 24, 2024

bdice left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bdice commented Oct 1, 2024

bdice commented Oct 1, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

revans2 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

revans2 commented Oct 7, 2024

aslobodaNV commented Oct 11, 2024

revans2 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bdice commented Oct 18, 2024

bdice commented Oct 18, 2024

bdice commented Oct 18, 2024

bdice commented Oct 18, 2024

revans2 left a comment

Choose a reason for hiding this comment

bdice commented Oct 22, 2024

bdice commented Oct 22, 2024

bdice commented Nov 13, 2024

aslobodaNV commented Jun 25, 2024 •

edited

Loading

madsbk Jun 27, 2024 •

edited

Loading

bdice left a comment •

edited

Loading