-
Notifications
You must be signed in to change notification settings - Fork 67
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Committing in initial version of code.
- Loading branch information
Showing
958 changed files
with
151,118 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# syntax=docker/dockerfile:1.5 | ||
|
||
ARG BASE | ||
ARG PYTHON_PACKAGE_MANAGER=conda | ||
|
||
FROM ${BASE} as pip-base | ||
|
||
ENV DEFAULT_VIRTUAL_ENV=rapids | ||
|
||
FROM ${BASE} as conda-base | ||
|
||
ENV DEFAULT_CONDA_ENV=rapids | ||
|
||
FROM ${PYTHON_PACKAGE_MANAGER}-base | ||
|
||
ARG CUDA | ||
ENV CUDAARCHS="RAPIDS" | ||
ENV CUDA_VERSION="${CUDA_VERSION:-${CUDA}}" | ||
|
||
ARG PYTHON_PACKAGE_MANAGER | ||
ENV PYTHON_PACKAGE_MANAGER="${PYTHON_PACKAGE_MANAGER}" | ||
|
||
ENV PYTHONSAFEPATH="1" | ||
ENV PYTHONUNBUFFERED="1" | ||
ENV PYTHONDONTWRITEBYTECODE="1" | ||
|
||
ENV SCCACHE_REGION="us-east-2" | ||
ENV SCCACHE_BUCKET="rapids-sccache-devs" | ||
ENV VAULT_HOST="https://vault.ops.k8s.rapids.ai" | ||
ENV HISTFILE="/home/coder/.cache/._bash_history" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,64 @@ | ||
# RAFT Development Containers | ||
|
||
This directory contains [devcontainer configurations](https://containers.dev/implementors/json_reference/) for using VSCode to [develop in a container](https://code.visualstudio.com/docs/devcontainers/containers) via the `Remote Containers` [extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) or [GitHub Codespaces](https://github.com/codespaces). | ||
|
||
This container is a turnkey development environment for building and testing the RAFT C++ and Python libraries. | ||
|
||
## Table of Contents | ||
|
||
* [Prerequisites](#prerequisites) | ||
* [Host bind mounts](#host-bind-mounts) | ||
* [Launch a Dev Container](#launch-a-dev-container) | ||
|
||
## Prerequisites | ||
|
||
* [VSCode](https://code.visualstudio.com/download) | ||
* [VSCode Remote Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) | ||
|
||
## Host bind mounts | ||
|
||
By default, the following directories are bind-mounted into the devcontainer: | ||
|
||
* `${repo}:/home/coder/cuvs` | ||
* `${repo}/../.aws:/home/coder/.aws` | ||
* `${repo}/../.local:/home/coder/.local` | ||
* `${repo}/../.cache:/home/coder/.cache` | ||
* `${repo}/../.conda:/home/coder/.conda` | ||
* `${repo}/../.config:/home/coder/.config` | ||
|
||
This ensures caches, configurations, dependencies, and your commits are persisted on the host across container runs. | ||
|
||
## Launch a Dev Container | ||
|
||
To launch a devcontainer from VSCode, open the RAFT repo and select the "Reopen in Container" button in the bottom right:<br/><img src="https://user-images.githubusercontent.com/178183/221771999-97ab29d5-e718-4e5f-b32f-2cdd51bba25c.png"/> | ||
|
||
Alternatively, open the VSCode command palette (typically `cmd/ctrl + shift + P`) and run the "Rebuild and Reopen in Container" command. | ||
|
||
## Using the devcontainer | ||
|
||
On startup, the devcontainer creates or updates the conda/pip environment using `cuvs/dependencies.yaml`. | ||
|
||
The container includes convenience functions to clean, configure, and build the various RAFT components: | ||
|
||
```shell | ||
$ clean-cuvs-cpp # only cleans the C++ build dir | ||
$ clean-pylibcuvs-python # only cleans the Python build dir | ||
$ clean-cuvs # cleans both C++ and Python build dirs | ||
|
||
$ configure-cuvs-cpp # only configures cuvs C++ lib | ||
|
||
$ build-cuvs-cpp # only builds cuvs C++ lib | ||
$ build-pylibcuvs-python # only builds cuvs Python lib | ||
$ build-cuvs # builds both C++ and Python libs | ||
``` | ||
|
||
* The C++ build script is a small wrapper around `cmake -S ~/cuvs/cpp -B ~/cuvs/cpp/build` and `cmake --build ~/cuvs/cpp/build` | ||
* The Python build script is a small wrapper around `pip install --editable ~/cuvs/cpp` | ||
|
||
Unlike `build.sh`, these convenience scripts *don't* install the libraries after building them. Instead, they automatically inject the correct arguments to build the C++ libraries from source and use their build dirs as package roots: | ||
|
||
```shell | ||
$ cmake -S ~/cuvs/cpp -B ~/cuvs/cpp/build | ||
$ CMAKE_ARGS="-Dcuvs_ROOT=~/cuvs/cpp/build" \ # <-- this argument is automatic | ||
pip install -e ~/cuvs/cpp | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
{ | ||
"build": { | ||
"context": "${localWorkspaceFolder}/.devcontainer", | ||
"dockerfile": "${localWorkspaceFolder}/.devcontainer/Dockerfile", | ||
"args": { | ||
"CUDA": "11.8", | ||
"PYTHON_PACKAGE_MANAGER": "conda", | ||
"BASE": "rapidsai/devcontainers:24.02-cpp-llvm16-cuda11.8-mambaforge-ubuntu22.04" | ||
} | ||
}, | ||
"hostRequirements": {"gpu": "optional"}, | ||
"features": { | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:24.2": {} | ||
}, | ||
"overrideFeatureInstallOrder": [ | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils" | ||
], | ||
"initializeCommand": ["/bin/bash", "-c", "mkdir -m 0755 -p ${localWorkspaceFolder}/../.{aws,cache,config,conda/pkgs,conda/${localWorkspaceFolderBasename}-cuda11.8-envs}"], | ||
"postAttachCommand": ["/bin/bash", "-c", "if [ ${CODESPACES:-false} = 'true' ]; then . devcontainer-utils-post-attach-command; . rapids-post-attach-command; fi"], | ||
"workspaceFolder": "/home/coder", | ||
"workspaceMount": "source=${localWorkspaceFolder},target=/home/coder/cuvs,type=bind,consistency=consistent", | ||
"mounts": [ | ||
"source=${localWorkspaceFolder}/../.aws,target=/home/coder/.aws,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.cache,target=/home/coder/.cache,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.config,target=/home/coder/.config,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.conda/pkgs,target=/home/coder/.conda/pkgs,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.conda/${localWorkspaceFolderBasename}-cuda11.8-envs,target=/home/coder/.conda/envs,type=bind,consistency=consistent" | ||
], | ||
"customizations": { | ||
"vscode": { | ||
"extensions": [ | ||
"ms-python.flake8", | ||
"nvidia.nsight-vscode-edition" | ||
] | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
{ | ||
"build": { | ||
"context": "${localWorkspaceFolder}/.devcontainer", | ||
"dockerfile": "${localWorkspaceFolder}/.devcontainer/Dockerfile", | ||
"args": { | ||
"CUDA": "11.8", | ||
"PYTHON_PACKAGE_MANAGER": "pip", | ||
"BASE": "rapidsai/devcontainers:24.02-cpp-llvm16-cuda11.8-ubuntu22.04" | ||
} | ||
}, | ||
"hostRequirements": {"gpu": "optional"}, | ||
"features": { | ||
"ghcr.io/rapidsai/devcontainers/features/ucx:24.2": {"version": "1.14.1"}, | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:24.2": {} | ||
}, | ||
"overrideFeatureInstallOrder": [ | ||
"ghcr.io/rapidsai/devcontainers/features/ucx", | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils" | ||
], | ||
"initializeCommand": ["/bin/bash", "-c", "mkdir -m 0755 -p ${localWorkspaceFolder}/../.{aws,cache,config/pip,local/share/${localWorkspaceFolderBasename}-cuda11.8-venvs}"], | ||
"postAttachCommand": ["/bin/bash", "-c", "if [ ${CODESPACES:-false} = 'true' ]; then . devcontainer-utils-post-attach-command; . rapids-post-attach-command; fi"], | ||
"workspaceFolder": "/home/coder", | ||
"workspaceMount": "source=${localWorkspaceFolder},target=/home/coder/cuvs,type=bind,consistency=consistent", | ||
"mounts": [ | ||
"source=${localWorkspaceFolder}/../.aws,target=/home/coder/.aws,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.cache,target=/home/coder/.cache,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.config,target=/home/coder/.config,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.local/share/${localWorkspaceFolderBasename}-cuda11.8-venvs,target=/home/coder/.local/share/venvs,type=bind,consistency=consistent" | ||
], | ||
"customizations": { | ||
"vscode": { | ||
"extensions": [ | ||
"ms-python.flake8", | ||
"nvidia.nsight-vscode-edition" | ||
] | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
{ | ||
"build": { | ||
"context": "${localWorkspaceFolder}/.devcontainer", | ||
"dockerfile": "${localWorkspaceFolder}/.devcontainer/Dockerfile", | ||
"args": { | ||
"CUDA": "12.0", | ||
"PYTHON_PACKAGE_MANAGER": "conda", | ||
"BASE": "rapidsai/devcontainers:24.02-cpp-mambaforge-ubuntu22.04" | ||
} | ||
}, | ||
"hostRequirements": {"gpu": "optional"}, | ||
"features": { | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:24.2": {} | ||
}, | ||
"overrideFeatureInstallOrder": [ | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils" | ||
], | ||
"initializeCommand": ["/bin/bash", "-c", "mkdir -m 0755 -p ${localWorkspaceFolder}/../.{aws,cache,config,conda/pkgs,conda/${localWorkspaceFolderBasename}-cuda12.0-envs}"], | ||
"postAttachCommand": ["/bin/bash", "-c", "if [ ${CODESPACES:-false} = 'true' ]; then . devcontainer-utils-post-attach-command; . rapids-post-attach-command; fi"], | ||
"workspaceFolder": "/home/coder", | ||
"workspaceMount": "source=${localWorkspaceFolder},target=/home/coder/cuvs,type=bind,consistency=consistent", | ||
"mounts": [ | ||
"source=${localWorkspaceFolder}/../.aws,target=/home/coder/.aws,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.cache,target=/home/coder/.cache,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.config,target=/home/coder/.config,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.conda/pkgs,target=/home/coder/.conda/pkgs,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.conda/${localWorkspaceFolderBasename}-cuda12.0-envs,target=/home/coder/.conda/envs,type=bind,consistency=consistent" | ||
], | ||
"customizations": { | ||
"vscode": { | ||
"extensions": [ | ||
"ms-python.flake8", | ||
"nvidia.nsight-vscode-edition" | ||
] | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
{ | ||
"build": { | ||
"context": "${localWorkspaceFolder}/.devcontainer", | ||
"dockerfile": "${localWorkspaceFolder}/.devcontainer/Dockerfile", | ||
"args": { | ||
"CUDA": "12.0", | ||
"PYTHON_PACKAGE_MANAGER": "pip", | ||
"BASE": "rapidsai/devcontainers:24.02-cpp-llvm16-cuda12.0-ubuntu22.04" | ||
} | ||
}, | ||
"hostRequirements": {"gpu": "optional"}, | ||
"features": { | ||
"ghcr.io/rapidsai/devcontainers/features/ucx:24.2": {"version": "1.14.1"}, | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:24.2": {} | ||
}, | ||
"overrideFeatureInstallOrder": [ | ||
"ghcr.io/rapidsai/devcontainers/features/ucx", | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils" | ||
], | ||
"initializeCommand": ["/bin/bash", "-c", "mkdir -m 0755 -p ${localWorkspaceFolder}/../.{aws,cache,config/pip,local/share/${localWorkspaceFolderBasename}-cuda12.0-venvs}"], | ||
"postAttachCommand": ["/bin/bash", "-c", "if [ ${CODESPACES:-false} = 'true' ]; then . devcontainer-utils-post-attach-command; . rapids-post-attach-command; fi"], | ||
"workspaceFolder": "/home/coder", | ||
"workspaceMount": "source=${localWorkspaceFolder},target=/home/coder/cuvs,type=bind,consistency=consistent", | ||
"mounts": [ | ||
"source=${localWorkspaceFolder}/../.aws,target=/home/coder/.aws,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.cache,target=/home/coder/.cache,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.config,target=/home/coder/.config,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.local/share/${localWorkspaceFolderBasename}-cuda12.0-venvs,target=/home/coder/.local/share/venvs,type=bind,consistency=consistent" | ||
], | ||
"customizations": { | ||
"vscode": { | ||
"extensions": [ | ||
"ms-python.flake8", | ||
"nvidia.nsight-vscode-edition" | ||
] | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# Copyright (c) 2022-2023, NVIDIA CORPORATION. | ||
|
||
[flake8] | ||
filename = *.py, *.pyx, *.pxd, *.pxi | ||
exclude = __init__.py, *.egg, build, docs, .git | ||
force-check = True | ||
ignore = | ||
# line break before binary operator | ||
W503, | ||
# whitespace before : | ||
E203 | ||
per-file-ignores = | ||
# Rules ignored only in Cython: | ||
# E211: whitespace before '(' (used in multi-line imports) | ||
# E225: Missing whitespace around operators (breaks cython casting syntax like <int>) | ||
# E226: Missing whitespace around arithmetic operators (breaks cython pointer syntax like int*) | ||
# E227: Missing whitespace around bitwise or shift operator (Can also break casting syntax) | ||
# E275: Missing whitespace after keyword (Doesn't work with Cython except?) | ||
# E402: invalid syntax (works for Python, not Cython) | ||
# E999: invalid syntax (works for Python, not Cython) | ||
# W504: line break after binary operator (breaks lines that end with a pointer) | ||
*.pyx: E211, E225, E226, E227, E275, E402, E999, W504 | ||
*.pxd: E211, E225, E226, E227, E275, E402, E999, W504 | ||
*.pxi: E211, E225, E226, E227, E275, E402, E999, W504 |
Validating CODEOWNERS rules …
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
#cpp code owners | ||
cpp/ @rapidsai/cuvs-cpp-codeowners | ||
|
||
#python code owners | ||
python/ @rapidsai/cuvs-python-codeowners | ||
|
||
#cmake code owners | ||
**/CMakeLists.txt @rapidsai/cuvs-cmake-codeowners | ||
**/cmake/ @rapidsai/cuvs-cmake-codeowners | ||
python/setup.py @rapidsai/cuvs-cmake-codeowners | ||
build.sh @rapidsai/cuvs-cmake-codeowners | ||
**/build.sh @rapidsai/cuvs-cmake-codeowners | ||
|
||
#build/ops code owners | ||
.github/ @rapidsai/ops-codeowners | ||
ci/ @rapidsai/ops-codeowners | ||
conda/ @rapidsai/ops-codeowners | ||
**/Dockerfile @rapidsai/ops-codeowners | ||
**/.dockerignore @rapidsai/ops-codeowners | ||
docker/ @rapidsai/ops-codeowners | ||
dependencies.yaml @rapidsai/ops-codeowners |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
--- | ||
name: Bug report | ||
about: Create a bug report to help us improve RAFT | ||
title: "[BUG]" | ||
labels: "? - Needs Triage, bug" | ||
assignees: '' | ||
|
||
--- | ||
|
||
**Describe the bug** | ||
A clear and concise description of what the bug is. | ||
|
||
**Steps/Code to reproduce bug** | ||
Follow this guide http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports to craft a minimal bug report. This helps us reproduce the issue you're having and resolve the issue more quickly. | ||
|
||
**Expected behavior** | ||
A clear and concise description of what you expected to happen. | ||
|
||
**Environment details (please complete the following information):** | ||
- Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)] | ||
- Method of RAFT install: [conda, Docker, or from source] | ||
- If method of install is [Docker], provide `docker pull` & `docker run` commands used | ||
|
||
|
||
**Additional context** | ||
Add any other context about the problem here. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
--- | ||
name: Documentation request | ||
about: Report incorrect or needed documentation | ||
title: "[DOC]" | ||
labels: "? - Needs Triage, doc" | ||
assignees: '' | ||
|
||
--- | ||
|
||
## Report incorrect documentation | ||
|
||
**Location of incorrect documentation** | ||
Provide links and line numbers if applicable. | ||
|
||
**Describe the problems or issues found in the documentation** | ||
A clear and concise description of what you found to be incorrect. | ||
|
||
**Steps taken to verify documentation is incorrect** | ||
List any steps you have taken: | ||
|
||
**Suggested fix for documentation** | ||
Detail proposed changes to fix the documentation if you have any. | ||
|
||
--- | ||
|
||
## Report needed documentation | ||
|
||
**Report needed documentation** | ||
A clear and concise description of what documentation you believe it is needed and why. | ||
|
||
**Describe the documentation you'd like** | ||
A clear and concise description of what you want to happen. | ||
|
||
**Steps taken to search for needed documentation** | ||
List any steps you have taken: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
--- | ||
name: Feature request | ||
about: Suggest an idea for RAFT | ||
title: "[FEA]" | ||
labels: "? - Needs Triage, feature request" | ||
assignees: '' | ||
|
||
--- | ||
|
||
**Is your feature request related to a problem? Please describe.** | ||
A clear and concise description of what the problem is. Ex. I wish I could use RAFT to do [...] | ||
|
||
**Describe the solution you'd like** | ||
A clear and concise description of what you want to happen. | ||
|
||
**Describe alternatives you've considered** | ||
A clear and concise description of any alternative solutions or features you've considered. | ||
|
||
**Additional context** | ||
Add any other context, code examples, or references to existing implementations about the feature request here. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
--- | ||
name: Submit question | ||
about: Ask a general question about RAFT | ||
title: "[QST]" | ||
labels: "? - Needs Triage, question" | ||
assignees: '' | ||
|
||
--- | ||
|
||
**What is your question?** |
Oops, something went wrong.