Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add String sorted topologicalSort #2696

Merged
merged 2 commits into from
Oct 11, 2024

Conversation

kpango
Copy link
Collaborator

@kpango kpango commented Oct 11, 2024

Description

SSIA

Related Issue

Versions

  • Vald Version: v1.7.13
  • Go Version: v1.23.2
  • Rust Version: v1.81.0
  • Docker Version: v27.3.1
  • Kubernetes Version: v1.31.1
  • Helm Version: v3.16.2
  • NGT Version: v2.2.4
  • Faiss Version: v1.9.0

Checklist

Special notes for your reviewer

Summary by CodeRabbit

  • New Features

    • Enhanced development environment setup with the addition of the RUSTUP_HOME environment variable.
    • Improvements to the build efficiency and configuration for various components.
    • New configuration files copied to the final image for specific applications.
  • Bug Fixes

    • Minor formatting adjustments to the ENTRYPOINT directives across multiple Dockerfiles for consistency.
  • Chores

    • Clean-up and formatting updates, including the addition of newlines for better readability in several Dockerfiles.

Copy link
Contributor

coderabbitai bot commented Oct 11, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The changes in this pull request primarily involve modifications to various Dockerfiles across multiple components. These modifications include adjustments to the ENTRYPOINT directive for consistency, the addition of new lines for formatting, and updates to environment variables and installation commands in specific Dockerfiles. No functional changes were made to the commands, and no alterations were made to the declarations of exported or public entities.

Changes

File Path Change Summary
dockers/agent/core/agent/Dockerfile Adjusted ENTRYPOINT for formatting; no functional changes.
dockers/agent/core/faiss/Dockerfile Minor ENTRYPOINT modification; added newline for formatting.
dockers/agent/core/ngt/Dockerfile Modified ENTRYPOINT; added newline to copy configuration file.
dockers/agent/sidecar/Dockerfile Minor ENTRYPOINT modification; no significant changes.
dockers/binfmt/Dockerfile Added newline at end; no content changes.
dockers/buildbase/Dockerfile Added newline at end; no content changes.
dockers/buildkit/Dockerfile Added newline at end; no content changes.
dockers/buildkit/syft/scanner/Dockerfile Added newline at end; no content changes.
dockers/ci/base/Dockerfile Minor ENTRYPOINT modification; no functional changes.
dockers/dev/Dockerfile Added RUSTUP_HOME environment variable; streamlined installation commands.
dockers/discoverer/k8s/Dockerfile Minor ENTRYPOINT modification for consistency; no functional changes.
dockers/gateway/filter/Dockerfile Removed newline at end; no changes to functionality.
dockers/gateway/lb/Dockerfile Minor ENTRYPOINT modification; no functional changes.
dockers/gateway/mirror/Dockerfile Minor ENTRYPOINT modification; structure unchanged.
dockers/index/job/correction/Dockerfile Specified base image, added environment variables, enhanced RUN command, copied configuration file.
dockers/index/job/creation/Dockerfile Minor ENTRYPOINT modification for formatting; no functional changes.
dockers/index/job/readreplica/rotate/Dockerfile Updated ENTRYPOINT for correct file paths; no functional changes.
dockers/index/job/save/Dockerfile Added line to copy configuration file; ENTRYPOINT unchanged.
dockers/index/operator/Dockerfile Minor ENTRYPOINT modification for consistency; no functional changes.
dockers/manager/index/Dockerfile Minor ENTRYPOINT modification for consistency; no functional changes.
dockers/operator/helm/Dockerfile Added several ARG and ENV declarations; updated RUN command; final stage uses distroless image.
dockers/tools/benchmark/job/Dockerfile Added environment variables and modified build process; ENTRYPOINT unchanged.
dockers/tools/benchmark/operator/Dockerfile Removed newline at end; no changes to functionality.
dockers/tools/cli/loadtest/Dockerfile Modified build process and final image configuration; ENTRYPOINT unchanged.

Possibly related PRs

  • Fix format #2534: This PR modifies the dockers/agent/core/agent/Dockerfile by repositioning the RUSTUP_HOME environment variable declaration, which is relevant to the changes made in the main PR regarding the Dockerfile for the agent component.
  • Backport PR #2534 to release/v1.7 for Fix format #2540: Similar to Fix format #2534, this PR also addresses the dockers/agent/core/agent/Dockerfile by reordering environment variable declarations, specifically RUSTUP_HOME, which connects it to the changes in the main PR.

Suggested reviewers

  • kmrmt
  • datelier

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between b958f95 and fbfbf59.

⛔ Files ignored due to path filters (1)
  • hack/docker/gen/main.go is excluded by !**/gen/**
📒 Files selected for processing (24)
  • dockers/agent/core/agent/Dockerfile (1 hunks)
  • dockers/agent/core/faiss/Dockerfile (1 hunks)
  • dockers/agent/core/ngt/Dockerfile (1 hunks)
  • dockers/agent/sidecar/Dockerfile (1 hunks)
  • dockers/binfmt/Dockerfile (1 hunks)
  • dockers/buildbase/Dockerfile (1 hunks)
  • dockers/buildkit/Dockerfile (1 hunks)
  • dockers/buildkit/syft/scanner/Dockerfile (1 hunks)
  • dockers/ci/base/Dockerfile (1 hunks)
  • dockers/dev/Dockerfile (1 hunks)
  • dockers/discoverer/k8s/Dockerfile (1 hunks)
  • dockers/gateway/filter/Dockerfile (1 hunks)
  • dockers/gateway/lb/Dockerfile (1 hunks)
  • dockers/gateway/mirror/Dockerfile (1 hunks)
  • dockers/index/job/correction/Dockerfile (1 hunks)
  • dockers/index/job/creation/Dockerfile (1 hunks)
  • dockers/index/job/readreplica/rotate/Dockerfile (1 hunks)
  • dockers/index/job/save/Dockerfile (1 hunks)
  • dockers/index/operator/Dockerfile (1 hunks)
  • dockers/manager/index/Dockerfile (1 hunks)
  • dockers/operator/helm/Dockerfile (1 hunks)
  • dockers/tools/benchmark/job/Dockerfile (1 hunks)
  • dockers/tools/benchmark/operator/Dockerfile (1 hunks)
  • dockers/tools/cli/loadtest/Dockerfile (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (24)
  • dockers/agent/core/agent/Dockerfile
  • dockers/agent/core/faiss/Dockerfile
  • dockers/agent/core/ngt/Dockerfile
  • dockers/agent/sidecar/Dockerfile
  • dockers/binfmt/Dockerfile
  • dockers/buildbase/Dockerfile
  • dockers/buildkit/Dockerfile
  • dockers/buildkit/syft/scanner/Dockerfile
  • dockers/ci/base/Dockerfile
  • dockers/dev/Dockerfile
  • dockers/discoverer/k8s/Dockerfile
  • dockers/gateway/filter/Dockerfile
  • dockers/gateway/lb/Dockerfile
  • dockers/gateway/mirror/Dockerfile
  • dockers/index/job/correction/Dockerfile
  • dockers/index/job/creation/Dockerfile
  • dockers/index/job/readreplica/rotate/Dockerfile
  • dockers/index/job/save/Dockerfile
  • dockers/index/operator/Dockerfile
  • dockers/manager/index/Dockerfile
  • dockers/operator/helm/Dockerfile
  • dockers/tools/benchmark/job/Dockerfile
  • dockers/tools/benchmark/operator/Dockerfile
  • dockers/tools/cli/loadtest/Dockerfile

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@kpango kpango requested review from a team, kmrmt, vankichi and hlts2 and removed request for a team and vankichi October 11, 2024 02:31
@vdaas-ci
Copy link
Collaborator

[CHATOPS:HELP] ChatOps commands.

  • 🙆‍♀️ /approve - approve
  • 🍱 /format - format codes and add licenses
  • /gen-test - generate test codes
  • 🏷️ /label - add labels
  • 🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

@kpango kpango force-pushed the refactor/hack-docker/add-string-sorted-topologycal-sort branch from f10c93a to b958f95 Compare October 11, 2024 02:35
Copy link

cloudflare-workers-and-pages bot commented Oct 11, 2024

Deploying vald with  Cloudflare Pages  Cloudflare Pages

Latest commit: fbfbf59
Status: ✅  Deploy successful!
Preview URL: https://2d3f7f0d.vald.pages.dev
Branch Preview URL: https://refactor-hack-docker-add-str.vald.pages.dev

View logs

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (3)
dockers/index/job/correction/Dockerfile (1)

Line range hint 20-20: Consider using a stable version for the build base image

The current Dockerfile uses ghcr.io/vdaas/vald/vald-buildbase:nightly as the base image for the build stage. While this ensures you're always using the latest version, it might introduce instability in your builds. Consider using a specific, stable version of the build base image to ensure reproducible builds.

You could modify the FROM instruction to use a specific version tag instead of nightly. For example:

FROM ghcr.io/vdaas/vald/vald-buildbase:v1.x.x AS builder

Replace v1.x.x with the latest stable version of the build base image.

dockers/tools/benchmark/job/Dockerfile (2)

96-96: LGTM: Minor formatting improvement.

The change adds a newline before the ENTRYPOINT instruction, slightly improving readability. While this is a positive change, it's minimal and doesn't affect the functionality of the Dockerfile.

Consider applying similar formatting changes consistently across the entire Dockerfile for better overall readability.


Line range hint 1-96: Overall, the Dockerfile is well-structured and follows best practices.

The Dockerfile uses multi-stage builds, leverages caching, and follows security best practices such as using a distroless base image and running as a non-root user. While the current change is minimal, here are some suggestions for future improvements:

  1. Consider using specific versions for base images instead of nightly to ensure reproducible builds.
  2. Evaluate if all installed packages in the builder stage are necessary; removing unnecessary ones can further reduce the image size.
  3. Consider adding a health check to the final stage to improve container orchestration.
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 3f18ac7 and b958f95.

⛔ Files ignored due to path filters (1)
  • hack/docker/gen/main.go is excluded by !**/gen/**
📒 Files selected for processing (24)
  • dockers/agent/core/agent/Dockerfile (1 hunks)
  • dockers/agent/core/faiss/Dockerfile (1 hunks)
  • dockers/agent/core/ngt/Dockerfile (1 hunks)
  • dockers/agent/sidecar/Dockerfile (1 hunks)
  • dockers/binfmt/Dockerfile (1 hunks)
  • dockers/buildbase/Dockerfile (1 hunks)
  • dockers/buildkit/Dockerfile (1 hunks)
  • dockers/buildkit/syft/scanner/Dockerfile (1 hunks)
  • dockers/ci/base/Dockerfile (1 hunks)
  • dockers/dev/Dockerfile (2 hunks)
  • dockers/discoverer/k8s/Dockerfile (1 hunks)
  • dockers/gateway/filter/Dockerfile (1 hunks)
  • dockers/gateway/lb/Dockerfile (1 hunks)
  • dockers/gateway/mirror/Dockerfile (1 hunks)
  • dockers/index/job/correction/Dockerfile (1 hunks)
  • dockers/index/job/creation/Dockerfile (1 hunks)
  • dockers/index/job/readreplica/rotate/Dockerfile (1 hunks)
  • dockers/index/job/save/Dockerfile (1 hunks)
  • dockers/index/operator/Dockerfile (1 hunks)
  • dockers/manager/index/Dockerfile (1 hunks)
  • dockers/operator/helm/Dockerfile (1 hunks)
  • dockers/tools/benchmark/job/Dockerfile (1 hunks)
  • dockers/tools/benchmark/operator/Dockerfile (1 hunks)
  • dockers/tools/cli/loadtest/Dockerfile (1 hunks)
✅ Files skipped from review due to trivial changes (18)
  • dockers/agent/core/agent/Dockerfile
  • dockers/agent/core/faiss/Dockerfile
  • dockers/agent/core/ngt/Dockerfile
  • dockers/agent/sidecar/Dockerfile
  • dockers/binfmt/Dockerfile
  • dockers/buildbase/Dockerfile
  • dockers/buildkit/Dockerfile
  • dockers/buildkit/syft/scanner/Dockerfile
  • dockers/ci/base/Dockerfile
  • dockers/discoverer/k8s/Dockerfile
  • dockers/gateway/filter/Dockerfile
  • dockers/gateway/lb/Dockerfile
  • dockers/gateway/mirror/Dockerfile
  • dockers/index/job/creation/Dockerfile
  • dockers/index/job/readreplica/rotate/Dockerfile
  • dockers/index/operator/Dockerfile
  • dockers/manager/index/Dockerfile
  • dockers/tools/benchmark/operator/Dockerfile
🧰 Additional context used
🔇 Additional comments (6)
dockers/index/job/correction/Dockerfile (2)

Line range hint 49-86: Excellent use of build optimizations and best practices

The RUN instruction in this Dockerfile demonstrates several good practices:

  1. Use of --mount options for caching APT packages and Go build cache, which can significantly speed up subsequent builds.
  2. Use of set -ex for better error handling and build verbosity.
  3. Configuring APT to keep downloaded packages and avoid installing recommended packages, which helps to keep the image size down.
  4. Cleaning up APT caches after installation to reduce image size.
  5. Setting up locales and timezone information correctly.

These practices contribute to faster, more reliable, and smaller Docker images. Well done!


86-87: Clarify the purpose of the sample configuration file

The Dockerfile copies a sample.yaml file to /etc/server/config.yaml in the final image. While providing a sample configuration can be helpful, it's important to ensure that this doesn't unintentionally become the default configuration in a production environment.

Could you please clarify:

  1. Is this sample configuration intended for production use, or is it just for demonstration purposes?
  2. If it's not for production, how will the actual configuration be provided to the container in a production environment?

Consider adding a comment in the Dockerfile to explain the purpose of this sample configuration file.

dockers/tools/cli/loadtest/Dockerfile (2)

96-96: Excellent change to ENTRYPOINT format

The modification of the ENTRYPOINT directive from shell form to exec form is a positive improvement. This change:

  1. Enhances clarity and readability of the Dockerfile.
  2. Prevents potential issues with signal handling in the container.
  3. Aligns with Docker best practices.
  4. Ensures consistent behavior across different shell environments.

Line range hint 1-96: Overall Dockerfile structure and practices

While only the ENTRYPOINT was changed in this PR, it's worth noting that the overall structure of this Dockerfile follows many best practices:

  1. Uses multi-stage builds to minimize the final image size.
  2. Employs a distroless base image for the final stage, enhancing security.
  3. Sets up proper environment variables and locale settings.
  4. Uses the principle of least privilege by running as a non-root user.
  5. Employs caching mechanisms to speed up builds.

These practices contribute to a more efficient, secure, and maintainable Docker image.

dockers/dev/Dockerfile (2)

48-48: LGTM: Good addition of RUSTUP_HOME environment variable

The addition of RUSTUP_HOME is a good practice. It separates the Rust toolchain directory from the cargo directory, which is beneficial for organization and potential version management. This change aligns well with the existing CARGO_HOME variable and follows Rust installation best practices.


148-148: ⚠️ Potential issue

Consider using a non-root user for enhanced security

While using root as the last USER in a Dockerfile might be necessary for certain development operations, it's generally discouraged for security reasons. Consider creating a non-root user for running the container, and only use root when absolutely necessary.

You could add a new user and switch to it at the end of the Dockerfile:

RUN useradd -m devuser
USER devuser

If root access is required for specific operations, you can use sudo or temporarily switch to root within the container.

dockers/index/job/save/Dockerfile Show resolved Hide resolved
dockers/operator/helm/Dockerfile Show resolved Hide resolved
vankichi
vankichi previously approved these changes Oct 11, 2024
@kpango kpango force-pushed the refactor/hack-docker/add-string-sorted-topologycal-sort branch from b958f95 to d70dc90 Compare October 11, 2024 02:52
deepsource-autofix bot added a commit that referenced this pull request Oct 11, 2024
This commit fixes the style issues introduced in d70dc90 according to the output
from Gofumpt and Prettier.

Details: #2696
@kpango kpango requested a review from vankichi October 11, 2024 02:55
@kpango kpango force-pushed the refactor/hack-docker/add-string-sorted-topologycal-sort branch from 578ac94 to 839536f Compare October 11, 2024 02:55
@github-actions github-actions bot added size/L and removed size/M labels Oct 11, 2024
@kpango kpango merged commit 9394f09 into main Oct 11, 2024
175 of 176 checks passed
@kpango kpango deleted the refactor/hack-docker/add-string-sorted-topologycal-sort branch October 11, 2024 04:16
vdaas-ci pushed a commit that referenced this pull request Oct 11, 2024
kpango added a commit that referenced this pull request Oct 11, 2024
@kpango kpango mentioned this pull request Oct 11, 2024
@coderabbitai coderabbitai bot mentioned this pull request Oct 16, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants