Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependabot times out because individual gem updates take too long #11294

Open
1 task done
fragkakis opened this issue Jan 14, 2025 · 0 comments
Open
1 task done

Dependabot times out because individual gem updates take too long #11294

fragkakis opened this issue Jan 14, 2025 · 0 comments
Labels
L: dart:pub Dart packages via pub L: ruby:bundler RubyGems via bundler T: bug 🐞 Something isn't working

Comments

@fragkakis
Copy link

fragkakis commented Jan 14, 2025

Is there an existing issue for this?

  • I have searched the existing issues

Package ecosystem

Bundler

Package manager version

2.4.10

Language version

Ruby 3.3.5

Manifest location and content before the Dependabot update

Below is a redacted version of our gemfile. There are 151 ~> gem definitions (which in my understanding are subject to update by dependabot):

source "https://rubygems.org"
ruby "3.3.5"

# Bundler
if Gem::Version.new(Bundler::VERSION) < Gem::Version.new("2.4.10")
  abort "Bundler version 2.4.10 is required! (found #{Bundler::VERSION})"
end

# Rails
gem "rails", "~> 7.1.4"

# Application server
gem "puma", "~> 6.4.3", require: false
gem "puma_worker_killer", "~> 0.3.1", require: false
gem "net-smtp", "~> 0.3.3", require: false

...

source "https://rubygems.pkg.github.com/FooRegistry" do
  gem "bar-sdk", "~> 4.2.0"
  gem "bar-token-verifier", "~> 0.5.7"
end

...

# Gems for development and testing
group :development, :test, :integration do
  gem "deprecation_toolkit", "~> 2.0.3", require: false
  ...
end

group :development, :test do
  gem "rails-controller-testing", "~> 1.0.5"
  ...
end

group :development do
  gem "lefthook"
  ...
end

group :test do
  gem "mocha", "~> 2.6.1"
  ...
end

group :test_contract do
  gem "pact", "~> 1.59.0"
  ...
end

group :production do
  gem "actionpack-cloudflare", "~> 1.1.0"
end

dependabot.yml content

# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

version: 2
registries:
  FooRegistry:
    type: rubygems-server
    url: https://rubygems.pkg.github.com/FooRegistry
    token: ${{ secrets.GH_PACKAGES_RO }}
updates:
  - package-ecosystem: "bundler" # See documentation for possible values
    insecure-external-code-execution: "allow" # for external repository to work
    directory: "/" # Location of package manifests
    schedule:
      interval: daily
    registries:
      - FooRegistry
    open-pull-requests-limit: 3
    versioning-strategy: lockfile-only
    allow:
      - dependency-type: direct
    ignore:
      - dependency-name: "jquery-ui-rails"
    labels:
      - "dependencies"
      - "ruby"
    groups:
      rubocop:
        patterns:
          - "rubocop*"

Updated dependency

No response

What you expected to see, versus what you actually saw

We are using dependabot in Github with the Github-hosted runners.

We expect dependabot to run and open 3 PRs, as per the configuration files.

Instead, dependabot times out after 55 minutes (that seems to be a hardcoded time limit). In my understanding, this is because the checks for individual gems take too long (> 1 min each, which adds up to > 55 mins). In the logs section, there is an excerpt showing the update for net-smtp, but the same happens for all gems where there is an update available.

In my understanding, this command took too long:

{"BUNDLE_PATH"=>"/home/dependabot/dependabot-updater/tmp/20250110-1258-szlcms/.bundle", "GEM_HOME"=>"/opt/bundler/v2/.bundle"} ruby /opt/bundler/v2/run.rb

The command takes 63.11 seconds, whereas a local bundle update net-smtp only takes 6 seconds.

There is also another thing that looks wrong: Despite the fact that the FooRegistry registry is only used in for 2 specific gems, when trying to update the net-smtp gem (which lives in the public rubygems.org repo), there is a call to the FooRegistry registry too. I don't think this is the problem, just pointing it out too. (UPDATE: Possibly #8554)

Native package manager behavior

A local bundle update net-smtp only takes 6 seconds.

Images of the diff or a link to the PR, issue, or logs

updater | 2025/01/10 09:37:10 INFO <job_946026945> Checking if net-smtp 0.3.4 needs updating
updater | 2025/01/10 09:37:10 INFO <job_946026945> Started process PID: 2397 with command: {"BUNDLE_PATH"=>"/home/dependabot/dependabot-updater/tmp/20250110-1258-szlcms/.bundle", "GEM_HOME"=>"/opt/bundler/v2/.bundle"} ruby /opt/bundler/v2/run.rb
updater | 2025/01/10 09:37:10 INFO <job_946026945> Process PID: 2397 completed with status: pid 2397 exit 0
updater | 2025/01/10 09:37:10 INFO <job_946026945> Total execution time: 0.23 seconds
  proxy | 2025/01/10 09:37:10 [688] GET https://rubygems.org:443/api/v1/versions/net-smtp.json
  proxy | 2025/01/10 09:37:10 [688] 200 https://rubygems.org:443/api/v1/versions/net-smtp.json
updater | 2025/01/10 09:37:10 INFO <job_946026945> Latest version is 0.5.0
updater | 2025/01/10 09:37:10 INFO <job_946026945> Started process PID: 2409 with command: {} git config --global credential.helper '!/home/dependabot/common/lib/dependabot/../../bin/git-credential-store-immutable --file /home/dependabot/dependabot-updater/git.store' {}
updater | 2025/01/10 09:37:10 INFO <job_946026945> Process PID: 2409 completed with status: pid 2409 exit 0
updater | 2025/01/10 09:37:10 INFO <job_946026945> Total execution time: 0.01 seconds
updater | 2025/01/10 09:37:10 INFO <job_946026945> Started process PID: 2416 with command: {} git config --global --replace-all url.https://github.com/.insteadOf ssh://[email protected]/ {}
updater | 2025/01/10 09:37:10 INFO <job_946026945> Process PID: 2416 completed with status: pid 2416 exit 0
2025/01/10 09:37:10 INFO <job_946026945> Total execution time: 0.01 seconds
updater | 2025/01/10 09:37:10 INFO <job_946026945> Started process PID: 2422 with command: {} git config --global --add url.https://github.com/.insteadOf ssh://[email protected]: {}
updater | 2025/01/10 09:37:10 INFO <job_946026945> Process PID: 2422 completed with status: pid 2422 exit 0
updater | 2025/01/10 09:37:10 INFO <job_946026945> Total execution time: 0.01 seconds
updater | 2025/01/10 09:37:10 INFO <job_946026945> Started process PID: 2428 with command: {} git config --global --add url.https://github.com/.insteadOf [email protected]: {}
updater | 2025/01/10 09:37:10 INFO <job_946026945> Process PID: 2428 completed with status: pid 2428 exit 0
2025/01/10 09:37:10 INFO <job_946026945> Total execution time: 0.01 seconds
updater | 2025/01/10 09:37:10 INFO <job_946026945> Started process PID: 2434 with command: {} git config --global --add url.https://github.com/.insteadOf [email protected]/ {}
updater | 2025/01/10 09:37:10 INFO <job_946026945> Process PID: 2434 completed with status: pid 2434 exit 0
updater | 2025/01/10 09:37:10 INFO <job_946026945> Total execution time: 0.01 seconds
updater | 2025/01/10 09:37:10 INFO <job_946026945> Started process PID: 2440 with command: {} git config --global --add url.https://github.com/.insteadOf git://github.com/ {}
updater | 2025/01/10 09:37:10 INFO <job_946026945> Process PID: 2440 completed with status: pid 2440 exit 0
updater | 2025/01/10 09:37:10 INFO <job_946026945> Total execution time: 0.01 seconds
updater | 2025/01/10 09:37:10 INFO <job_946026945> Started process PID: 2446 with command: {} git reset HEAD --hard {}
updater | 2025/01/10 09:37:10 INFO <job_946026945> Process PID: 2446 completed with status: pid 2446 exit 0
updater | 2025/01/10 09:37:10 INFO <job_946026945> Total execution time: 0.16 seconds
updater | 2025/01/10 09:37:10 INFO <job_946026945> Started process PID: 2452 with command: {} git clean -fx {}
updater | 2025/01/10 09:37:10 INFO <job_946026945> Process PID: 2452 completed with status: pid 2452 exit 0
updater | 2025/01/10 09:37:10 INFO <job_946026945> Total execution time: 0.07 seconds
updater | 2025/01/10 09:37:10 INFO <job_946026945> Started process PID: 2458 with command: {"BUNDLE_PATH"=>"/home/dependabot/dependabot-updater/tmp/20250110-1258-szlcms/.bundle", "GEM_HOME"=>"/opt/bundler/v2/.bundle"} ruby /opt/bundler/v2/run.rb
  proxy | 2025/01/10 09:37:12 [690] GET https://rubygems.pkg.github.com:443/FooRegistry/versions
  proxy | 2025/01/10 09:37:12 [690] 304 https://rubygems.pkg.github.com:443/FooRegistry/versions
  proxy | 2025/01/10 09:37:13 [692] GET https://index.rubygems.org:443/versions
2025/01/10 09:37:13 [692] 304 https://index.rubygems.org:443/versions
  proxy | 2025/01/10 09:38:03 Posting metrics to remote API endpoint
  proxy | 2025/01/10 09:38:03 Successfully posted metrics data via api client
updater | 2025/01/10 09:38:13 INFO <job_946026945> Process PID: 2458 completed with status: pid 2458 exit 0
updater | 2025/01/10 09:38:13 INFO <job_946026945> Total execution time: 63.11 seconds
updater | 2025/01/10 09:38:13 INFO <job_946026945> Requirements to unlock update_not_possible
2025/01/10 09:38:13 INFO <job_946026945> Requirements update strategy lockfile_only
2025/01/10 09:38:13 INFO <job_946026945> No update possible for net-smtp 0.3.4
  proxy | 2025/01/10 09:38:13 [694] POST /update_jobs/946026945/record_ecosystem_meta
  proxy | 2025/01/10 09:38:13 [694] 204 /update_jobs/946026945/record_ecosystem_meta

Smallest manifest that reproduces the issue

No response

@fragkakis fragkakis added the T: bug 🐞 Something isn't working label Jan 14, 2025
@github-actions github-actions bot added L: dart:pub Dart packages via pub L: ruby:bundler RubyGems via bundler labels Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
L: dart:pub Dart packages via pub L: ruby:bundler RubyGems via bundler T: bug 🐞 Something isn't working
Projects
Status: No status
Development

No branches or pull requests

1 participant