Amazon linux support #2
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hello, are you considering this pull request? |
crohr
pushed a commit
that referenced
this pull request
Mar 30, 2020
# This is the 1st commit message:
Failing test for Bundler 2 and Ruby 2.5.5
When you deploy an app with Bundler 2.0.1 in the Gemfile.lock with Bundler 2.0.2 on the system, it works the first time, but on the second deploy it fails:
```
1) Bundler deploys with version 2.x
Failure/Error: app.push!
Hatchet::App::FailedDeploy:
Could not deploy 'hatchet-t-31aafb8954' (default_ruby) using 'Hatchet::GitApp' at path: './repos/rack/default_ruby'
if this was expected add `allow_failure: true` to your deploy hash.
output:
Buildpack: nil
Repo: https://git.heroku.com/hatchet-t-31aafb8954.git
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Ruby app detected
remote: -----> Compiling Ruby/Rack
remote: -----> Using Ruby version: ruby-2.5.5
remote: -----> Installing dependencies using bundler 2.0.2
remote: Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
remote: Activating bundler (2.0.1) failed:
remote: Could not find 'bundler' (2.0.1) required by your /tmp/build_a9c801af0c0fc42d984cfda6569e532c/Gemfile.lock.
remote: To update to the latest version installed on your system, run `bundle update --bundler`.
remote: To install the missing version, run `gem install bundler:2.0.1`
remote: Checked in 'GEM_PATH=vendor/bundle/ruby/2.5.0', execute `gem env` for more information
remote:
remote: To install the version of bundler this project requires, run `gem install bundler -v '2.0.1'`
remote: Bundler Output: Activating bundler (2.0.1) failed:
remote: Could not find 'bundler' (2.0.1) required by your /tmp/build_a9c801af0c0fc42d984cfda6569e532c/Gemfile.lock.
remote: To update to the latest version installed on your system, run `bundle update --bundler`.
remote: To install the missing version, run `gem install bundler:2.0.1`
remote: Checked in 'GEM_PATH=vendor/bundle/ruby/2.5.0', execute `gem env` for more information
remote:
remote: To install the version of bundler this project requires, run `gem install bundler -v '2.0.1'`
remote:
remote: !
remote: ! Failed to install gems via Bundler.
remote: !
remote: ! Push rejected, failed to compile Ruby app.
remote:
remote: ! Push failed
remote: Verifying deploy...
remote:
remote: ! Push rejected to hatchet-t-31aafb8954.
remote:
To https://git.heroku.com/hatchet-t-31aafb8954.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://git.heroku.com/hatchet-t-31aafb8954.git'
```
The first deploy succeeds because there is no cache. The second deploy fails because the cache exists and `which bundler` points to a file generated by this bundler template (in `vendor/bundle/bin/bundle`): https://github.com/bundler/bundler/blob/905dce42705d0e92fa5c74ce4b9133c7d77a6fb1/lib/bundler/templates/Executable.bundler
It fails due to this code which is run:
```
def activate_bundler(bundler_version)
if Gem::Version.correct?(bundler_version) && Gem::Version.new(bundler_version).release < Gem::Version.new("2.0")
bundler_version = "< 2"
end
gem_error = activation_error_handling do
gem "bundler", bundler_version
end
```
The `bundler_version` here will be 2.0.1 but it sees that it's own version is 2.0.2.
This can potentially be mitigated by manually setting `BUNDLER_VERSION=2.0.2`, however I'm not in love with that solution as it doesn't seem like it should be required.
Another solution is to not call this `vendor/bundle/bin/bundle` executable directly and instead go through a shim such as:
```
#!/usr/bin/env ruby
require 'rubygems'
version = "#{bundler.version}"
if ARGV.first
str = ARGV.first
str = str.dup.force_encoding("BINARY") if str.respond_to? :force_encoding
if str =~ /\A_(.*)_\z/ and Gem::Version.correct?($1) then
version = $1
ARGV.shift
end
end
if Gem.respond_to?(:activate_bin_path)
load Gem.activate_bin_path('bundler', 'bundle', version)
else
gem "bundler", version
load Gem.bin_path("bundler", "bundle", version)
end
```
This is similar (if not identical) to the shim that rubygems would install and use on a "normal" system. On Heroku we do not use this system since we do not actually install bundler through rubygems, but rather we pre-package it as a tarball, put it on S3 and then download it. We invoke bundler, not through rubygems, but by placing it's executable on the PATH and then letting the OS find it directly.
There are two fixes listed here:
- Use BUNDLE_VERSION env var
- Pros: Fixes the problem. A relatively small change.
- Cons: Requires us to propagate this env var into the dyno and/or remember to set it any time we invoke bundler. This will be very hard to catch incorrect uses as failure to include it does not always make things fail right away, but they may fail in the future. There's also no guarantee that this will continue to work into the future. I.e. this might be a fix for today
- Write our own shim file and pretend to be rubygems
- Pros: Fixes the problem
- Cons: We're effectively forking Rubygems shim logic with this method, any bugfixes or upstream changes to Rubygems would need to be mirrored in the buildpack, not great. Possibly brittle. A large change.
I'm interested in looking for a third solution, one where Heroku will behave more like a "normal" install of Ruby w/ Bundler. Ideally we would find a way for all `bundle` calls when the client's codebase does not include a `bin/bundle` binstub, to go through whatever version of Rubygems is on the system for all invocations of `bundle` as this is how it will eventually be executed on the Dyno.
- Pros: Fixes the problem. No need to worry about following upstream fixes or changes to Rubygems shim logic.
- Cons: There might be additional Rubygems bugs that we're successfully avoiding today due to our current (accidental?) logic. This would be the largest change.
# This is the commit message #2:
Update to 2.5.7
crohr
pushed a commit
that referenced
this pull request
Nov 30, 2020
* Revert "Revert "[close heroku#943] Use Bundler env vars for config (heroku#1039)" (heroku#1047)" This reverts commit 858adcb. * Fix gem installation location bug with Bundler env vars When the move from bundler flags to bundler env vars was first merged in an issue was reported heroku#1046 , this lead to an investigation and bug report to bundler rubygems/rubygems#3890 which lead to some older issues/prs: - rubygems/bundler#3552 - rubygems/bundler#6628 The issue is that global configuration and local configuration results in different behavior when installing and using some features in bundler, notedly the ability to specify install path. Due to this change, when the switch to bundler environment variables happened, it caused the size of some applications' slugs to increase dramatically, this was because the gems were essentially being installed twice. The issue appears to not be in Bundler 2.1.4 so we're bumping the version for 2.x series. In the 1.x series an environment variable `BUNDLE_GLOBAL_PATH_APPENDS_RUBY_SCOPE=1` can be used to force the desired behavior. # This is the commit message #2: Co-authored-by: Ed Morley <[email protected]> Co-authored-by: Ed Morley <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It was raising
Can't find binaries for distribution. Aborting.on amazon linux, so I made it use thecentos-6binaries.I don't know if this is the better way to fix this. I'm happy to hear your suggestions.