Apple M1: rbenv install 3.1.2 fails but installing ruby 3.1.2 from source succeeds

[milosivanovic]

Hardware: Apple M1 MacBook Air running Big Sur 11.6.8
OpenSSL: Both 1.1 and 3.0 installed using brew with 3.0 as the linked default

Step 1: Run rbenv install 3.1.2

milos@air:~$ rbenv install 3.1.2
To follow progress, use 'tail -f /var/folders/y5/vfyf5lqj655g_zbm8pvdgmkr0000gn/T/ruby-build.20221029165815.98770.log' or pass --verbose
Downloading openssl-3.0.5.tar.gz...
-> https://dqw8nmjcqpjn7.cloudfront.net/aa7d8d9bef71ad6525c55ba11e5f4397889ce49c2c9349dcea6d3e4f0b024a7a
Installing openssl-3.0.5...
Installed openssl-3.0.5 to /Users/milos/.rbenv/versions/3.1.2

Downloading ruby-3.1.2.tar.gz...
-> https://cache.ruby-lang.org/pub/ruby/3.1/ruby-3.1.2.tar.gz
Installing ruby-3.1.2...
ruby-build: using readline from homebrew
ruby-build: using gmp from homebrew

BUILD FAILED (macOS 11.6.8 using ruby-build 20221004)

Inspect or clean up the working tree at /var/folders/y5/vfyf5lqj655g_zbm8pvdgmkr0000gn/T/ruby-build.20221029165815.98770.ztN4k5
Results logged to /var/folders/y5/vfyf5lqj655g_zbm8pvdgmkr0000gn/T/ruby-build.20221029165815.98770.log

Last 10 log lines:
        from ./tool/rbinstall.rb:899:in `block in install'
        from ./tool/rbinstall.rb:713:in `no_write'
        from ./tool/rbinstall.rb:899:in `install'
        from ./tool/rbinstall.rb:1068:in `block (2 levels) in <main>'
        from ./tool/rbinstall.rb:1043:in `foreach'
        from ./tool/rbinstall.rb:1043:in `block in <main>'
        from ./tool/rbinstall.rb:1127:in `block in <main>'
        from ./tool/rbinstall.rb:1124:in `each'
        from ./tool/rbinstall.rb:1124:in `<main>'
make: *** [do-install-all] Error 1

Step 2: Notice that rbenv versions does not show 3.1.2 because it failed to install using ruby-build

milos@air:~$ rbenv versions
  system
* 2.7.4 (set by /Users/milos/.rbenv/version)
  3.0.4

Step 3: Install 3.1.2 manually from source, bypassing ruby-build entirely

wget https://cache.ruby-lang.org/pub/ruby/3.1/ruby-3.1.2.tar.gz
tar xf ruby-3.1.2.tar.gz
cd ruby-3.1.2
./configure --prefix=/Users/milos/.rbenv/versions/3.1.2 # add `--with-openssl-dir=/opt/homebrew/opt/openssl\@1.1` or @3 as preferred, if needed
make -j9
make install

Step 4: Notice that rbenv versions now shows 3.1.2 successfully installed

milos@air:~$ rbenv versions
  system
* 2.7.4 (set by /Users/milos/.rbenv/version)
  3.0.4
  3.1.2
milos@air:~$ rbenv global 3.1.2

milos@air:~$ ruby -v
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [arm64-darwin20]

milos@air:~$ ruby -ropenssl -e 'puts OpenSSL::OPENSSL_VERSION'
OpenSSL 3.0.5 5 Jul 2022

Please do not move this to the Discussions tab as the installation works when not using ruby-build. This leads me to believe that it is indeed a ruby-build issue.

For what it's worth, I can install ruby 3.1.2 this way with either OpenSSL 3.0 or OpenSSL 1.1 depending on what I provide in the configure step. I can provide --with-openssl-dir=/opt/homebrew/opt/openssl\@1.1 to override the 3.0 default that brew linked. Both work. However, again, with ruby-build the installation fails as per my report in #1961

Thanks to #1961 (reply in thread) for finding this workaround that makes the source build usable with rbenv.

[hsbt]

Please share full of your logs.

[milosivanovic]

They are in #1961

[hsbt]

No, I want to see full of /var/folders/y5/vfyf5lqj655g_zbm8pvdgmkr0000gn/T/ruby-build.20221029165815.98770.log.

[milosivanovic]

ruby-build.20221029165815.98770.log

[milosivanovic]

The following is the output of running make install manually as per Step 3 above:

milos@air:~/d/ruby/ruby-3.1.2$ make install
        BASERUBY = /Users/milos/.rbenv/shims/ruby --disable=gems
        CC = clang
        LD = clang
        LDSHARED = clang -dynamiclib
        CFLAGS = -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -W
old-style-definition -Wmissing-noreturn -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-u
nused-value -Wunused-variable -Wextra-tokens -Wundef -std=gnu99  -pipe
        XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -fPIE -I. -I.ext/include/arm64-darwin20 -I./include -I. -I./enc/unicode/13.0.0
        CPPFLAGS = -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT
        DLDFLAGS = -Wl,-undefined,dynamic_lookup -Wl,-multiply_defined,suppress -fstack-protector-strong -Wl,-pie -framework CoreFoundation
        SOLIBS = -lpthread -ldl -lobjc
        LANG = en_US.UTF-8
        LC_ALL =
        LC_CTYPE =
        MFLAGS =
Apple clang version 13.0.0 (clang-1300.0.29.30)
Target: arm64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
./revision.h unchanged
generating enc.mk
making srcs under enc
make[1]: Nothing to be done for `srcs'.
generating transdb.h
transdb.h unchanged
generating makefiles ext/configure-ext.mk
ext/configure-ext.mk updated
generating makefile exts.mk
exts.mk unchanged
./revision.h unchanged
make[1]: Nothing to be done for `note'.
making enc
make[1]: Nothing to be done for `enc'.
making trans
make[1]: Nothing to be done for `./enc/trans'.
making encs
make[1]: Nothing to be done for `encs'.
Generating RDoc documentation
Parsing sources...
100% [917/917]  yjit_utils.c

Generating RI format into /Users/milos/d/ruby/ruby-3.1.2/.ext/rdoc...

  Files:        917

  Classes:      983 ( 322 undocumented)
  Modules:      206 (  59 undocumented)
  Constants:   2200 ( 597 undocumented)
  Attributes:   984 ( 290 undocumented)
  Methods:     9458 (2161 undocumented)

  Total:      13831 (3429 undocumented)
   75.21% documented

  Elapsed: 14.6s

generating arm64-darwin20-fake.rb
arm64-darwin20-fake.rb updated
./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems -r./arm64-darwin20-fake ./tool/rbinstall.rb --make="/Applications/Xcode.app/Contents/Developer/usr/bin/make" --dest-dir="" --extout=".ext" --ext-buil
d-dir="./ext" --mflags="" --make-flags="" --data-mode=0644 --prog-mode=0755 --installed-list .installed.list --mantype="doc"  --gnumake --install=all --rdoc-output=".ext/rdoc" --html-output=".ext/html"
installing binary commands:         /Users/milos/.rbenv/versions/3.1.2/bin
installing base libraries:          /Users/milos/.rbenv/versions/3.1.2/lib
installing arch files:              /Users/milos/.rbenv/versions/3.1.2/lib/ruby/3.1.0/arm64-darwin20
installing pkgconfig data:          /Users/milos/.rbenv/versions/3.1.2/lib/pkgconfig
installing extension objects:       /Users/milos/.rbenv/versions/3.1.2/lib/ruby/3.1.0/arm64-darwin20
installing extension objects:       /Users/milos/.rbenv/versions/3.1.2/lib/ruby/site_ruby/3.1.0/arm64-darwin20
installing extension objects:       /Users/milos/.rbenv/versions/3.1.2/lib/ruby/vendor_ruby/3.1.0/arm64-darwin20
installing extension headers:       /Users/milos/.rbenv/versions/3.1.2/include/ruby-3.1.0/arm64-darwin20
installing extension scripts:       /Users/milos/.rbenv/versions/3.1.2/lib/ruby/3.1.0
installing extension scripts:       /Users/milos/.rbenv/versions/3.1.2/lib/ruby/site_ruby/3.1.0
installing extension scripts:       /Users/milos/.rbenv/versions/3.1.2/lib/ruby/vendor_ruby/3.1.0
installing extension headers:       /Users/milos/.rbenv/versions/3.1.2/include/ruby-3.1.0/ruby
installing rdoc:                    /Users/milos/.rbenv/versions/3.1.2/share/ri/3.1.0/system
installing html-docs:               /Users/milos/.rbenv/versions/3.1.2/share/doc/ruby
installing capi-docs:               /Users/milos/.rbenv/versions/3.1.2/share/doc/ruby
installing command scripts:         /Users/milos/.rbenv/versions/3.1.2/bin
installing library scripts:         /Users/milos/.rbenv/versions/3.1.2/lib/ruby/3.1.0
installing common headers:          /Users/milos/.rbenv/versions/3.1.2/include/ruby-3.1.0
installing manpages:                /Users/milos/.rbenv/versions/3.1.2/share/man/man1
installing default gems from lib:   /Users/milos/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0
                                    abbrev 0.1.0
                                    base64 0.1.1
                                    benchmark 0.2.0
                                    bundler 2.3.7
                                    cgi 0.3.1
                                    csv 3.2.2
                                    delegate 0.2.0
                                    did_you_mean 1.6.1
                                    drb 2.1.0
                                    english 0.7.1
                                    erb 2.2.3
                                    error_highlight 0.3.0
                                    fileutils 1.6.0
                                    find 0.1.1
                                    forwardable 1.3.2
                                    getoptlong 0.1.1
                                    ipaddr 1.2.4
                                    irb 1.4.1
                                    logger 1.5.0
                                    mutex_m 0.1.1
                                    net-http 0.2.0
                                    net-protocol 0.1.2
                                    observer 0.1.1
                                    open-uri 0.2.0
                                    open3 0.1.1
                                    optparse 0.2.0
                                    ostruct 0.5.2
                                    pp 0.3.0
                                    prettyprint 0.1.1
                                    pstore 0.1.1
                                    racc 1.6.0
                                    rdoc 6.4.0
                                    readline 0.0.3
                                    reline 0.3.0
                                    resolv 0.2.1
                                    resolv-replace 0.1.0
                                    rinda 0.1.1
                                    ruby2_keywords 0.0.5
                                    securerandom 0.1.1
                                    set 1.0.2
                                    shellwords 0.1.0
                                    singleton 0.1.1
                                    tempfile 0.1.2
                                    time 0.2.0
                                    timeout 0.2.0
                                    tmpdir 0.1.2
                                    tsort 0.1.0
                                    un 0.2.0
                                    uri 0.11.0
                                    weakref 0.1.1
                                    yaml 0.2.0
installing default gems from ext:   /Users/milos/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0
                                    bigdecimal 3.1.1
                                    date 3.2.2
                                    digest 3.1.0
                                    etc 1.3.0
                                    fcntl 1.0.1
                                    fiddle 1.1.0
                                    io-console 0.5.11
                                    io-nonblock 0.1.0
                                    io-wait 0.2.1
                                    json 2.6.1
                                    nkf 0.1.1
                                    openssl 3.0.0
                                    pathname 0.2.0
                                    psych 4.0.3
                                    readline-ext 0.1.4
                                    stringio 3.0.1
                                    strscan 3.0.1
                                    syslog 0.1.0
                                    zlib 2.1.1
installing bundled gems:            /Users/milos/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0
                                    minitest 5.15.0
                                    power_assert 2.0.1
                                    rake 13.0.6
                                    test-unit 3.5.3
                                    rexml 3.2.5
                                    rss 0.2.9
                                    net-ftp 0.1.3
                                    net-imap 0.2.3
                                    net-pop 0.1.1
                                    net-smtp 0.3.1
                                    matrix 0.4.2
                                    prime 0.1.2
                                    rbs 2.1.0
Building native extensions. This could take a while...
                                    typeprof 0.21.2
                                    debug 1.4.0
Building native extensions. This could take a while...
installing bundled gem cache:       /Users/milos/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/cache

The rbs-2.1.0 gem successfully compiles/links/loads, and thus the installation fully completes.

However, in the ruby-build.20221029165815.98770.log log file from my previous comment (which is the output from rbenv install 3.1.2), the rbs-2.1.0 gem installation step fails, I believe while linking (dyld: could not load inserted library...)

So the question becomes: Why does the rbs-2.1.0 gem install successfully in the manual make install step, but not with rbenv install 3.1.2?

[hsbt]

Thanks.

ruby-3.1.2/libruby.3.1.dylib: incompatible cpu-subtype: 0x00000000

The above line is not related with ruby-build. Maybe, build system/toolchains of Ruby core does. I'll investigate its detail at next week.

[milosivanovic]

I just found that there is a major difference in the ./configure output of the rbenv installation vs manual installation:

rbenv:

Configuration summary for ruby version 3.1.2

   * Installation prefix: /Users/milos/.rbenv/versions/3.1.2
   * exec prefix:         ${prefix}
   * arch:                arm64-darwin20
   * site arch:           ${arch}
   * RUBY_BASE_NAME:      ruby
   * enable shared:       yes
   * ruby lib prefix:     ${libdir}/${RUBY_BASE_NAME}
   * site libraries path: ${rubylibprefix}/${sitearch}
   * vendor path:         ${rubylibprefix}/vendor_ruby
   * target OS:           darwin20
   * compiler:            clang
   * with pthread:        yes
   * with coroutine:      arm64
   * enable shared libs:  yes
   * dynamic library ext: bundle
   * CFLAGS:              -fdeclspec ${optflags} ${debugflags} ${warnflags}
   * LDFLAGS:             -L. -L/Users/milos/.rbenv/versions/3.1.2/lib  \
                          -fstack-protector-strong
   * DLDFLAGS:            -L/Users/milos/.rbenv/versions/3.1.2/lib  \
                          -Wl,-undefined,dynamic_lookup \
                          -Wl,-multiply_defined,suppress
   * optflags:            -O3 -fno-fast-math
   * debugflags:          -ggdb3
   * warnflags:           -Wall -Wextra -Wdeprecated-declarations \
                          -Wdivision-by-zero \
                          -Wimplicit-function-declaration -Wimplicit-int \
                          -Wmisleading-indentation -Wpointer-arith \
                          -Wshorten-64-to-32 -Wwrite-strings \
                          -Wold-style-definition -Wmissing-noreturn \
                          -Wno-constant-logical-operand -Wno-long-long \
                          -Wno-missing-field-initializers \
                          -Wno-overlength-strings -Wno-parentheses-equality \
                          -Wno-self-assign -Wno-tautological-compare \
                          -Wno-unused-parameter -Wno-unused-value \
                          -Wunused-variable -Wextra-tokens -Wundef
   * strip command:       strip -A -n
   * install doc:         rdoc
   * JIT support:         yes
   * man page type:       doc
   * BASERUBY -v:         ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) \
                          [arm64-darwin20]

manual:

Configuration summary for ruby version 3.1.2

   * Installation prefix: /Users/milos/.rbenv/versions/3.1.2
   * exec prefix:         ${prefix}
   * arch:                arm64-darwin20
   * site arch:           ${arch}
   * RUBY_BASE_NAME:      ruby

   * ruby lib prefix:     ${libdir}/${RUBY_BASE_NAME}
   * site libraries path: ${rubylibprefix}/${sitearch}
   * vendor path:         ${rubylibprefix}/vendor_ruby
   * target OS:           darwin20
   * compiler:            clang
   * with pthread:        yes
   * with coroutine:      arm64
   * enable shared libs:  no
   * dynamic library ext: bundle
   * CFLAGS:              -fdeclspec ${optflags} ${debugflags} ${warnflags}
   * LDFLAGS:             -L. -fstack-protector-strong

   * DLDFLAGS:            -Wl,-undefined,dynamic_lookup \
                          -Wl,-multiply_defined,suppress
                          
   * optflags:            -O3 -fno-fast-math
   * debugflags:          -ggdb3
   * warnflags:           -Wall -Wextra -Wdeprecated-declarations \
                          -Wdivision-by-zero \
                          -Wimplicit-function-declaration -Wimplicit-int \
                          -Wmisleading-indentation -Wpointer-arith \
                          -Wshorten-64-to-32 -Wwrite-strings \
                          -Wold-style-definition -Wmissing-noreturn \
                          -Wno-constant-logical-operand -Wno-long-long \
                          -Wno-missing-field-initializers \
                          -Wno-overlength-strings -Wno-parentheses-equality \
                          -Wno-self-assign -Wno-tautological-compare \
                          -Wno-unused-parameter -Wno-unused-value \
                          -Wunused-variable -Wextra-tokens -Wundef
   * strip command:       strip -A -n
   * install doc:         rdoc
   * JIT support:         yes
   * man page type:       doc
   * BASERUBY -v:         ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) \
                          [arm64-darwin20]

We can see that * enable shared libs: no is shown when compiling manually per Step 3. This means that the difference between the the manual step vs rbenv is that rbenv enables shared libraries by default, and for some reason that creates an incompatible libruby3.1.dylib. Since we do not specify --enable-shared when compiling manually, we do not get the cpu subtype error since there is no shared library to load.

To recap:

When compiling manually, the full ./configure command (as per Step 3) is:

./configure --prefix=/Users/milos/.rbenv/versions/3.1.2

If we then add --enable-shared like this:

./configure --prefix=/Users/milos/.rbenv/versions/3.1.2 --enable-shared

Then we get the same error as with rbenv.

So I think you may be right - it may not be a ruby-build issue after all. Please do try to check if it's an issue with ruby's build system or toolchain and let me know if I can help with troubleshooting since I can reproduce both scenarios consistently.

[mislav]

Thanks for narrowing this down, @milosivanovic.

[mislav]

@hsbt Were you ever able to take a look into this?

I'm wondering whether there is something we should fix in ruby-build regarding this. In the meantime, this is what I suggest as workaround to affected users: #1961 (reply in thread)

[hsbt]

Yes, I'll track this issue now. But I couldn't reproduce this with Ventura 13.1 beta and Xcode 14 🤔

$ RUBY_CONFIGURE_OPTS='--enable-shared --disable-install-doc' rbenv install 3.1.2
To follow progress, use 'tail -f /var/folders/_z/1_hsk6zx07v57yg5wlsjxvc80000gn/T/ruby-build.20221117160651.3444.log' or pass --verbose
Installing openssl-3.0.7...
Installed openssl-3.0.7 to /Users/hsbt/.local/share/rbenv/versions/3.1.2

Installing ruby-3.1.2...
ruby-build: using readline from homebrew
ruby-build: using gmp from homebrew
Installed ruby-3.1.2 to /Users/hsbt/.local/share/rbenv/versions/3.1.2

$ otool -L ~/.local/share/rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/debug-1.4.0/ext/debug/debug.bundle
/Users/hsbt/.local/share/rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/debug-1.4.0/ext/debug/debug.bundle:
        /Users/hsbt/.local/share/rbenv/versions/3.1.2/lib/libruby.3.1.dylib (compatibility version 3.1.0, current version 3.1.2)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

We will release all stable versions includes the fixes of Ventura and Xcode14 in Nov 2022. We will wait to try them for users who get same issues.

[SeanSith]

@milosivanovic, you should follow up over here with your discussion over in ruby-build! #1961

[mislav]

@SeanSith Thanks for the ping. Closing since Ruby 3.1.3 should have fixed this for Xcode 14 and macOS Ventura. If there are new issues with building 3.1.3, please open a new thread. If the failure with building 3.1.3 is exactly the same as originally reported in this thread, please reopen. Thank you!

[SeanSith]

@mislav Unfortunately I'm still having the same issues noted in the original thread. The discussion over in ruby-build got me going (having to install openssl@3, disable shared-libraries), so it's not a straight-up asdf install ruby 3.1.3 here, but I figured that the discussion over there would help some folks out at least.

My command was:

RUBY_CONFIGURE_OPTS="--disable-install-doc --disable-shared --with-openssl-dir=$(brew --prefix openssl@3)" asdf install ruby 3.1.3

❯ pkgutil --pkg-info=com.apple.pkg.CLTools_Executables | grep version
version: 14.1.0.0.1.1666437224
❯ sw_vers
ProductName:		macOS
ProductVersion:		13.0.1
BuildVersion:		22A400
❯ asdf --version
v0.10.2
❯ head -n 3 ~/.asdf/plugins/ruby/lib/utils.sh | grep VERSION
RUBY_BUILD_VERSION="${ASDF_RUBY_BUILD_VERSION:-v20221124}"

[GOI17]

I finally install successfully ruby 3.1.3 with with this. Thank u

[mislav]

@SeanSith Sorry for the trouble. Are you saying that downloading a Ruby 3.1.3 tarball and running ./configure --prefix=/path/to/ruby && make && make install there works, but not when built through ruby-build? And your solution to that was to add --disable-shared?

It would help us if you posted the full failed build log using a gist. Thanks!

[milosivanovic]

@mislav the issue regarding the mismatched architecture still occurs on my machine with 3.1.3. I am still on Big Sur and still have Xcode 13.2.1.

Steps to reproduce:

1. wget https://cache.ruby-lang.org/pub/ruby/3.1/ruby-3.1.3.tar.gz
2. tar xf ruby-3.1.3.tar.gz
3. cd ruby-3.1.3
4. ./configure --enable-shared (make sure your openssl path is symlinked correctly, or provide it manually)
5. make -j9
6. make install

During the make install step, it tries to build gems, and fails on the rbs-2.7.0 gem.

installing bundled gems:            /Users/milos/.rbenv/versions/3.1.3/lib/ruby/gems/3.1.0
                                    minitest 5.15.0
                                    power_assert 2.0.1
                                    rake 13.0.6
                                    test-unit 3.5.3
                                    rexml 3.2.5
                                    rss 0.2.9
                                    net-ftp 0.1.3
                                    net-imap 0.2.3
                                    net-pop 0.1.1
                                    net-smtp 0.3.1
                                    matrix 0.4.2
                                    prime 0.1.2
                                    rbs 2.7.0
Building native extensions. This could take a while...
/Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/builder.rb:102:in `run': ERROR: Failed to build gem native extension. (Gem::Ext::BuildError)

    current directory: /Users/milos/.rbenv/versions/3.1.3/lib/ruby/gems/3.1.0/gems/rbs-2.7.0/ext/rbs_extension
/Users/milos/.rbenv/versions/3.1.3/bin/ruby -I /Users/milos/d/temp/ruby-3.1.3/lib extconf.rb
checking for whether -std=c99 is accepted as CFLAGS... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/Users/milos/.rbenv/versions/3.1.3/bin/$(RUBY_BASE_NAME)
/Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:490:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:616:in `block in try_compile'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:563:in `with_werror'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:616:in `try_compile'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:680:in `try_cflags'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:1025:in `block (2 levels) in append_cflags'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:989:in `block in checking_for'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:354:in `block (2 levels) in postpone'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:324:in `open'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:354:in `block in postpone'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:324:in `open'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:350:in `postpone'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:988:in `checking_for'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:1024:in `block in append_cflags'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:1023:in `each'
        from /Users/milos/d/temp/ruby-3.1.3/lib/mkmf.rb:1023:in `append_cflags'
        from extconf.rb:3:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /Users/milos/.rbenv/versions/3.1.3/lib/ruby/gems/3.1.0/extensions/arm64-darwin-20/3.1.0/rbs-2.7.0/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /Users/milos/.rbenv/versions/3.1.3/lib/ruby/gems/3.1.0/gems/rbs-2.7.0 for inspection.
Results logged to /Users/milos/.rbenv/versions/3.1.3/lib/ruby/gems/3.1.0/extensions/arm64-darwin-20/3.1.0/rbs-2.7.0/gem_make.out
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/ext_conf_builder.rb:28:in `build'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/builder.rb:171:in `build_extension'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/builder.rb:205:in `block in build_extensions'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/builder.rb:202:in `each'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/builder.rb:202:in `build_extensions'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/installer.rb:843:in `build_extensions'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/installer.rb:326:in `install'
        from ./tool/rbinstall.rb:899:in `block in install'
        from ./tool/rbinstall.rb:713:in `no_write'
        from ./tool/rbinstall.rb:899:in `install'
        from ./tool/rbinstall.rb:1060:in `block (2 levels) in <main>'
        from ./tool/rbinstall.rb:1044:in `foreach'
        from ./tool/rbinstall.rb:1044:in `block in <main>'
        from ./tool/rbinstall.rb:1116:in `block in <main>'
        from ./tool/rbinstall.rb:1113:in `each'
        from ./tool/rbinstall.rb:1113:in `<main>'
/Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/builder.rb:102:in `run': extconf failed, exit code 1 (Gem::InstallError)
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/ext_conf_builder.rb:28:in `build'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/builder.rb:171:in `build_extension'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/builder.rb:205:in `block in build_extensions'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/builder.rb:202:in `each'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/ext/builder.rb:202:in `build_extensions'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/installer.rb:843:in `build_extensions'
        from /Users/milos/d/temp/ruby-3.1.3/lib/rubygems/installer.rb:326:in `install'
        from ./tool/rbinstall.rb:899:in `block in install'
        from ./tool/rbinstall.rb:713:in `no_write'
        from ./tool/rbinstall.rb:899:in `install'
        from ./tool/rbinstall.rb:1060:in `block (2 levels) in <main>'
        from ./tool/rbinstall.rb:1044:in `foreach'
        from ./tool/rbinstall.rb:1044:in `block in <main>'
        from ./tool/rbinstall.rb:1116:in `block in <main>'
        from ./tool/rbinstall.rb:1113:in `each'
        from ./tool/rbinstall.rb:1113:in `<main>'
make: *** [do-install-all] Error 1

The contents of the /Users/milos/.rbenv/versions/3.1.3/lib/ruby/gems/3.1.0/extensions/arm64-darwin-20/3.1.0/rbs-2.7.0/mkmf.log file are:

DYLD_FALLBACK_LIBRARY_PATH=.:/Users/milos/.rbenv/versions/3.1.3/lib:/Users/milos/d/temp/ruby-3.1.3 "clang -o conftest -I/Users/milos/.rbenv/versions/3.1.3/include/ruby-3.1.0/arm64-darwin20 -I/Users/milos/.rbenv/versions/3.1.3/include/ruby-3.1.0/ruby/backward -I/Users/milos/.rbenv/versions/3.1.3/include/ruby-3.1.0 -I. -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef  -fno-common -pipe conftest.c  -L. -L/Users/milos/.rbenv/versions/3.1.3/lib -L. -fstack-protector-strong     -lruby.3.1   "
dyld: could not load inserted library '/Users/milos/d/temp/ruby-3.1.3/libruby.3.1.dylib' because no suitable image found.  Did find:
        /Users/milos/d/temp/ruby-3.1.3/libruby.3.1.dylib: incompatible cpu-subtype: 0x00000000 in /Users/milos/d/temp/ruby-3.1.3/libruby.3.1.dylib
        /Users/milos/d/temp/ruby-3.1.3/libruby.3.1.dylib: stat() failed with errno=1

checked program was:
/* begin */
1: #include "ruby.h"
2:
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

The built shared library from the make -j9 step is there:

milos@air:~/d/temp/ruby-3.1.3$ ls -l libruby*
-rw-r--r-- 1 milos staff 20791968 Dec  1 10:31 libruby.3.1-static.a
-rwxr-xr-x 1 milos staff  4243614 Dec  1 10:31 libruby.3.1.dylib
lrwxr-xr-x 1 milos staff       17 Dec  1 10:31 libruby.dylib -> libruby.3.1.dylib
<...snip...>

And it's showing as having arm64 architecture:

milos@air:~/d/temp/ruby-3.1.3$ file libruby.3.1.dylib
libruby.3.1.dylib: Mach-O 64-bit dynamically linked shared library arm64

[milosivanovic]

Here is the full workaround with shared library support enabled:

1. wget https://cache.ruby-lang.org/pub/ruby/3.1/ruby-3.1.3.tar.gz
2. tar xf ruby-3.1.3.tar.gz
3. cd ruby-3.1.3
4. grep -v DYLD_INSERT_LIBRARIES configure > configuretmp && mv configuretmp configure && chmod +x ./configure
5. ./configure --enable-shared (make sure your openssl path is symlinked correctly, or provide it manually)
6. make -j9
7. make install

It installs successfully without any errors.

I was able to validate that it was built with shared libraries:

milos@air:~/.rbenv/versions/3.1.3/lib/ruby/gems/3.1.0/gems/rbs-2.7.0/ext/rbs_extension$ otool -L rbs_extension.bundle
rbs_extension.bundle:
        /Users/milos/.rbenv/versions/3.1.3/lib/libruby.3.1.dylib (compatibility version 3.1.0, current version 3.1.3)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)

I then redid it but this time building with just ./configure (i.e. without --enable-shared) , and was able to confirm that the gems were built statically:

milos@air:~/.rbenv/versions/3.1.3/lib/ruby/gems/3.1.0/gems/rbs-2.7.0/ext/rbs_extension$ otool -L rbs_extension.bundle
rbs_extension.bundle:
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)

The --enable-shared flag works fine without DYLD_INSERT_LIBRARIES in the configure script, so I believe it is safe to remove.

Could you test the removal of DYLD_INSERT_LIBRARIES on your system and then propose to make this change upstream if there are no concerns?

[hsbt]

Can you check your environmental variable of DYLD_INSERT_LIBRARIES at first?

I still couldn't reproduce this with --enable-shared option. my environment doesn't have this environmental variable.

$ env | grep DYLD_INSERT_LIBRARIES

and this seems not ruby-build issue. We should move to https://bugs.ruby-lang.org/

[milosivanovic]

The environment variable is not set in our standard bash terminal, which is why it was tricky to detect in the first place. It's the configure script which sets it: https://github.com/ruby/ruby/blob/master/configure.ac#L3050

Before ./configure --enable-shared is run:

milos@air:~/d/temp/ruby-3.1.3$ grep -rF DYLD_INSERT_LIBRARIES
configure.ac:                   : ${PRELOADENV=DYLD_INSERT_LIBRARIES}
configure:                      : ${PRELOADENV=DYLD_INSERT_LIBRARIES}

After ./configure --enable-shared is run:

milos@air:~/d/temp/ruby-3.1.3$ grep -rF DYLD_INSERT_LIBRARIES
configure.ac:                   : ${PRELOADENV=DYLD_INSERT_LIBRARIES}
configure:                      : ${PRELOADENV=DYLD_INSERT_LIBRARIES}
Makefile:       quote 'PRELOADENV       "DYLD_INSERT_LIBRARIES"'; \
config.status:S["PRELOADENV"]="DYLD_INSERT_LIBRARIES"
config.log:PRELOADENV='DYLD_INSERT_LIBRARIES'

After make -j9, it's set in even more places:

milos@air:~/d/temp/ruby-3.1.3$ grep -rF DYLD_INSERT_LIBRARIES
grep: miniruby: binary file matches
configure.ac:                   : ${PRELOADENV=DYLD_INSERT_LIBRARIES}
rbconfig.rb:  CONFIG["PRELOADENV"] = "DYLD_INSERT_LIBRARIES"
mjit_config.h:#define PRELOADENV       "DYLD_INSERT_LIBRARIES"
configure:                      : ${PRELOADENV=DYLD_INSERT_LIBRARIES}
Makefile:       quote 'PRELOADENV       "DYLD_INSERT_LIBRARIES"'; \
grep: mjit.o: binary file matches
grep: libruby.3.1.dylib: binary file matches
grep: libruby.3.1-static.a: binary file matches
config.status:S["PRELOADENV"]="DYLD_INSERT_LIBRARIES"
config.log:PRELOADENV='DYLD_INSERT_LIBRARIES'

That's where miniruby gets the DYLD_INSERT_LIBRARIES environment variable from when it calls extconf.rb, which calls clang.

And, yes, this is a ruby issue. I will report it at https://bugs.ruby-lang.org/ later today and propose that DYLD_INSERT_LIBRARIES be removed from configure.ac, since it is causing incompatibilities with some systems, and does not seem to be necessary to have.

[hsbt]

Why only your system affects with DYLD_INSERT_LIBRARIES?

This environmental variable used by injecting another libraries like jemalloc. We have no chance to remove it without your root cause.

[milosivanovic]

It's not only my system:

• Unable to install ruby 3.1.x on M1 (Apple Silicon) #1961 (comment)
• Unable to install ruby 3.1.x on M1 (Apple Silicon) #1961 (comment)
• Unable to install ruby 3.1.x on M1 (Apple Silicon) #1961 (reply in thread)
• Unable to install ruby 3.1.x on M1 (Apple Silicon) #1961 (comment)
• Unable to install ruby 3.1.x on M1 (Apple Silicon) #1961 (comment)

There are also quite a few who just gave a thumbs up without commenting.

But yes, you're right - when we want to override the default memory allocator with jemalloc, we would need DYLD_INSERT_LIBRARIES. I'll report my findings to the ruby team and hopefully we can make more progress on this.

[mislav]

Thanks so much @milosivanovic for excellent sleuthing and @hsbt for helping.

Agreed this should be primarily handled in the Ruby issue tracker. But, is there something that we could do from ruby-build in the meantime as a workaround for all people affected?

[milosivanovic]

@hsbt I was able to compile and run ruby with jemalloc even after removing DYLD_INSERT_LIBRARIES. Here are my steps:

1. wget https://cache.ruby-lang.org/pub/ruby/3.1/ruby-3.1.3.tar.gz
2. tar xf ruby-3.1.3.tar.gz
3. cd ruby-3.1.3
4. grep -v DYLD_INSERT_LIBRARIES configure > configuretmp && mv configuretmp configure && chmod +x ./configure
5. ./configure LDFLAGS=-L$(brew --prefix jemalloc)/lib CPPFLAGS=-I$(brew --prefix jemalloc)/include --enable-shared --with-jemalloc
6. make -j9
7. make install

Confirm jemalloc is in use by running MALLOC_CONF=stats_print:true /Users/milos/d/temp/ruby-3.1.3/ruby -e "exit":

milos@air:~/d/temp/ruby-3.1.3$ MALLOC_CONF=stats_print:true ./ruby -e "exit"
___ Begin jemalloc statistics ___
Version: "5.3.0-0-g54eaed1d8b56b1aa528be3bdd1877e59c56fa90c"
Build-time option settings
  config.cache_oblivious: true
  config.debug: false
  config.fill: true
  config.lazy_lock: false
<...snip...>

So it does not appear DYLD_INSERT_LIBRARIES is necessary to exist in the manner that it does for jemalloc to be injectable.

[milosivanovic]

@mislav My (temporary) suggestion would be for ruby-build to strip out that line from the configure script, as per my repro steps. I'm open to hearing if there are other concerns.

[mislav]

@milosivanovic Thanks for the suggestion. I do not know what DYLD_INSERT_LIBRARIES does, so I'd be careful about stripping it out and potentially negatively affecting builds on other platforms. I welcome suggestions of others who have more knowledge about this and why it's needed in the configure steps 🙇

[milosivanovic]

@mislav DYLD_INSERT_LIBRARIES cannot affect other platforms as it's only for macOS/darwin. If you check the configure script, it's in the darwin block only: https://github.com/ruby/ruby/blob/master/configure.ac#L3053.

The linux version of that variable is LD_PRELOAD.

When a dynamically-built binary tries to reference a symbol from a shared library, the library that the symbol comes from can be overridden with LD_PRELOAD/DYLD_INSERT_LIBRARIES. A specific example would be to replace a malloc implementation without recompiling the binary. This is exactly what @hsbt was referring to when referring to jemalloc. However, as per my testing, jemalloc still appears to work fine on on macOS without DYLD_INSERT_LIBRARIES in the configure step.

To your point, if jemalloc can work without DYLD_INSERT_LIBRARIES in the configure script, then I'm also unclear on when it's actually there. I think jemalloc still works because (in my latest example) we are actually specifically setting up ruby to be compiled against it with -ljemalloc, rather than us trying to override symbols linked against the standard macOS implementation.

I'll be submitting all of these findings over to the ruby core team this weekend, so hopefully someone with more knowledge can help us understand what's happening. Thanks @hsbt and @mislav for your patience and help thus far!

[kateinoigakukun]

Hi there, I've investigated this issue a little bit, and finally got a repro on my end. Let me put my idea before going that direction.

The essential requirement for repro is that SIP is disabled.

TL;DR: Please re-enable SIP by following the official instruction and retry rbenv install 3.1.3

Why?

ruby-3.1.2/libruby.3.1.dylib: incompatible cpu-subtype: 0x00000000

The error repoted during rbs_extension build is produced by the dynamic loader dyld. It says that the Mach-O executable file and the inserted Mach-O dylib by DYLD_INSERT_LIBRARIES have incompatible cpu-subtype. Especially, the executable (/usr/bin/clang) is compiled for arm64e but the dylib (libruby.3.1.dylib) is compiled for arm64. (arm64e is a new arch supported by recent Apple devices basically arm64 + pointer authentication support.)

Okay, so we can't set DYLD_INSERT_LIBRARIES while running /usr/bin/clang. But why it fails only when SIP is disabled?

Executable binaries that is code-signed with Apple certificates are called platform binaries: https://developer.apple.com/documentation/endpointsecurity/es_process_t/3228979-is_platform_binary
Platform binaries are usually located under /usr/bin, and macOS applies additional security protections for them when SIP is enabled. (Hardened Runtime: https://developer.apple.com/documentation/security/hardened_runtime).

The additional security protections, Hardened Runtime, disables DYLD_* environment variables, so DYLD_INSERT_LIBRARIES does not have any effect under SIP. Therefore the arch incompatibility error is not raised under SIP.

Considering the above discussion, we have two workarounds for now:

Workaround 1: Re-enable SIP

This is a little bit tricky, but enabling SIP would also solve the issue because it prevents loading incompatible library through DYLD_INSERT_LIBRARIES

(Potential) Workaround 2: Use clang compiled for arm64

Warning
I couldn't succeed to build by this way, but I think it works in theory. I think there should be something wrong in configure options.

/usr/bin/clang is compiled for arm64e, but it's just a shim and the actual clang binary is usually compiled for arm64:

$ xcrun -f clang
/Applications/Xcode-14.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
$ file $(xcrun -f clang)
/Applications/Xcode-14.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
/Applications/Xcode-14.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang (for architecture x86_64):    Mach-O 64-bit executable x86_64
/Applications/Xcode-14.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang (for architecture arm64):     Mach-O 64-bit executable arm64

So building with explicit compiler path may solve the issue:

$ export SDKROOT="$(xcrun -show-sdk-path -sdk macosx)" # because the /usr/bin/clang shim automatically set it
$ export RUBY_CONFIGURE_OPTS="CC=$(xcrun -f clang)"
$ rbenv install 3.1.3

[hsbt]

I partly agreed to backport this patch to ruby-build until releasing the next stable versions of Ruby because this DYLD_INSERT_LIBRARIES is too old logic and unnecessary today.

But I'm against to support non-SIP environment. I don't have a motivation to maintain under the non-SIP environment because it's not normal usage of macOS.

[milosivanovic]

@hsbt Implementing the changes does not mean supporting non-SIP-enabled environments. As I have mentioned before, the only reason SIP-enabled environments work is due to Apple's restriction on this variable, which means that the clang compiler does not get to read this variable, making it 100% ignored. On non-SIP-enabled machines, the variable is able to be read, and that's where it fails. I hope you understand, but if not, let me know how I can help clarify.

[hsbt]

I totally understood this mechanism. I also spent many of time about non-SIP environment. I'm tired.

[mislav]

@hsbt I have no strong opinion on SIP vs non-SIP. Even though I always use my macOS on its default settings (plus full disk encryption), if there is a known bug that affects Ruby builds for some macOS users and we know what caused it, I think it would be a nice touch to solve it with a patch. I think enough humans have already spent enough total hours breaking their brains over this.

[hsbt]

Yes. I agreed to care about #2106 (reply in thread). I want to share my concerns about non-SIP environment while @mislav and me.

[ydarbleoj]

Tried the first workaround and was able to install 3.1.3.

MacOS: Ventura 13.0.1 and Xcode 14.1 - M1

[milosivanovic]

@kateinoigakukun Those are excellent findings, thank you for diving deep into this with us.

You are right, I have SIP disabled! I will try enabling SIP on the affected machine later today, but it sounds like @ydarbleoj already confirmed this - nice! That might also explain why some users who reinstalled their system saw the issue go away - they likely did not re-enable SIP, or did so after their installation of ruby succeeded.

Regarding the architecture differences: The arm64/arm64e is one of the first things I noticed which is why I asked @hsbt to run those commands and compare with my system, which also shows clang built for arm64e. Despite the difference in binary format, I assumed the issue might be something else since rbenv install 3.1.3 still works for others who have an arm64e clang build. Now it makes total sense why it "works" (it doesn't really!) Thanks again for confirming this!

Your theoretical workaround definitely makes sense - I had wondered myself if clang had been built against arm64 that maybe this error wouldn't happen since there wouldn't be an arm64/arm64e incompatibility. I wasn't sure how to test this, so thank you for providing a potential way to do this.

Having said all of this, if with SIP enabled (which is the default on all macOS systems) the DYLD_* variables are silently dropped, it means that dropping the DYLD_INSERT_LIBRARIES variable from the configure script is a completely safe solution - no out-of-the-box, SIP-enabled systems will ever be able to read it. And even if they did, as @kateinoigakukun mentioned, it would immediately lead to an arm64/arm64e architecture incompatibility with /usr/bin/clang (arm64e) trying to use an arm64 shared library.