[dev env]: Running tests locally on Mac OS

[ljoss17]

Summary

Currently dependencies for running starknet-integration-tests is done by using nix develop ..#rust from the relayer/ crate. But the behaviour varies depending on the machine used.
This makes running tests locally more tedious than needed.

Status

MacOS Darwin (M1 ARM)

Running nix develop ..#rust will result in the error:

error: builder for '/nix/store/y9qcjbr5sb85pfaz7l3a76xc8fp0qc7g-cairo.drv' failed with exit code 101;
       last 10 log lines:
       >       sudo dnf install pkgconf perl-FindBin perl-IPC-Cmd openssl-devel
       >       # On Alpine Linux
       >       apk add pkgconf openssl-dev
       >
       >   See rust-openssl documentation for more information:
       >
       >       https://docs.rs/openssl
       >
       >   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
       > warning: build failed, waiting for other jobs to finish...
       For full logs, run 'nix log /nix/store/y9qcjbr5sb85pfaz7l3a76xc8fp0qc7g-cairo.drv'.
error: 1 dependencies of derivation '/nix/store/fgh7wflf12gbr3rvyk247srdxa56w5wa-nix-shell-env.drv' failed to build

MacOS Darwin running Debian ARM VM

Running nix develop ..#rust will result in the error:

error: hash mismatch in fixed-output derivation '/nix/store/jx0c40df9756sqqmjxhvfxc7c9i1m4yn-osmosis-v27.0.1-go-modules.drv':
         specified: sha256-wiEixpZPbnwMdhyNrQvHz3cLZF/GXJRa7ho0YaAnVuc=
            got:    sha256-8ArPPTt7gTRrJNTxoaHweJ7JjPT6JSGthn6RqScC6yo=
error: 1 dependencies of derivation '/nix/store/n1m8wbgnfligzivi1qycbpx1ylpf00wh-osmosis-v27.0.1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/gv4llkw4sw856v13jkbfakj9jszkmqb1-nix-shell-env.drv' failed to build

MacOS Darwin running Debian AMD VM

error: hash mismatch in fixed-output derivation '/nix/store/ib00x6gscfs9crcsijl5lr8pqpxmv18r-osmosis-v27.0.1-go-modules.drv':
         specified: sha256-wiEixpZPbnwMdhyNrQvHz3cLZF/GXJRa7ho0YaAnVuc=
            got:    sha256-8ArPPTt7gTRrJNTxoaHweJ7JjPT6JSGthn6RqScC6yo=
error: 1 dependencies of derivation '/nix/store/2k0afs890lcam8pnyah14iq5q7h0yxri-osmosis-v27.0.1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/ckviyyyj0z9xg2r3r6nyv5b8a2713jk7-nix-shell-env.drv' failed to build

Linux (Arch) AMD

No issues

[ljoss17]

It seems that the hash for Osmosis differs from one Mac VM to another since I still get the hash mismatch if using this fork https://github.com/Farhad-Shabani/cosmos.nix/tree/update-osmosis-vendorhash but @Farhad-Shabani doesn't get the hash mismatch with the fork

[ljoss17]

Tests which do not require Osmosis can be run on MacOS using a Linux VM with ARM by running: nix shell ..#cargo-nextest ..#starknet-devnet ..#wasm-simapp -c cargo nextest run --test-threads=2

[Farhad-Shabani]

It seems that the hash for Osmosis differs from one Mac VM to another since I still get the hash mismatch if using this fork https://github.com/Farhad-Shabani/cosmos.nix/tree/update-osmosis-vendorhash but @Farhad-Shabani doesn't get the hash mismatch with the fork

@soareschen, do you have any idea what's causing this?

[soareschen]

Hmm one possibility might be that one of the git tag on Osmosis or its dependencies had been force pushed, and the local git cache for Nix becomes stale. But Nix should have caught any inconsistency somewhere, unless you have a locally updated flake.lock file.

In any case, you could try running nix-collect-garbage to clear everything from the Nix store, and then build again. Alternatively, we should probably try to update the Osmosis version on Cosmos.nix, and see if the problem goes away.

[ljoss17]

Tried running nix-collect-garbage the issue persists.

Let's wait for the Osmosis release with the Wasm fix and then update Cosmos.Nix to see if this fixes the issue then

[soareschen]

It seems like different Nix versions might also produce different hashes from fetchgit due to bugs in Nix. You might also want to try updating to the latest Nix, particularly if it only fails on your computer but not on CI.

[ljoss17]

The CI seems to be running Nix 2.22.1, https://github.com/informalsystems/ibc-starknet/actions/runs/12948587943/job/36117548337#step:3:321
If I isolate the issue, which is building Osmosis on a Linux VM from a MacOS, when running nix build .#osmosis I get the following:

On Linux VM with nix (Nix) 2.21.1:

error: hash mismatch in fixed-output derivation '/nix/store/jx0c40df9756sqqmjxhvfxc7c9i1m4yn-osmosis-v27.0.1-go-modules.drv':
         specified: sha256-wiEixpZPbnwMdhyNrQvHz3cLZF/GXJRa7ho0YaAnVuc=
            got:    sha256-8ArPPTt7gTRrJNTxoaHweJ7JjPT6JSGthn6RqScC6yo=

On Linux VM with nix (Nix) 2.22.1:

error: hash mismatch in fixed-output derivation '/nix/store/jx0c40df9756sqqmjxhvfxc7c9i1m4yn-osmosis-v27.0.1-go-modules.drv':
         specified: sha256-wiEixpZPbnwMdhyNrQvHz3cLZF/GXJRa7ho0YaAnVuc=
            got:    sha256-8ArPPTt7gTRrJNTxoaHweJ7JjPT6JSGthn6RqScC6yo=

So same hash for both versions.

But if I run it on MacOS directly with nix (Nix) 2.18.1 I don't get any errors

[ljoss17]

Ok, tried out building Osmosis from Cosmos Nix and had the option to configure the 'trusted-public-key': do you want to allow configuration setting 'trusted-public-keys'.

If I don't I get the same hash mismatch, but if I do I end up being able to build Osmosis. And afterwards I can correctly build Osmosis on ibc-starknet.

@Farhad-Shabani Could you try these steps?

[soareschen]

Ah, then maybe there are some inconsistencies on the artifacts from different Cachix stores. Probably you should also run cachix add ibc-starknet to add the Cachix store for ibc-starknet.

[Farhad-Shabani]

tried out building Osmosis from Cosmos Nix and had the option to configure the 'trusted-public-key': do you want to allow configuration setting 'trusted-public-keys'.
Could you try these steps?

I tried with allowing the trusted-public-key in the cosmos.nix repo, and it worked perfectly. I was able to build osmosis.

After that, I got curious and wondered if it might be something with the Nix version. So, I uninstalled Nix, deleted all the related files, and reinstalled the latest version (2.26.1) on my Orbstack Linux. Then, I ran the relayer's integration tests locally, and everything worked fine with no hash mismatches. I’m starting to think, like Soares, that this might actually be a bug in Nix.