anchor build and anchor test nuke each other's target build caches

[johnny-smitherson]

Description

There is a significant problem when switching between anchor build and anchor test. Each time you switch from one command to the other, the build is nuked.

Also, if you run $ anchor test --skip-local-validator again and again, since internally it builds the main program, then deploys it, then compiles the test, each time it manages to do a full recompile.

Operating system checked:

• Debian Stable (on host),
• Debian Stable (docker guest over linux host),
• Debian Stable (docker guest over windows host) - bug shows in all

Versions checked:

• anchor-cli 0.30.1
• solana: both 1.18 and 2.1.6

Steps to see the bug

Starting from the default anchor init --test-template rust code and running some build & test commands:

• anchor build #1 (256 packages) = 15s
• anchor build #2 (4 packages) = 1s
• anchor test #1 = 15s (anchor build 256 packages) + 3s deploy + 1min (anchor test build 590 packages)
• anchor test #2 = 15s (anchor build 256 packages) + 3s deploy + 1min (anchor test build 590 packages)
• anchor build #3 (256 packages) = 28s

$ anchor build

...
Building [=======>                  ] 83/256:
...
    Finished `test` profile [unoptimized + debuginfo] target(s) in 15.76s

$ anchor build
...
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.82s

$ anchor test --skip-local-validator
...
Building [============>            ] 134/256:
...
    Finished `test` profile [unoptimized + debuginfo] target(s) in 15.76s
...
Building [===>                     ] 108/590:
...
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 14s
...

$ anchor test --skip-local-validator

... it recompiles everything again

$ anchor test --skip-local-validator

... and again.

cargo test-sbf and cargo build-sbf

This bug is not present when following native rust tutorial - after initial compilation, both cargo build-sbf and cargo test-sbf take 1s each, even if they are alternated

Workaround?

First idea is to manually set CARGO_TARGET_DIR to folder1 when running anchor build and to folder2 when running anchor test.

Does NOT work, since anchor does not listen to CARGO_TARGET_DIR when placing so/idl files (#958 #2402). Faking it with a symlink works, but then, the anchor test command somehow invalidates its own folder between its build program and build tests runs.

At this point I believe anchor to be unusable because of this bug - you can't run the tests without waiting 2min every time.

Any ideas/suggestions?

[acheroncrypto]

I've just tried the steps you shared, but I wasn't able to reproduce it (it works on my machine™). I get something similar to the following each time I run anchor test:

Finished `release` profile [optimized] target(s) in 0.20s
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.15s

However, I think this might be happening because of --test-template rust using the cargo test command:

anchor/cli/src/rust_template.rs

Line 657 in 0922dc3

Self::Rust => "cargo test".to_owned(),

anchor build's IDL generation step also runs cargo test, but with the idl-build feature enabled:

anchor/idl/src/build.rs

Lines 147 to 154 in 0922dc3

	let output = Command::new("cargo")
	.args([
	&toolchain,
	"test",
	"__anchor_private_print_idl",
	"--features",
	"idl-build",
	])

Running cargo test with and without a feature might explain the build cache invalidation (starting from scratch still seems strange though).

If this is indeed the reason, you can try the following to get around this problem:

• Pass in --no-idl flag to the anchor test command to avoid IDL generation (after the initial creation, and no API changes were made)

• Use the cargo test-sbf command instead. For example, this is what the new mollusk template (test templates: add mollusk #3352) uses:

  anchor/cli/src/rust_template.rs

  Line 658 in 0922dc3

  Self::Mollusk => "cargo test-sbf".to_owned(),