Feature parity with Stack

[NorfairKing]

I would love to switch away from stack for development, but there are some real and important things missing to do so.
In this issue I'd like to list those with checkboxes, in case feature-parity with stack is something the cabal team would want to achieve:

• cabal test all --no-rerun-tests: Don't rerun test suites that have already passed.
• cabal test all --coverage: Multi-package code coverage reports
• cabal build all --file-watch: File watching. This is arguably not cabal's responsibility because you can do something close-enough with feedback for example, but cabal is haskell-build-process-aware and could do better than feedback can.
• Automatically executing hpack when necessary.
• --pedantic
• Dependency graph

Not available and not desirable IMO:

• GHC installation

---

UPD More ideas from the discussion below:

• Loading multiple packages in cabal repl a la stack repl.
• Better support for snapshots (details: Feature parity with Stack #8605 (comment))
• Docker integration (see stack documentation)

[ulysses4ever]

Would help to have links to respective stack documentation sections.

[NorfairKing]

@ulysses4ever Good idea, I added some links.

[tfausak]

Although I have switched from Stack to Cabal for development, I agree with this list and would like to see these things implemented. I think there are existing issues for some of them:

cabal build all --file-watch

• cabal watch command #5252

Automatically executing hpack

• Module listing discovery (wildcard glob) #5343 (related issue suggesting new feature that hpack currently has)
• Migrate from the .cabal format to a widely supported format #7548 (related issue suggesting a new file format entirely)

[fgaz]

Automatically executing hpack when necessary

• shell hooks for augmenting how cabal-install does various actions? #7394

[chrisdone-artificial]

Regarding feature parity

• Loading multiple packages in cabal repl a la stack repl would also be nice.

• A thing that I get mileage out of with Stack is (1) specifying a Stackage snapshot in one line (rather than a big lock file), and (2) the fact that I can then cherry-pick and pin individual package versions down to the SHA hash. This experience is quite different to the big lock file specifying all dep versions, and the hash pinning allows you to not trust the package server.

• Personally, I think GHC management is a big deal. But it's worth noting that the Haskell community is split into those who use Nix and those who don't. So it's common for a lot of Cabal-using devs to defer certain features (such as package pinning or GHC management) to their Nix configuration, similar to using entr/watchexec instead of --file-watch, but that leaves the other half of the community preferring to stick with Stack!

(Although maybe like @tfausak some people just ignore the few missing features and use Cabal.)

Regarding docs

I'd eagerly read some kind of "cabal for stack enthusiasts" article which explains how to achieve all the things you're used to with stack from a proper stack user's perspective (not a cabal user who's vaguely aware of what stack can do or used it a little bit).

Contributing

I'm afraid I can't contribute any work towards this. I've done my time on Haskell tooling. 😂 Around 2015, I wrote the first commit on Stack and shaped some of its direction, alongside Michael Snoyman, Michael Sloan, Emanuel Borsboom. Tooling is Hard, especially Haskell as it's a moving target. The best outcome would be that the community converges again on one build tool. But it's taken years to get to this point and may take some more years yet!

[fgaz]

Loading multiple packages in cabal repl

• Multiple Home Units support #8238

specifying a Stackage snapshot in one line

• RFC: Cabal support for LTS Snapshots #7556
• [Feature] remote freeze files #6528
• Document Stackage use with remote project config #8083

the fact that I can then cherry-pick and pin individual package versions

• Rest of RFC: Cabal support for LTS Snapshots #7556 (package collection algebra)

Personally, I think GHC management is a big deal

• shell hooks for augmenting how cabal-install does various actions? #7394

[xplat]

I think both automatic hpack execution and auto-expanding Stackage snapshot names to lockfiles are best done via hooks, since they break off nicely into discrete tasks that already have well-known public interfaces. (I admit for hpack specifically I am biased in that I don't like hpack's design decisions and don't want Cabal to endorse it.)

[gbaz]

I am curious how cabal test all --no-rerun-tests would work specifically? It sounds intriguing and a ticket on it would be welcome, especially coupled with some explanation of the semantics of the existing stack feature.

A ticket for multi-package code coverage would also be welcome.

[JakobBruenker]

Not available and not desirable IMO:

• GHC installation

I think I agree, but I do think it would be nice if cabal could somehow select among the installed GHC versions for a particular project, perhaps by interfacing with ghcup.

[merijn]

I think I agree, but I do think it would be nice if cabal could somehow select among the installed GHC versions for a particular project, perhaps by interfacing with ghcup.

GHC by default includes executables with complete version numbers, cabal supports -w to specify a compiler to use, and this can be stored in cabal.project/cabal.project.local files.

Personally, I have a setup where my "default" ghc is first in path, followed by all other installs. This causes ghc to refer to the first (i.e. my default) ghc executable in my path, while version tagged GHCs are found later on path. This way simply writing cabal build -w ghc-8.10.7 will use the relevant GHC and this can be stored in aforementioned config files too.

All that would be needed to point a project to a specific GHC installed via ghcup would be: cabal configure -w path/to/ghc-version and from then on cabal build will automatically use that one.

[JakobBruenker]

@merijn that is good to know, thank you

[NorfairKing]

I am curious how cabal test all --no-rerun-tests would work specifically? It sounds intriguing and a ticket on it would be welcome, especially coupled with some explanation of the semantics of the existing stack feature.

@gbaz Stack keeps track of which test suites have been run since their dependencies have last changed and whether the run was successful (exit code 0, I guess). Then it doesn't rerun the test suite if it's already succeeded and the dependencies haven't changed.

[AshleyYakeley]

Not available and not desirable IMO:

• GHC installation

So, not actual parity, then?

Another stack feature not yet mentioned:

• Docker integration

[santiweight]

I'm currently looking into IDE support for tests. I think that workflow has a lot of similarities, due to needing to report information about the available tests etc.

We have some discussion ongoing at discourse.

I will look into how stack handles this too...

Edit: Commit for --no-rerun-tests support here

[gbaz]

Can someone specify what docker integration is and how it might work / what it does?

[AshleyYakeley]

https://docs.haskellstack.org/en/stable/docker_integration/

[hasufell]

Not available and not desirable IMO:

• GHC installation

I think I agree, but I do think it would be nice if cabal could somehow select among the installed GHC versions for a particular project, perhaps by interfacing with ghcup.

The link has already been posted twice in this thread. Yes, this is easily possible via #7394

It would work the same way stack interfaces with ghcup today:

• https://docs.haskellstack.org/en/stable/yaml_configuration/#ghc-installation-customisation
• https://github.com/haskell/ghcup-hs/blob/master/scripts/hooks/stack/ghc-install.sh

My guess is that it's trivial to implement.

Hooks are a superior design to what stack does today. They allow to implement hpack support, GHC installation, patching of packages, custom toolchain checks, automatic caching of dependencies to S3,. ...
The possibilities are only limited by the information we expose to the hook. Cabal can ship curated hooks, similar to what git does.

[NorfairKing]

Only two things are missing for me:

• --no-rerun-tests
• Automatically running hpack when necessary

See: https://twitter.com/kerckhove_ts/status/1649526134193680384

[ulysses4ever]

@NorfairKing could you please reference documentation for --no-rerun-tests? I fail to find it in the stack manual.

[NorfairKing]

@ulysses4ever
I don't know if it's in the docs, but this comment found the relevant commit:
#8605 (comment)

[ulysses4ever]

@NorfairKing fair, thank you. Do you know if it skips succeeded tests when only the testsuite itself changes? Or the library under test too? If it's the latter, it can bring false negatives, right? (Your change to the library broke a test that used to work but you don't notice that because the test is not rerun.)

[NorfairKing]

@ulysses4ever Every reverse-dependenty test suite is rerun when code changes, so there won't be false positives but sometimes too many test-runs.

[hasufell]

Automatically running hpack when necessary

Unlikely to happen, IMO.

But once the hooks design is actually generically implemented, you can run whatever you want before a build.

[NorfairKing]

@hasufell Where can I follow up on that progress?

[hasufell]

#7394

[TeofilC]

Dependency graph

I'm guessing this is referring to stack dot? You can do something similar with cabal-plan

[philderbeast]

Stack warns about unused package flags, #7813 (comment)

[NorfairKing]

Another cabal bug that stack hasn't:
cabal build all does not build all.

https://x.com/kerckhove_ts/status/1858455925767500021

[fgaz]

Eh, I disagree on it being a bug. If I choose to disable tests/benchmarks, I want them disabled. We could enable tests and/or benchmarks by default though (didn't we already do that?)

[ulysses4ever]

I agree cabal build all not building all by default is a huge UX flaw. @fgaz if you want to disable some of it, you should be able to do so, of course, but this is a separate concern: e.g. if you add tests: False in your config or your project file, then, od course, cabal build all shouldn't build tests. The question here (as you rightly point out) is about the default behavior --- the one that applies when there are no explicit instructions from the user.

And no, we didn't enable tests (#7883) and/or benchmarks by default. The last development on this front, I think, was enabling docs by default when the user calls cabal haddock. That didn't go down well, though: #8725 Doing the same for cabal test led to another mega-thread but in the end, it looked like there was a path forward, and just no one bothered implementing it.

I wonder if the easiest (and reasonable) way to improve that would be to generate the default config with tests: True, benchmarks: True. We can mention in the docs that to avoid building tests/benches for a particular project you can always cabal configure --disable-tests --disable-benchmarks. The downside of this cheap solution would be that even when selecting a particular component (cabal build blah), cabal will try to solve for everything (#4087).

[tbidne]

Another (backwards-compat) solution would be to introduce a new target everything that solves and attempts to build literally all targets. This arguably would retroactively justify all's current behavior, analogous to -Wall vs -Weverything.

[hasufell]

This arguably would retroactively justify all's current behavior, analogous to -Wall vs -Weverything.

Let's not pretend that's good design. It's atrocious and after 10 years of Haskell I'd fail every interview that asks me what the difference of both GHC flags are.

[philderbeast]

With #9744, cabal-install will be able to select and show a subset of targets . The stack ide targets command was added to stack in Jun 2016.

[philderbeast]

As for the comments in this thread about cabal build all building or not building tests and benchmarks and whether "all" includes those and whether cabal build should be equivalent to cabal build all. All of this is much easier to explore with #9744. Here's an example from this repository (cabal target is the same as cabal target all):

$ cabal run cabal-install:exe:cabal -- target
...
Fully qualified target forms:
 - Cabal-QuickCheck:lib:Cabal-QuickCheck
 - Cabal-described:lib:Cabal-described
 - Cabal-hooks:lib:Cabal-hooks
 - Cabal-syntax:lib:Cabal-syntax
 - Cabal-tests:lib:Cabal-tests
 - Cabal-tests:test:check-tests
 - Cabal-tests:test:custom-setup-tests
 - Cabal-tests:test:hackage-tests
 - Cabal-tests:test:no-thunks-test
 - Cabal-tests:test:parser-tests
 - Cabal-tests:test:rpmvercmp
 - Cabal-tests:test:unit-tests
 - Cabal-tree-diff:lib:Cabal-tree-diff
 - Cabal:lib:Cabal
 - buildinfo-reference-generator:exe:buildinfo-reference-generator
 - cabal-benchmarks:test:cabal-benchmarks
 - cabal-install-solver:lib:cabal-install-solver
 - cabal-install-solver:test:unit-tests
 - cabal-install:exe:cabal
 - cabal-install:lib:cabal-install
 - cabal-install:test:integration-tests2
 - cabal-install:test:long-tests
 - cabal-install:test:mem-use-tests
 - cabal-install:test:unit-tests
 - cabal-testsuite:exe:cabal-tests
 - cabal-testsuite:exe:setup
 - cabal-testsuite:exe:test-runtime-deps
 - cabal-testsuite:lib:cabal-testsuite
 - cabal-validate:exe:cabal-validate
 - solver-benchmarks:exe:hackage-benchmark
 - solver-benchmarks:lib:solver-benchmarks
 - solver-benchmarks:test:unit-tests
Found 29 targets matching all.

Tests are included above because cabal.project enables them. Let's remove that¹ and try again :

$ git diff
diff --git a/cabal.project b/cabal.project
index a2075cfdc..c2742cd46 100644
--- a/cabal.project
+++ b/cabal.project
@@ -3,7 +3,6 @@ import: project-cabal/ghc-latest.config
 import: project-cabal/pkgs.config
 import: project-cabal/constraints.config
 
-tests: True

$ cabal run cabal-install:exe:cabal -- target
...
Fully qualified target forms:
 - Cabal-QuickCheck:lib:Cabal-QuickCheck
 - Cabal-described:lib:Cabal-described
 - Cabal-hooks:lib:Cabal-hooks
 - Cabal-syntax:lib:Cabal-syntax
 - Cabal-tests:lib:Cabal-tests
 - Cabal-tree-diff:lib:Cabal-tree-diff
 - Cabal:lib:Cabal
 - buildinfo-reference-generator:exe:buildinfo-reference-generator
 - cabal-install-solver:lib:cabal-install-solver
 - cabal-install:exe:cabal
 - cabal-install:lib:cabal-install
 - cabal-testsuite:exe:cabal-tests
 - cabal-testsuite:exe:setup
 - cabal-testsuite:exe:test-runtime-deps
 - cabal-testsuite:lib:cabal-testsuite
 - cabal-validate:exe:cabal-validate
 - solver-benchmarks:exe:hackage-benchmark
 - solver-benchmarks:lib:solver-benchmarks
Found 15 targets matching all.

Footnotes

1. cabal target --disable-tests would have had the same effect. ↩