add a tutorial on packaging existing software

[proofconstruction]

Closes #601, or will eventually.

Following (part of) the requirement specification in that Issue, this tutorial currently focuses on the experience of encountering a few common build failures and their solution by passing different attributes to mkDerivation. This has the potential to grow substantially if we want to cover more mkDerivation features in tutorial style rather than a how-to guide, but I think we should implement that elsewhere as a collection of disconnected (or at least narratively-unrelated) recipes, rather than in this tutorial.

This tries to address most points raised in previous discussions in Documentation Team and Learning Journey Working Group meetings, in particular by

• resolving missing dependency errors (requiring use of both buildInputs and nativeBuildInputs),
• writing a (simple) custom installPhase (this project is actually organically "broken", in the sense that its Makefile does not include an install target, which helpfully dispels some of the Nix magic), and
• packaging a non-trivial but uncomplicated project -- icat:
  • arose naturally in Module system deep dive tutorial #645,
  • wasn't already packaged, and
  • is simple enough to use for an intro tutorial

I explain this choice to use this package in the prose by linking to that not-yet-merged tutorial, so this PR currently should not pass CI; we can remove that justification without harming anything

Anticipating one comment: the (current) final section where I walk through preparing our package for contribution to nixpkgs perhaps should be moved, but I chose to include it here because I believe we should always aim to help users of the configuration features (like the majority of NixOS users) build facility with mkDerivation so they can competently use Nix to package their own stuff, as well as growing competent-packagers into contributors and maintainers.

Concerns this doesn't yet address:

• setting cflags, configure & install prefixes, and pkg-config includes
• using lib, dev, or other outputs for missing headers and development libraries (although xorg.libX11.dev is one of the dependencies; maybe the part involving that can be expanded to discuss this briefly?)
• platform-specific dependencies; this icat package is too simple to have issues here, and builds fine on x86_64-linux (NixOS), aarch64-linux (NixOS), and aarch64-darwin

It turns out that the icat @infinisil uses in #645 is actually the kitty feature by the same name; an entirely different project with equivalent functionality

[fricklerhandwerk]

I tried to focus on the overall structure in this first pass.

The overall structure is great, and I see this becoming a very valuable piece of learning material. The two major conceptual issues I'd like to address first are:

1. There are a few superfluous movements.

We should guide learners along the happy path with minimal friction, and only run them into problems where there is something to be learned for reuse (such as reading build errors – I very much like how you progress through the build dependencies). Just show them how to do the right thing immediately. Words of warning, highlighting common user error patterns, and lightweight explanations along with links to reference documentation are fine, but they should complement the text, not be an integral part of the narrative.

2. The narrative style is a bit too verbose for the purpose here (although I find it hard to separate that requirement from my own taste, so take it with a grain of salt).

We're dumping a lot of new stuff on learners, and we should rather use words sparingly, only saying what's needed. Not everyone is fluent in English (in fact, most of the world's population), and a loose style, while it may seem easy-going for you or who you imagine your readers to be, will likely impose a significant cognitive overhead. This may sound like it's in conflict with sounding friendly or compassionate, but the real service we're providing here is to make the substance as easy to understand as possible. I'd rather take the risk of sounding cold or impersonal than hiding the essentials between semantically empty words.

If we skip the Nixpkgs contribution part, we definitely need a "next steps" section that would point to largely non-existent material. What we also should definitely add are pointers to ongoing discussion around package structure and coding conventions, to note that much of that stuff is still in flux. But would already be polish.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-07-20-documentation-team-meeting-notes-65/30876/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-07-31-documentation-team-meeting-notes-68/31127/1

[roberth]

Good expectation management. Some thoughts

• Flaws also include downloading from the internet
• Discovering dependencies by letting the build fail may be a good practice. We don't want to add unused dependencies.

[proofconstruction]

Discovering dependencies by letting the build fail may be a good practice. We don't want to add unused dependencies.

We do this!

Flaws also include downloading from the internet

What do you mean by this?

[infinisil]

Reviewed together in the Learning journey meeting

Take a look at the notes in https://discourse.nixos.org/t/2023-07-20-learning-journey-working-group-meeting-notes-18/30926#reviewing-nixdev-650-3

@zmitchell: Wording can be more concise, a bit too conversational, can make sentences shorter.
@zmitchell: Take a look at https://nix.dev/contributing/style-guide#voice regarding "we" and "you"

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-08-03-learning-journey-working-group-meeting-notes-20/31251/1

[proofconstruction]

In a previous Learning Journey Working Group meeting, we discussed (among other things) the following points, which are either addressed now or I would like clarification/further discussion on:

@zmitchell: Wording can be more concise, a bit too conversational, can make sentences shorter.

I cut this back a little bit, but I think there's little unnecessary prose here, especially after the latest changes.

@zmitchell: Take a look at https://nix.dev/contributing/style-guide#voice regarding "we" and "you"

• Replaced all relevant instances of "we", "our", etc.

• Have a default.nix containing a let pkgs = ...; in { hello = callPackage ./hello.nix { }; } (why attribute? Easier to later add more, good practice, can use the same command as for Nixpkgs)
• Have a hello.nix as in the tutorial, instead of nix-build -E ..., nix-build -A hello

• Updated the tutorial with this workflow.

• Don’t use builtin fetchers for sources (generally, only use builtin fetchers for private sources or Nix expressions (things needed for evaluation))

How should it be done otherwise?

• Then people also wouldn’t run into the builtins.fetchFromGitHub error

• The reader is now directed to use pkgs.fetchFromGitHub as a replacement for builtins.fetchTarball, avoiding the builtins.fetchFromGitHub error.

• Introduce pkgs set and how it interacts with callPackage when first used, not only once lib is needed. Also: Don’t have pkgs in the arguments, bad practice.

Why is this bad practice?

• fakeSha256 not needed anymore, empty hash is supported by recent releases (need to check if the stable Nix version in NixOS 22.11 (2.11) supports this already)

I find the lib.fakeSha256 more explicit, but it's a nice convenience if we don't need it anymore.

[infinisil]

Overall this looks pretty good, nice work! I think we can merge this after the things I mention are addressed

[infinisil]

Looking good to me!

[zmitchell]

I've made suggested edits where I thought they were warranted or made the prose flow better, but there's only one or two places where I think the actual content could be adjusted (using hello.nix vs. default.nix, for example).

Generally I would err on the side of too few words. It's much easier to add words later than it is to condense a paragraph.

[zmitchell]

Approved pending one or two changes

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-08-10-learning-journey-meeting-notes-21/31556/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-08-10-documentation-team-meeting-notes-71/31558/1

[infinisil]

Needs a rebase after #671

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-08-14-documentation-team-meeting-notes-72/31722/1