crystal tool dependencies: Displaying compile-time source dependencies

[HertzDevil]

The require order between the same files can change the semantics of the program even when the file contents remain unchanged, for example:

• If a file defines class T, it must be required before any other file that defines class T::U, otherwise T would become a module;
• If multiple overloads of the same method are not stricter than one other, and they are defined across multiple files, the require order determines the overload order;
• Top-level code and macros in the prelude are executed according to the require order.

It seems there is no way to extract this information from the compiler, so I propose a new tool dependencies that shows the require chain and perhaps files loaded in macros as well. The default output format would be a tree of files similar to hierarchy:

- test.cr
  |
  +- src/prelude.cr
     .   from prelude
     |
     +- src/crystal/once.cr
     |      from `require "crystal/once"`
     |      at src/prelude.cr:11:1
     |
     +- src/lib_c.cr
     |      from `require "lib_c"`
     |      at src/prelude.cr:12:1
     |
     +- src/macros.cr
     |      from `require "macros"`
     |      at src/prelude.cr:13:1
     |
     +- src/object.cr
     |      from `require "object"`
     |      at src/prelude.cr:14:1
     |
     +- src/comparable.cr
     |      from `require "comparable"`
     |      at src/prelude.cr:15:1
     |
     +- src/exception.cr
     |  .   from `require "exception"`
     |  .   at src/prelude.cr:19:1
     |  |
     |  +- src/exception/call_stack.cr
     |  |  .   from `require "./exception/call_stack"`
     |  |  .   at src/exception.cr:1:1
     |  |  |
     |  |  +- src/exception/call_stack/libunwind.cr
     |  |  |  .   from `require "./call_stack/libunwind"`
     |  |  |  .   at src/exception/call_stack.cr:3:1
     |  |  |  |
     |  |  |  +- src/lib_c/x86_64-linux-gnu/c/dlfcn.cr
     |  |  |  |      from `require "c/dlfcn"`
     |  |  |  |      at src/exception/call_stack/libunwind.cr:1:1
...
     |  |
     |  +- src/system_error.cr
     |         from `require "system_error"`
     |         at src/exception.cr:2:1
...
     +- src/char.cr
     |  .   from `require "char"`
     |  .   at src/prelude.cr:34:1
     |  |
     |  +- src/comparable.cr (already required)
     |  |  .   from `require "comparable"`
     |  |  .   at src/char.cr:1:1
...

Files already required do not branch again, so that the tree would always be finite.

Some additional output formats might be a flattened list of only the file names, a JSON, or a list of Makefile dependencies.

[straight-shoota]

I'm wondering if the tree format could be more succinct. For example, the pathname of each require location is equivalent to the name of the parent item.
Also I'm not sure if the exact require expression needs to be shown.

Maybe this could be an extended format, but the default tree view would only show the pathnames themselves?

Additionally, I'd also like to have a plain list view. Possibly sorted by pathname instead of require order.

[straight-shoota]

I've started this feature with a very basic implementation and plan to make progressive enhancements against the feature branch (PR: #13631).

At the current state, this prints all required files.
This usually means a lot of noise. The prelude alone is 335 files. Anything else you require explicitly from stdlib or shards will add lots more files. Usually you only care about local code, not library code.

So #13632 adds filtering. By default, file paths in CRYSTAL_PATH are filtered out.
You can opt into showing more paths via the --include option. For example --include lib would show requires of files in the lib folder.

The output can still show you the top of a filtered tree. For example if there's a require "baz" anywhere, it will show up, but all nested requires are hidden. This allows you to see where which dependency is pulled in.

Example:

src/prelude.cr (filtered)
test/foo.cr
  test/bar.cr
    test/foo.cr (duplicate skipped)
lib/baz/src/baz.cr (filtered)

I think this information is usually not that helpful as you're mostly interested to see files from the project local source tree. So I think the default should be cleaner and have all lines annotated with parentheses hidden. This additional information can be shown with --verbose flag.