Coordinate macro invocations

[NyxCode]

I initially just wanted to share a few interesting crates i found, but this ended up as a place for me to explore different approaches of how we could achieve coordination between macro invocations, possibly as an alternative to the CLI.

For a while, we've had #304 and #334 open.

I have to admit that I'm somewhat hesitant to get them merged at this time.
For one, I don't really like CLIs. It's an separate tool, yet an other executable users need to install, keep up to date, etc. I feel like the problem we're trying to solve is simple enough that we should get by without one. At least that's my hope.
This doesn't mean I definitely don't want the CLI, but I'm still curious if we can do without.

The main motivation for the CLI is to coordinate between macro invocations. That would allow

• (non-hacky) merging of exports into one file
• generating an index file
• using a global configuration (e.g for type overrides)

The old approach

The obvious solution to all this is to require the exports to be done in one place.

ts_rs::export! {
   use index "index.ts",
   use override u64 = "number",
   Type1, Type2, Type3 => "file1.ts",
   Type4, Type5 => "file2.ts",
};

Very early versions had this feature, and we could bring it back in a backwards-compatible fashion.

A problem I had with this in my projects was visibility. All modules and types ended up public, just so I could refer to them in one place.
That's not a huge deal, but I strongly felt that a tool to generate bindings shouldn't force me to give up encapsulation of modules.

Also, our #[ts(export)] is just so damn convenient, and i think it would be sad to make any of this more complicated for our users.

The manual approach

To keep encapsulation, modules could export a pub fn export_bindings(ctx: &mut ExportContext), but not their actual types.
That could look something like this:

// main.rs
#[test] 
fn export() { 
  let mut ctx = ts_rs::begin_export();
  ctx.override::<u64>("number");
  ctx.index("index.ts");

  mod1::export_bindings(&mut ctx);
  mod2::export_bindings(&mut ctx);
}

// mod1.rs
pub fn export_bindings(ctx: &mut ExportContext) {
  ctx.export::<MyType>();
  
  nested_module::export_bindings(&mut ctx);
}

This keeps encapsulation, but is extremely verbose - Especially compared to how convenient our current #[ts(export)] is!
Maybe a better API and macros could reduce this boilerplate. But I feel like having any boilerplate might be unacceptable to just get some bindings.

The magic approach

There are crates like ctor, inventory, and linkme, which seem very promising although somewhat magical (and hacky) to me.

ctor

ctor allows to run code "before main".
That would, for example, allow us to do shenanigans like this, allowing users to define a global config within rust:

// in users' code
ts_rs::configure!(index = true, override u64 = "number", ...);

// within ts_rs:
pub static CONFIG: Mutex<Config> = Mutex::new(Config::default()); // default config - no overrides, no index
macro_rules! configure { // magic macro that emits a function which would run before `main` or any tests
  (.......) => {
    #[ctor]
    fn foo() {
      let config = CONFIG.lock();
      config.index = true;
      config.overrides.push(.....);
    }
}
impl<K, V> TS for HashMap<K, V> { // during runtime, we could then read from that config
  fn name() -> String {
    if let Some(override) = CONFIG.lock().get_override(...) { ... }
    ...
  }
}

linkme

I do not fully understand how linkme works, but it can "collect" statics scattered around the codebase into one single collection.
We would define this static collection within ts_rs:

#[linkme::distributed_slice]
static EXPORTED_TYPES: [fn(&mut ts_rs::Config)];

And modify our derive macro to generate the following, which would insert something (a function pointer to the export function, in this example) into that collection:

impl TS for MyType { .. }

#[distributed_slice(ts_rs::EXPORTED_TYPES)]
static EXPORT_ME: fn(&mut ts_rs::Config) = MyType::export_with_config;

Now, we would need some entrypoint to start exporting.
Users could define a #[test] fn export() { ts_rs::export_everything(); }, or do that in some other fashion.
Or we could keep generating export #[test]s for every type, but making sure only the first one runs and all subsequent tests do nothing.

Whatever the entrypoint would be, there'd only be one call to ts_rs::export_everything().
That function would then iterate through the ts_rs::EXPORTED_TYPES collection, exporting all the types at once.

linkme could allow users to globally define the config using a macro as well, similar to the ctor example above.
That macro would also put something into a global collection, which we'd then read when exporting.

The more that I think about this, the more excited I get about linkme. But there are still concerns.
For one, linkme manually supports a variety of platforms. The big ones are supported, but WebAssembly does not seem to be amongst them currently.
The whole library seems to work by somewhat abusing the linker, which does seem a bit hacky (though I don't understand it enough to make that judgement). If we only had this capability in the standard library..

inventory

inventory seems like it would provide us with the same affordances as linkme, though possibly with less concerns about platform support (and general hackiness).

[NyxCode]

Some resources shared by Helix on the Rust discord:

• https://internals.rust-lang.org/t/global-registration-a-kind-of-pre-rfc/20813/4
• Tracking Issue for #[global_registration] rust-lang/rust#125119

Other resources I've found on the topic:

• https://donsz.nl/blog/global-registration/

[gustavo-shigueo]

Hey @NyxCode, I seem to remember an issue asking about inventory support and the author offering to implement it, maybe we can ask him to make a rough prototype for us to take a look?

[gustavo-shigueo]

Hey @NyxCode, I seem to remember an issue asking about inventory support

Found it, #370

[gustavo-shigueo]

I have to admit that I'm somewhat hesitant to get them merged at this time.
For one, I don't really like CLIs. It's an separate tool, yet an other executable users need to install, keep up to date, etc

About the CLI, I believe that if we get it right before we merge, it's probably not gonna be needing to many updates, as, at the end of the day, all it does is gather some config flags and turn them into a cargo test invocation, optionally merging the output after. If we manage to get this right, it should only ever need updating if we add new config options, and even then, we could add support to giving cargo test custom arguments, like in cargo run <ARGS_FOR_CARGO> -- <ARGS_FOR_THE_EXECUTABLE>, we could have ts-rs <TS_RS_ARGS> -- <CARGO_TEST_ARGS>