Add TypeScript and general refactor

[andreogle]

Motivation

If we want this package to be the "single source of truth" for API3 related projects, then I think it needs to be as robust as possible. However, the current setup has some issues. Specifically:

1. Lack of types or any kind of safety
2. It can only be run in a Node.js environment since it depends on fs
3. It's easy to mess up the JSON files in the chains/ directory
4. No tests

This PR is an attempt to address points 1-3. Tests can be added in a followup PR if you're happy with this approach

API changes

The currently exported functions have been changed to the following:

import {
  CHAINS, // The static list of supported chains. Replaces `getChains()`
  getChainByAlias, // Replaces `getChain()`
  hardhatConfigNetworks, // The same
  etherscanNetworks, //  The same
  getEnvVariables, // Replaces `writeEnvFile()`
  Chain, // and other types
} from '@api3dao/chains';

Scripts

generate-chains

This script is intended to be run whenever one or more JSON files are changed in the chains/ directory. It reads all of the files and creates a new TS file at src/generated/chains.ts that has the static CHAINS export. This provides type safety to users and allows the package to be included in any environment.

It is run as part of yarn build but can also be run with yarn dev. yarn dev will watch the directory for changes and re-write the file immediately.

validate-chains

This script uses zod defined schemas to validate each of the JSON files in chains/. For example, we can now validate that certain fields are present and conform to required formats (URLs, numbers etc.) If there are errors, the script exits with status 1. All errors are logged to stdout using console.log.

This script is run automatically before publishing to NPM and as part of (newly added) CI.

write-env-variables

The old writeEnvFile has been extracted to a script to allow the package to be used in all environments. The new getEnvVariables function allows for the list of environment variables to be created, while the script actually writes the file.

Usage: yarn env:write --path .env

Considerations

1. This is a fairly big refactor and would require a new major version if accepted
2. I'm not too familiar with the various options for building TypeScript packages, so I copied the tsconfig.json files from https://github.com/api3dao/promise-utils. I also tried publishing under my own account and everything seemed to work. See: https://github.com/andreogle/chains
3. I will update the README if the PR is accepted.
4. I still want to add tests

[Siegrift]

Left some minor comments. Overall I really like this change.

[Siegrift]

Btw. thanks for the detailed README. Couple of my remarks to it.

4. No tests

I don't see this like a huge deal. The idea behind this repo is to preserve the source of truth for chain informations. I'd be lenient with tests (I am not really sure what you would want to test) but instead validate as much as possible.

This script is run automatically before publishing to NPM and as part of (newly added) CI.

nit: pre-push hook would be nice as well.

write-env-variables

I'd also use this to populate env.example. Also the script is a bit dangerous in that it overrides the already existing .env secrets. It might be good idea to make it smarter and only append/remove the new/unnecessary ones.

[andreogle]

I've updated the README.md file with the API changes. Let me know what you think. It could still be improved further, but it at least describes the minimum.

Also, the validate:providers script is proving to be pretty flakey. I'm not sure if this should be included in CI.

[bbenligiray]

Say I add a particular version of @api3/chains to airnode-protocol-v1. I don't understand the intended flow to generate example.env in airnode-protocol-v1.

I think validate:providers is misnamed in that what it does is very different to what validate:chains does. I think it should be called something else and not be added to CI. This repo should export multiple providers per chain at some point (one suitable for contract deployment, one suitable for RRP for ChainAPI to use as the default public provider, etc.) and @bdrhn9 should maintain the provider URLs along with implementing the necessary monitoring (either within this CI or externally).

[andreogle]

Say I add a particular version of @api3/chains to airnode-protocol-v1. I don't understand the intended flow to generate example.env in airnode-protocol-v1.

Do you mean that you want chains at a certain point in time/package version? I didn't realise that this was a requirement. I assumed that the latest package version was always going to be preferable.

I guess there are several options

1. Pull the repo and checkout the release commit. Run yarn env:write --path .env
2. Continue importing the specific package version, but instead use getEnvVariables() and do the writing (fs.writeFileSync()) in the importing project.
3. Figure out the npx command for something like npx @api3/[email protected] env:write --path .env. I haven't looked into this at all yet. Not sure how feasible it is.

I still feel that a script is better suited here and the workarounds above can be used if needed.

I think validate:providers is misnamed

Sure I'm happy to change the name. The previous name was check-provider - I can use that or anything similar (providers:check?, providers:ping?). I'll also remove it from CI

[bbenligiray]

Ah okay, I hadn't noticed that getEnvVariables() was exported. I don't like the fact that all depending projects will duplicate that piece of code but I don't see any better way.

providers:ping sounds good. I think we'll have other provider scripts in the future that will do other stuff (for example "calculate the average block time over the last month and update the respective chains object" for #5).

[bbenligiray]

Instead of chains:rename, maybe this should be checked by validate:chains and we should expect the files to be renamed manually. I feel the need to manually check all changes whenever I run that script and it makes any changes anyway.

[bbenligiray]

#1 (comment) can also be an issue

[andreogle]

Ah okay, I hadn't noticed that getEnvVariables() was exported. I don't like the fact that all depending projects will duplicate that piece of code but I don't see any better way.

Yeah it's maybe not ideal, but dropping the fs & path dependencies are more important imo

I've updated the validate:chains script to also validate that .json file names are the respective chain's alias and removed the rename script

[andreogle]

I think this is ready to be merged now. Let me know if you want me to hit the button. I assume we'll want to publish v2.0.0 for this too

[bbenligiray]