Add w3 CLI script for w3s driver #17
Conversation
There was a problem hiding this comment.
LGTM either way. We can even keep this script on this separate branch until we figure this out better.
re: distributing to catalyst, have you given a try on tools like pkg? These should create a hopefully distributable binary to run the node script. Might be simpler than adding node support in catalyst container (if we don't have that yet)
drivers/w3s.go
Outdated
| return nil, fmt.Errorf("invalid UCAN proof format: %s", err) | ||
| } | ||
| cmd.Env = os.Environ() | ||
| cmd.Env = append(cmd.Env, fmt.Sprintf("W3_PRINCIPAL_KEY='%s'", W3sUcanKey)) |
There was a problem hiding this comment.
I wonder if there is an opp to simplify how this key is configured. I see that right now the flow is:
- Pass value in a flag for a catalyst-api
- Set the package var
W3sUcanKeywith the value - Read
W3sUcanKeyand set it as an env var when calling the livepeer-w3 script
An alternative might be:
- Set
W3_PRINCIPAL_KEYin the environment that catalyst-api is running in i.e. in the Docker container - livepeer-w3 just reads the
W3_PRINCIPAL_KEYenv var value directly without any additional value passing which reduces the # of code paths that have to be aware of this value
I see we've already merged the relevant catalyst-api PRs so maybe consider whether this might be a worthwhile simplification separately.
There was a problem hiding this comment.
My initial thinking was that not storing the private key in the environment variable is more secure because even if you access the container, you still cannot access it.
However, now I think you're right because anyway we pass the key in the env variable to catalyst. So, maybe let's get rid of the Catalyst flag and just use the private key passed in the W3_PRINCIPAL_KEY. Thanks for this comment @yondonfu
I tried |
fix livepeer/catalyst#383
kudos to @red-0ne for starting this work in his PoC PR: livepeer/catalyst-api#387
Context
In the w3s driver, we use w3 CLI to interact with web3.storage. With the current implementation, we use the credentials created by
w3CLI (stored in the local FS). In other words, currently, we don't use the credentials provided by the user, which is not correct and not what we need for web3.storage integration.What we want is to pass the credentials (UCAN private key configured with the catalyst-api flag and the delegation proof passed by the user) into
w3CLI and use them to interact with the web3.storage service. There are, however, two issues:w3CLI does not provide a way to import the private key (though it allows importing deletion proofs)w3CLI stores its credentials (and its state) in the local FS; at the same time we need multi-tenancy because different users need to use w3 with different proofs.Solution
This PR adds a JS script that can be used in the same way as
w3CLI for our use cases, but it allows passing private key and poof as env variables. So, instead of executingw3 can store add <car-file>, we will execute the following command.Additional comments:
Dockerfileto install node andnpmdependencieslivepeer/catalystorlivepeer/catalyst-apiAlternative solutions
1. Implement this as a feature in
w3CLIThe best for us would be if this is actually implemented in the
w3CLI. I've sent a PR to their repo (storacha/w3cli#42). I'm not sure it'll be merged, but let's try it.2. Import credentials into
w3CLIWe could write a script (@red-0ne already wrote it here) and then use w3 profiles for multi-tenancy.
Nevertheless, I think that solution is worse because it would still mean maintaining our own "import" script and additionally:
3. Fork
w3CLIWe could fork the w3cli repo and maintain our own version, but I'm not sure it's worth it.
4. Move this script to a separate repo
Instead of keeping this script in
livepeer/go-tools(orlivepeer/catalyst/livepeer/catalyst-api), we could create a separate repo and store it as a complete JS module (withpackage.json, etc.).