Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add the permit-generator github action #69

Closed
wants to merge 47 commits into from
Closed
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
25932b2
feat: add the permit-generator github action
hhio618 Sep 15, 2024
f0b5b9d
feat: add permit-generator script [WIP]
hhio618 Sep 23, 2024
36e3245
feat: add the permit-generator script
hhio618 Sep 24, 2024
b2038b0
feat: post permits to github issues
hhio618 Sep 27, 2024
c3b3c8d
fix: few adjustments
hhio618 Sep 27, 2024
798aa5c
fix: add missing envs
hhio618 Sep 29, 2024
e397fe0
refactor: downgrade to ethers@v5
cohow Sep 15, 2024
787979f
fix: add yarnrc file back
cohow Sep 15, 2024
2e61614
chore: change yarn version
cohow Sep 16, 2024
ba0d046
chore: remove packageManager
cohow Sep 16, 2024
a225135
fix: remove the env decoder
hhio618 Sep 29, 2024
f926e74
fix: type/typo fix
hhio618 Sep 29, 2024
1620035
chore: add logs
hhio618 Sep 29, 2024
0ac5357
fix: extend context env type
hhio618 Sep 29, 2024
917ae26
refactor: switch over action core lib
hhio618 Sep 29, 2024
f828cb3
chore: add log
hhio618 Sep 29, 2024
8540751
refactor: add USERS_AMOUNTS env
hhio618 Sep 29, 2024
133090e
chore: add more logs
hhio618 Sep 29, 2024
f4fb78c
Merge branch 'ubiquity-os:development' into development
hhio618 Sep 30, 2024
a7c05c9
fix: supabase wallets query
hhio618 Oct 3, 2024
7191ee8
feat: use workflow_dispatch runId as issueNodeId
hhio618 Oct 3, 2024
011e368
chore: cleanup
hhio618 Oct 3, 2024
81884f6
fix: add missing env X25519_PRIVATE_KEY
hhio618 Oct 3, 2024
04ecb36
test: hard-code fastest provider
hhio618 Oct 3, 2024
f31251d
fix: pipeline data flow
hhio618 Oct 3, 2024
203433f
fix: report request
hhio618 Oct 3, 2024
153dfe4
fix: report request
hhio618 Oct 3, 2024
55423fb
refactor: runId type
hhio618 Oct 8, 2024
52cb2c3
Merge branch 'ubiquity-os:development' into development
hhio618 Oct 8, 2024
d5f79bc
fix: unit-tests
hhio618 Oct 8, 2024
479ce93
chore: cleanup
hhio618 Oct 8, 2024
b945bdc
feat: use precompiled script
hhio618 Oct 8, 2024
41f85f5
refactor: revert script execution command
hhio618 Oct 8, 2024
b147da4
chore: remove unused ncc dep
hhio618 Oct 8, 2024
d19d962
refactor: rename USERS_AMOUNTS to PAYMENT_REQUESTS
hhio618 Oct 12, 2024
6aaa1ce
feat: add step to log permits data
hhio618 Oct 12, 2024
5e858dd
refactor: remove the contributionType
hhio618 Oct 12, 2024
99d2feb
feat: use uuid for nonce
hhio618 Oct 14, 2024
8e5cd8f
chore: fix knip
hhio618 Oct 15, 2024
4aadfc4
chore: add @types/uuid
hhio618 Oct 15, 2024
2db0505
chore: cleanup
hhio618 Oct 15, 2024
ff0d9d9
chore: fix format:cspell
hhio618 Oct 15, 2024
5b20c35
refactor: remove runId
hhio618 Oct 16, 2024
c22e4c2
refactor: simplify the env vars
hhio618 Oct 16, 2024
c78613a
refactor: rem return to kernel
hhio618 Oct 17, 2024
9fe5fda
Merge branch 'ubiquity-os:development' into development
hhio618 Oct 17, 2024
fe3e60a
chore: cleanup
hhio618 Oct 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/permit-generator.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: Permit Generator

on:
workflow_dispatch:
inputs:
users_amounts:
description: "A JSON array containing usernames and associated amounts"
required: true
# example: '[{"user1": 100}, {"user2": 150}]'

jobs:
run:
runs-on: ubuntu-latest
permissions: write-all

steps:
- uses: actions/checkout@v4

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: "20.10.0"

- name: Process permit requests
id: parse
run: |
echo "Received input: ${{ github.event.inputs.users_amounts }}"
npx tsx scripts/github-action-permit-generator.ts permits.txt
shell: bash
env:
X25519_PRIVATE_KEY: ${{ secrets.X25519_PRIVATE_KEY }}
EVM_NETWORK_ID: ${{ secrets.EVM_NETWORK_ID }}
EVM_PRIVATE_KEY: ${{ secrets.EVM_PRIVATE_KEY }}
EVM_TOKEN_ADDRESS: ${{ secrets.EVM_TOKEN_ADDRESS }}
SUPABASE_URL: ${{ secrets.SUPABASE_URL}}
SUPABASE_KEY: ${{ secrets.SUPABASE_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
USERS_AMOUNTS: ${{ github.event.inputs.users_amounts }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename this to payment requests


- name: Report by opening an issue
run: |
export PERMITS=$(cat permits.txt)
curl -X POST \
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
-H "Content-Type: application/json" \
-d "{\"title\": \"Workflow Dispatch Report\", \"body\": \"$PERMITS\"}" \
"https://api.github.com/repos/${{ github.repository }}/issues"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this might make sense to just echo or log in the CI instead

116 changes: 116 additions & 0 deletions scripts/github-action-permit-generator.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
import * as github from "@actions/github";
import { Octokit } from "@octokit/rest";
import { createClient } from "@supabase/supabase-js";
import { createAdapters } from "../src/adapters";
import { Database } from "../src/adapters/supabase/types/database";
import { generatePayoutPermit } from "../src/handlers";
import { Context } from "../src/types/context";
import { PermitGenerationSettings, PermitRequest } from "../src/types/plugin-input";
import { Value } from "@sinclair/typebox/value";
import { envSchema } from "../src/types/env";
import * as fs from "fs";

/**
* Generates all the permits based on the current github workflow dispatch.
*/
export async function generatePermitsFromGithubWorkflowDispatch() {
const runId = github.context.runId;

// These are necessary to ensure the type checks and tests pass.
process.env["NFT_MINTER_PRIVATE_KEY"] = "";
process.env["NFT_CONTRACT_ADDRESS"] = "";
0x4007 marked this conversation as resolved.
Show resolved Hide resolved
const env = Value.Decode(envSchema, process.env);

if (!env.EVM_NETWORK_ID) {
throw new Error("EVM_NETWORK_ID env not provided or empty");
}
if (!env.EVM_PRIVATE_KEY) {
throw new Error("EVM_PRIVATE_KEY env not provided or empty");
}
if (!env.EVM_TOKEN_ADDRESS) {
throw new Error("EVM_TOKEN_ADDRESS env not provided or empty");
}
if (!env.USERS_AMOUNTS) {
throw new Error("USERS_AMOUNTS env not provided or empty");
}

console.log(`Received: ${env.USERS_AMOUNTS}`);
const userAmounts = JSON.parse(env.USERS_AMOUNTS);

// Populate the permitRequests from the user_amounts payload

const permitRequests: PermitRequest[] = userAmounts.flatMap((userObj: { [key: string]: number }) =>
Object.entries(userObj).map(([user, amount]) => ({
type: "ERC20",
username: user,
amount: amount,
contributionType: "custom",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems wrong. I suppose we should pass this in? What is it for again? @whilefoo is it the nft thing? What's the solution in this context?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean contributionType? Yes it was for the NFT. If we are generating ERC20 reward we don't need it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then this line should be removed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
contributionType: "custom",

tokenAddress: env.EVM_TOKEN_ADDRESS,
}))
);

const config: PermitGenerationSettings = {
evmNetworkId: Number(env.EVM_NETWORK_ID),
evmPrivateEncrypted: env.EVM_PRIVATE_KEY,
permitRequests: permitRequests,
runId: runId,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
runId: runId,

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

};

const octokit = new Octokit({ auth: env.GITHUB_TOKEN });
const supabaseClient = createClient<Database>(env.SUPABASE_URL, env.SUPABASE_KEY);

const context: Context = {
eventName: "workflow_dispatch",
config: config,
octokit,
payload: userAmounts,
env,
logger: {
debug(message: unknown, ...optionalParams: unknown[]) {
console.debug(message, ...optionalParams);
},
info(message: unknown, ...optionalParams: unknown[]) {
console.log(message, ...optionalParams);
},
warn(message: unknown, ...optionalParams: unknown[]) {
console.warn(message, ...optionalParams);
},
error(message: unknown, ...optionalParams: unknown[]) {
console.error(message, ...optionalParams);
},
fatal(message: unknown, ...optionalParams: unknown[]) {
console.error(message, ...optionalParams);
},
},
adapters: {} as ReturnType<typeof createAdapters>,
};

context.adapters = createAdapters(supabaseClient, context);

const permits = await generatePayoutPermit(context, config.permitRequests);
await returnDataToKernel(env.GITHUB_TOKEN, "todo_state", permits);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure why you are returning data to kernel if this workflow is manually triggered by someone

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially thought it was meant to be part of the process. Should I remove it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this script is only called manually and not by the kernel

const out = Buffer.from(JSON.stringify(permits)).toString("base64");
fs.writeFile(process.argv[2], out, (err) => {
if (err) {
throw err;
}
});
}

async function returnDataToKernel(repoToken: string, stateId: string, output: object) {
const octokit = new Octokit({ auth: repoToken });
await octokit.repos.createDispatchEvent({
owner: github.context.repo.owner,
repo: github.context.repo.repo,
event_type: "return_data_to_ubiquibot_kernel",
client_payload: {
state_id: stateId,
output: JSON.stringify(output),
},
});
}
generatePermitsFromGithubWorkflowDispatch()
.then((result) => console.log(`result: ${result}`))
.catch((error) => {
console.error(error);
});
7 changes: 5 additions & 2 deletions src/adapters/supabase/helpers/wallet.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,11 @@ export class Wallet extends Super {
throw error;
}

console.info("Successfully fetched wallet", { userId, address: data.wallets?.address });
return data.wallets?.address;
// Check if wallets is an array, if so, return the first element's address
const address = Array.isArray(data.wallets) ? data.wallets[0]?.address : data.wallets?.address;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should only be a single wallet associated per user. Why did you check for arrays

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I keep getting an array here. Even though the docs say wallets should be an object, it’s probably showing up as an array because Supabase treats it like a one-to-many relationship, even if the user only has a single wallet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@whilefoo @rndquu can you concur this is correct?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't you use single() to retrieve only one? The type should be properly deduced.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still, keep getting an array.
Here's the code:

  async getWalletByUserId(userId: number) {
    const { data, error } = await this.supabase.from("users").select("wallets(*)").eq("id", userId).single();
    if (error) {
      console.error("Failed to get wallet", { userId, error });
      throw error;
    }

    console.info("Result: ", { wallets: data.wallets });
    // Check if wallets is an array, if so, return the first element's address
    const address = Array.isArray(data.wallets) ? data.wallets[0]?.address : data.wallets?.address;

    console.info("Successfully fetched wallet", { userId, address: address });
    return address;
  }

Results after run:

Result:  {
  wallets: [
    {
      address: '0x6321286F9B73f427C72e1f9F1bC6b3d25eF06605',
      user_id: 1272158,
      wallet_id: 1
    }
  ]
}
Successfully fetched wallet {
  userId: 1272158,
  address: '0x6321286F9B73f427C72e1f9F1bC6b3d25eF06605'
}

Copy link
Contributor

@whilefoo whilefoo Oct 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it's correct for that to be an array because it's querying an user with wallets as a relation. if you wanted to get just one record you would need to query the wallet table directly

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@whilefoo The code is already in the base branch but isn't working without the changes in this PR. Should I move the hotfix to a separate PR?


console.info("Successfully fetched wallet", { userId, address: address });
return address;
}

async upsertWallet(userId: number, address: string) {
Expand Down
2 changes: 2 additions & 0 deletions src/handlers/generate-erc20-permit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ export async function generateErc20PermitSignature(
issueNodeId = contextOrPayload.payload.issue.node_id;
} else if ("pull_request" in contextOrPayload.payload) {
issueNodeId = contextOrPayload.payload.pull_request.node_id;
} else if (contextOrPayload.config.runId) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
} else if (contextOrPayload.config.runId) {
} else {

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review, removed.

issueNodeId = contextOrPayload.config.runId.toString();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The node ID should not be the CI run ID? This doesnt make any sense to me

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because when you are manually generating a permit with Action workflow dispatch, it's not associated to any issue so you have to use a unique nonce - that could be the run ID or just a uuid

} else {
throw new Error("Issue Id is missing");
}
Expand Down
4 changes: 4 additions & 0 deletions src/types/env.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ export const envSchema = T.Object({
SUPABASE_KEY: T.String(),
NFT_MINTER_PRIVATE_KEY: T.String(),
NFT_CONTRACT_ADDRESS: T.String(),
EVM_NETWORK_ID: T.Optional(T.String()),
EVM_PRIVATE_KEY: T.Optional(T.String()),
EVM_TOKEN_ADDRESS: T.Optional(T.String()),
USERS_AMOUNTS: T.Optional(T.String()),
});

export type Env = StaticDecode<typeof envSchema>;
1 change: 1 addition & 0 deletions src/types/plugin-input.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ export const permitGenerationSettingsSchema = T.Object({
evmNetworkId: T.Number(),
evmPrivateEncrypted: T.String(),
permitRequests: T.Array(permitRequestSchema),
runId: T.Optional(T.Number()),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about this line @whilefoo

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why this is needed

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@whilefoo whilefoo Oct 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes sense but just using uuid to generate a nonce would be sufficient instead of cluttering the config

});

export type PermitGenerationSettings = StaticDecode<typeof permitGenerationSettingsSchema>;