Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] ci-mgmt onboarding #1474

Closed
wants to merge 47 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
a43ddb6
Remove unnecessary codegen
blampe Nov 5, 2024
8dc029b
Track .pulumi/version
blampe Nov 5, 2024
3d4a6f8
Customize default make targets
blampe Nov 5, 2024
0cd8790
Rename existing custom workflows so ci-mgmt doesn't wipe them
blampe Nov 5, 2024
d76c4c4
Initial ci-mgmt config
blampe Nov 6, 2024
6def75a
Generate initial workflows
blampe Nov 6, 2024
498df3d
remove upstream references
blampe Nov 6, 2024
e9804ed
remove custom license check
blampe Nov 6, 2024
45e2312
bump tests
blampe Nov 6, 2024
906fde8
try renaming to get tests to run
blampe Nov 6, 2024
8f6fc85
add missing license
blampe Nov 6, 2024
f5cbaae
remove more upstream references
blampe Nov 6, 2024
87c2b40
Revert Makefile to master for now
blampe Nov 6, 2024
ba6776d
Add some targets for ci-mgmt compatibility
blampe Nov 6, 2024
1ec9fac
make provider_no_deps -> make provider
blampe Nov 6, 2024
4c417bb
add missing .actionlint.yml
blampe Nov 6, 2024
5454fba
fix install_plugins
blampe Nov 6, 2024
899773e
make test_provider was never getting called?
blampe Nov 6, 2024
d283d1c
dist targets
blampe Nov 6, 2024
2b7ba98
Revert "Remove unnecessary codegen"
blampe Nov 6, 2024
d22bfe7
fix windows target
blampe Nov 6, 2024
913667c
add install_sdks target
blampe Nov 6, 2024
f3dbb28
use dist for provider upload
blampe Nov 6, 2024
c92c040
Don't rebuild SDK
blampe Nov 6, 2024
83a79b9
fix shard invocation
blampe Nov 6, 2024
d6f1200
set profile credentials as part of the test instead of the workflow
blampe Nov 6, 2024
4037a01
fix codecov
blampe Nov 6, 2024
d2aeb0c
remove noparallels
blampe Nov 6, 2024
4f41807
reduce parallelism
blampe Nov 6, 2024
aa648f6
free disk space before build
blampe Nov 7, 2024
71b630a
Don't set ALT_AWS_PROFILE
blampe Nov 7, 2024
49ad44e
don't depend on default profile
blampe Nov 12, 2024
8564594
Use --profile instead of AWS_PROFILE
blampe Nov 12, 2024
047c829
Merge branch 'master' of github.com:pulumi/pulumi-eks into blampe/ci-…
blampe Nov 12, 2024
d7e2c2c
revert workflows
blampe Nov 15, 2024
dbec150
Merge branch 'master' of github.com:pulumi/pulumi-eks into blampe/ci-…
blampe Nov 15, 2024
3f0ba46
regenerate workflows
blampe Nov 15, 2024
9623791
update .pulumi/version
blampe Nov 15, 2024
203ddbc
fix test directory
blampe Nov 15, 2024
47c86a8
shard target
blampe Nov 15, 2024
0aa1daa
hm
blampe Nov 15, 2024
acbc56f
doh
blampe Nov 16, 2024
c97347a
quoting
blampe Nov 16, 2024
1eda819
bump
blampe Nov 16, 2024
e05412f
test prerelease flow
blampe Nov 18, 2024
cb1ddd9
hard-code an alpha version for testing
blampe Nov 18, 2024
40cc045
Merge branch 'master' of github.com:pulumi/pulumi-eks into blampe/ci-…
blampe Nov 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .ci-mgmt.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
provider: eks
major-version: 3
aws: true
env:
ALT_AWS_ACCESS_KEY_ID: ${{ secrets.ALT_AWS_ACCESS_KEY_ID }}
ALT_AWS_SECRET_ACCESS_KEY: ${{ secrets.ALT_AWS_SECRET_ACCESS_KEY }}
AWS_REGION: us-west-2
GITHUB_TOKEN: ${{ secrets.PULUMI_BOT_TOKEN }}
GOLANGCI_LINT_VERSION: v1.61.0
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
PROVIDER: eks
PUBLISH_REPO_PASSWORD: ${{ secrets.OSSRH_PASSWORD }}
PUBLISH_REPO_USERNAME: ${{ secrets.OSSRH_USERNAME }}
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
PULUMI_API: https://api.pulumi-staging.io
PULUMI_GO_DEP_ROOT: ${{ github.workspace }}/..
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
SIGNING_KEY: ${{ secrets.JAVA_SIGNING_KEY }}
SIGNING_KEY_ID: ${{ secrets.JAVA_SIGNING_KEY_ID }}
SIGNING_PASSWORD: ${{ secrets.JAVA_SIGNING_PASSWORD }}
PULUMI_ENABLE_RESOURCE_REFERENCES: 1
PULUMI_LOCAL_NUGET: ${{ github.workspace }}/nuget
template: generic
freeDiskSpaceBeforeTest: true
69 changes: 69 additions & 0 deletions .github/ISSUE_TEMPLATE/bug.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: Bug Report
description: Report something that's not working correctly
labels: ["kind/bug", "needs-triage"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
You can also ask questions on our [Community Slack](https://slack.pulumi.com/).
- type: textarea
id: what-happened
attributes:
label: Describe what happened
description: Please summarize what happened, including what Pulumi commands you ran, as well as
an inline snippet of any relevant error or console output.
validations:
required: true
- type: textarea
id: sample-program
attributes:
label: Sample program
description: |
<details><summary>Provide a reproducible sample program</summary>
If this is a bug you encountered while running a Pulumi command, please provide us with a minimal,
self-contained Pulumi program that reproduces this behavior so that we can investigate on our end.
Without a functional reproduction, we will not be able to prioritize this bug.
**Note:** If the program output is more than a few lines, please send us a Gist or a link to a file.
</details>
validations:
required: true
- type: textarea
id: log-output
attributes:
label: Log output
description: |
<details><summary>How to Submit Logs</summary>
If this is something that is dependent on your environment, please also provide us with the output of
`pulumi up --logtostderr --logflow -v=10` from the root of your project.
We may also ask you to supply us with debug output following [these steps](https://www.pulumi.com/docs/using-pulumi/pulumi-packages/debugging-provider-packages/).
**Note:** If the log output is more than a few lines, please send us a Gist or a link to a file.
</details>
- type: textarea
id: resources
attributes:
label: Affected Resource(s)
description: Please list the affected Pulumi Resource(s) or Function(s).
validations:
required: false
- type: textarea
id: versions
attributes:
label: Output of `pulumi about`
description: Provide the output of `pulumi about` from the root of your project.
validations:
required: true
- type: textarea
id: ctx
attributes:
label: Additional context
description: Anything else you would like to add?
validations:
required: false
- type: textarea
id: voting
attributes:
label: Contributing
value: |
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
35 changes: 35 additions & 0 deletions .github/ISSUE_TEMPLATE/epic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
name: Epic
about: Tracks a shippable unit of work
title: '[Epic] {your-title-here}'
labels: kind/epic
projects: ['pulumi/32']
assignees: ''
type: Epic
---

## Overview
<!-- Start with a one- to three-sentence summary that should be understandable by any Pulumian or community member, even those without any context on the work. -->

## Key KPIs
<!-- What KPIs should this Epic will move; what could we measure to observe that this project was successful? -->

## Key Stakeholders
- Product and Engineering: <!-- Teams and individuals involved in the design and implementation -->
- Documentation: <!-- Representative from the docs team -->
- Marketing/Partnerships: <!-- Representative from the Marketing team -->
- Customers: <!-- [Tracking Issue](add-link-and-uncomment) -->

## Key Deliverables
<!-- List any discrete chunks of work or milestones that are planned in the epic (eg. subcomponent A, dogfood release, beta, etc.) -->

### References 📔

<!-- Any project that is more than one iteration should have a Project Board using this [template](https://github.com/orgs/pulumi/projects/131). -->
- [ ] Project View <!-- [Link](add-link-and-uncomment) -->
- [ ] PR/FAQ <!-- [Link](add-link-and-uncomment) -->
- [ ] Design Doc <!-- [Link](add-link-and-uncomment) -->
- [ ] UX Designs <!-- [Link](add-link-and-uncomment) -->
- [ ] Decision Log <!-- [Link](add-link-and-uncomment) -->

<!-- Work items should be added to the project board linked above -->
16 changes: 16 additions & 0 deletions .github/actions/download-bin/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Download binary assets
description: Downloads the provider and tfgen binaries to `bin/`.

runs:
using: "composite"
steps:
- name: Download provider + tfgen binaries
uses: actions/download-artifact@v4
with:
name: eks-provider.tar.gz
path: ${{ github.workspace }}/bin
- name: Untar provider binaries
shell: bash
run: |
tar -zxf ${{ github.workspace }}/bin/provider.tar.gz -C ${{ github.workspace}}/bin
find ${{ github.workspace }} -name "pulumi-*-eks" -print -exec chmod +x {} \;
7 changes: 2 additions & 5 deletions .github/actions/setup-tools/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,7 @@ runs:
if: inputs.tools == 'all' || contains(inputs.tools, 'go')
uses: actions/setup-go@v5
with:
# TODO: here pulumi-eks diverges from the 1.21.x standard, to upgrade separately.
# go-version: "1.21.x"
go-version: "1.18.x"
go-version: "1.21.x"
cache-dependency-path: |
provider/*.sum
upstream/*.sum
Expand Down Expand Up @@ -53,8 +51,7 @@ runs:
if: inputs.tools == 'all' || contains(inputs.tools, 'nodejs')
uses: actions/setup-node@v4
with:
# TODO: here pulumi-eks diverges from 20.x standard; will need to update.
node-version: 18.x
node-version: 20.x
registry-url: https://registry.npmjs.org

- name: Setup DotNet
Expand Down
15 changes: 15 additions & 0 deletions .github/actions/upload-bin/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Upload bin assets
description: Uploads the provider binaries to `bin/`.

runs:
using: "composite"
steps:
- name: Tar provider binaries
shell: bash
run: tar -zcf ${{ github.workspace }}/bin/provider.tar.gz -C ${{ github.workspace }}/bin/ pulumi-resource-eks
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: eks-provider.tar.gz
path: ${{ github.workspace }}/bin/provider.tar.gz
retention-days: 30
20 changes: 20 additions & 0 deletions .github/actions/upload-sdk/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Upload SDK asset
description: Upload the SDK for a specific language as an asset for the workflow.

inputs:
language:
required: true
description: One of nodejs, python, dotnet, go, java

runs:
using: "composite"
steps:
- name: Compress SDK folder
shell: bash
run: tar -zcf sdk/${{ inputs.language }}.tar.gz -C sdk/${{ inputs.language }} .
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.language }}-sdk.tar.gz
path: ${{ github.workspace}}/sdk/${{ inputs.language }}.tar.gz
retention-days: 30
55 changes: 55 additions & 0 deletions .github/workflows/build_provider.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: "Build Provider"

on:
workflow_call:
inputs:
version:
required: true
type: string
description: Version of the provider to build

jobs:
build_provider:
name: Build ${{ matrix.platform.os }}-${{ matrix.platform.arch }}
runs-on: ubuntu-latest
env:
PROVIDER_VERSION: ${{ inputs.version }}
strategy:
fail-fast: true
matrix:
platform:
- os: linux
arch: amd64
- os: linux
arch: arm64
- os: darwin
arch: amd64
- os: darwin
arch: arm64
- os: windows
arch: amd64
steps:
- name: Checkout Repo
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Setup tools
uses: ./.github/actions/setup-tools
with:
tools: pulumictl, go
- name: Download schema-embed.json
uses: actions/download-artifact@v4
with:
# Use a pattern to avoid failing if the artifact doesn't exist
pattern: schema-embed.*
# Avoid creating directories for each artifact
merge-multiple: true
path: provider/cmd/pulumi-resource-eks/schema-embed.json
- name: Build & package provider
run: make provider_dist-${{ matrix.platform.os }}-${{ matrix.platform.arch }}
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: pulumi-resource-eks-v${{ inputs.version }}-${{ matrix.platform.os }}-${{ matrix.platform.arch }}.tar.gz
path: dist/pulumi-resource-eks-v${{ inputs.version }}-${{ matrix.platform.os }}-${{ matrix.platform.arch }}.tar.gz
retention-days: 30
84 changes: 84 additions & 0 deletions .github/workflows/build_sdk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
name: "Build SDK"

on:
workflow_call:
inputs:
version:
required: true
type: string

env:
ALT_AWS_ACCESS_KEY_ID: ${{ secrets.ALT_AWS_ACCESS_KEY_ID }}
ALT_AWS_SECRET_ACCESS_KEY: ${{ secrets.ALT_AWS_SECRET_ACCESS_KEY }}
AWS_REGION: us-west-2
GITHUB_TOKEN: ${{ secrets.PULUMI_BOT_TOKEN }}
GOLANGCI_LINT_VERSION: v1.61.0
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
NUGET_PUBLISH_KEY: ${{ secrets.NUGET_PUBLISH_KEY }}
PROVIDER: eks
PUBLISH_REPO_PASSWORD: ${{ secrets.OSSRH_PASSWORD }}
PUBLISH_REPO_USERNAME: ${{ secrets.OSSRH_USERNAME }}
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
PULUMI_API: https://api.pulumi-staging.io
PULUMI_ENABLE_RESOURCE_REFERENCES: "1"
PULUMI_GO_DEP_ROOT: ${{ github.workspace }}/..
PULUMI_LOCAL_NUGET: ${{ github.workspace }}/nuget
PYPI_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
PYPI_USERNAME: __token__
SIGNING_KEY: ${{ secrets.JAVA_SIGNING_KEY }}
SIGNING_KEY_ID: ${{ secrets.JAVA_SIGNING_KEY_ID }}
SIGNING_PASSWORD: ${{ secrets.JAVA_SIGNING_PASSWORD }}
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
TF_APPEND_USER_AGENT: pulumi
PROVIDER_VERSION: ${{ inputs.version }}

jobs:
build_sdk:
name: build_sdk
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
language:
- nodejs
- python
- dotnet
- go
- java
steps:
- name: Checkout Repo
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Cache examples generation
uses: actions/cache@v4
with:
path: |
.pulumi/examples-cache
key: ${{ runner.os }}-${{ hashFiles('provider/go.sum') }}
- name: Setup tools
uses: ./.github/actions/setup-tools
with:
tools: pulumictl, pulumicli, ${{ matrix.language }}
- name: Download bin
uses: ./.github/actions/download-bin
- name: Install plugins
run: make install_plugins
- name: Update path
run: echo "${{ github.workspace }}/bin" >> "$GITHUB_PATH"
- name: Build SDK
run: make build_${{ matrix.language }}
- name: Check worktree clean
uses: pulumi/git-status-check-action@v1
with:
allowed-changes: |
sdk/**/pulumi-plugin.json
sdk/dotnet/*.csproj
sdk/go/**/pulumiUtilities.go
sdk/nodejs/package.json
sdk/python/pyproject.toml
- name: Upload SDK
uses: ./.github/actions/upload-sdk
with:
language: ${{ matrix.language }}
Loading
Loading