Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Add report creation to synthetic correctness test #1845

Open
wants to merge 45 commits into
base: main
Choose a base branch
from

Conversation

tristanvuong2021
Copy link
Contributor

test: Add report creation to synthetic correctness test

@tristanvuong2021 tristanvuong2021 marked this pull request as ready for review October 9, 2024 19:57
@wfa-reviewable
Copy link

This change is Reviewable

@tristanvuong2021 tristanvuong2021 force-pushed the tristanvuong-add-report-creation-to-correctness-test branch from 798bccd to 6151fe4 Compare November 6, 2024 22:32
Copy link
Member

@SanjayVas SanjayVas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 11 of 11 files at r1, all commit messages.
Reviewable status: all files reviewed, 6 unresolved discussions (waiting on @tristanvuong2021)


.github/workflows/run-k8s-tests.yml line 55 at r1 (raw file):

        MC_API_KEY: ${{ secrets.MC_API_KEY }}
        GCLOUD_PROJECT: ${{ vars.GCLOUD_PROJECT }}
        REPORTING_PUBLIC_API_TARGET: v2alpha.reporting.dev.halo-cmm.org:8443

I went ahead and added the variable for all 3 environments

Suggestion:

${{ vars.REPORTING_PUBLIC_API_TARGET }}

src/test/kotlin/org/wfanet/measurement/integration/k8s/SyntheticGeneratorCorrectnessTest.kt line 51 at r1 (raw file):

 * Test for correctness of an existing CMMS on Kubernetes where the EDP simulators use
 * [SyntheticGeneratorEventQuery] with [SyntheticGenerationSpecs.SYNTHETIC_POPULATION_SPEC_LARGE].
 * The computation composition is using ACDP by assumption.

nit: document that this assuming that there's a Reporting system connected to the CMMS


src/main/kotlin/org/wfanet/measurement/loadtest/reporting/ReportingUserSimulator.kt line 134 at r1 (raw file):

          listEventGroupsRequest {
            parent = measurementConsumerName
            pageSize = 1000

nit: technically you still should be looping through to exhaust all pages regardless of what page size you set rather than relying on the fact that our current impl will give you the results


src/main/kotlin/org/wfanet/measurement/loadtest/reporting/ReportingUserSimulator.kt line 160 at r1 (raw file):

    try {
      return reportingSetsClient
        .withAuthenticationKey(apiAuthenticationKey)

The API key is only for the Kingdom public API.

Code quote:

.withAuthenticationKey(apiAuthenticationKey)

src/main/kotlin/org/wfanet/measurement/loadtest/reporting/ReportingUserSimulator.kt line 234 at r1 (raw file):

        Report.State.UNRECOGNIZED,
        Report.State.STATE_UNSPECIFIED -> delay(5000)
      }

Add some delay here so we're not polling as fast as possible.


src/test/kotlin/org/wfanet/measurement/integration/k8s/EmptyClusterCorrectnessTest.kt line 179 at r1 (raw file):

    override val reportingTestHarness: ReportingUserSimulator?
      get() = null

So this will only work when run on cloud? I assume we're hitting some GHA limit if we try to do this in a local cluster?

Code quote:

null

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants