[ENG-10412] Upload projectMetadata file #2149

khamilowicz · 2023-12-13T10:35:24Z

Why

ENG-10412 We want to allow user to easily check the content of build archive. This should streamline debugging for certain cases.

How

CLI generates file containing a list of files present in project archive. This file is uploaded together with project archive to the same bucket.

The reason for generating this file at this step is that:

GCP does not provide means to peek into archive content
The only place on the backend where the project archive is downloaded is in the worker, which is well isolated from other parts of the app, including DBs and GCP, so it would required creating some new service just for the purpose of downloading and listing files in this archive
it is much easier to do it on user's computer where the archive is created
data is available immediately, regardless of worker availability

Uploading the file to GCP (as opposed to, for example, storing it in the DB) has following advantages:

doesn't pollute DB with unnecessary, read-only data, which would be used by a small minority of users
makes it automatically follow project archive TTL policy - obsolete data is automatically deleted

Test Plan

Create new build.

Deploy plan

linear · 2023-12-13T10:35:28Z

ENG-10412 UI to inspect the contents of EAS Build archive for a build

szdziedzic

It looks reasonable, I just have few more questions before merging it

packages/eas-cli/src/build/build.ts

packages/eas-cli/src/build/utils/repository.ts

packages/eas-cli/src/build/build.ts

packages/eas-cli/src/build/utils/repository.ts

packages/eas-cli/src/build/build.ts

packages/eas-cli/src/build/utils/repository.ts

szdziedzic

It might be good to add deployment order to each PR description, so it is clear in which order the changes should be deployed.

Last few comments:

packages/eas-cli/src/build/utils/repository.ts

packages/eas-cli/src/build/build.ts

packages/eas-cli/src/build/graphql.ts

packages/eas-cli/src/build/utils/repository.ts

szdziedzic

I think that it looks good at the moment. Nice! 👏

Thanks for applying my suggestions 🙇

My final suggestion is to specify a deployment order, so it is clear what should land when

packages/eas-cli/src/build/graphql.ts

packages/eas-cli/src/build/build.ts

github-actions · 2024-01-30T13:19:17Z

Size Change: +578 B (0%)

Total Size: 51.4 MB

Filename	Size	Change
`./packages/eas-cli/dist/eas-linux-x64.tar.gz`	51.4 MB	+578 B (0%)

_{compressed-size-action}

codecov · 2024-01-30T13:30:56Z

Codecov Report

Attention: Patch coverage is 8.51064% with 43 lines in your changes are missing coverage. Please review.

Project coverage is 53.67%. Comparing base (26ba071) to head (7a40db5).

Files	Patch %	Lines
packages/eas-cli/src/build/utils/repository.ts	8.00%	22 Missing and 1 partial ⚠️
packages/eas-cli/src/build/build.ts	4.77%	17 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2149      +/-   ##
==========================================
- Coverage   53.78%   53.67%   -0.11%     
==========================================
  Files         525      525              
  Lines       19115    19158      +43     
  Branches     4037     4043       +6     
==========================================
+ Hits        10279    10281       +2     
- Misses       8114     8151      +37     
- Partials      722      726       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

packages/eas-cli/src/build/build.ts

packages/eas-cli/src/build/utils/repository.ts

packages/eas-cli/src/build/build.ts

sjchmiela

Looks better. A few comments!

szdziedzic

Few minor comments, code looks reasonable to me 👍 If the manual testing works fine after merging WWW it should be good to go 👏

Please rebase/merge main into this PR

CHANGELOG.md

packages/eas-cli/src/build/build.ts

packages/eas-cli/graphql.schema.json

sjchmiela · 2024-02-12T15:48:31Z

packages/eas-cli/src/build/utils/repository.ts

+  );
+  startTimer(timerLabel);
+
+  const metadataLocation = path.join(getTmpDirectory(), `${uuidv4()}-eas-build-metadata.json`);


Is the filename preserved — do GCS files have the same name? Maybe it would be better to call them eas-build-metadata-${uuidv4()}.json if that's the case?

sjchmiela · 2024-02-12T15:55:23Z

packages/eas-cli/src/build/build.ts

-        return bucketKey;
+        const { metadataLocation } = await uploadMetadataFileAsync<TPlatform>(projectTarball, ctx);
+        if (metadataLocation) {
+          return { bucketKey, metadataLocation };


I remain on the position the fact we have different names (key, location) for the same thing in the same object and the newer name is of less specific meaning is wrong. This comment has been left unanswered for a couple iterations now (first mentioned here) and the conversation marked as resolved twice which is confusing for me.

packages/eas-cli/src/graphql/generated.ts

github-actions · 2024-04-03T09:21:14Z

✅ Thank you for adding the changelog entry!

khamilowicz requested a review from szdziedzic as a code owner December 13, 2023 10:35

khamilowicz force-pushed the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch from 5306103 to 621303b Compare December 13, 2023 12:55

szdziedzic reviewed Dec 14, 2023

View reviewed changes

packages/eas-cli/src/build/build.ts Outdated Show resolved Hide resolved

packages/eas-cli/src/build/build.ts Outdated Show resolved Hide resolved

packages/eas-cli/src/build/utils/repository.ts Outdated Show resolved Hide resolved

szdziedzic requested review from radoslawkrzemien and sjchmiela December 14, 2023 11:06

sjchmiela reviewed Dec 14, 2023

View reviewed changes

khamilowicz force-pushed the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch from 621303b to 56b5c8b Compare December 16, 2023 16:18

khamilowicz requested review from sjchmiela and szdziedzic December 18, 2023 09:51

szdziedzic reviewed Dec 18, 2023

View reviewed changes

packages/eas-cli/src/build/build.ts Outdated Show resolved Hide resolved

packages/eas-cli/src/build/utils/repository.ts Outdated Show resolved Hide resolved

khamilowicz force-pushed the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch from 56b5c8b to 6ec3bfc Compare December 18, 2023 17:08

khamilowicz requested a review from szdziedzic December 18, 2023 17:09

szdziedzic reviewed Dec 19, 2023

View reviewed changes

khamilowicz force-pushed the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch 2 times, most recently from 7995c38 to 05354c6 Compare December 19, 2023 11:39

khamilowicz requested a review from szdziedzic December 19, 2023 11:41

szdziedzic reviewed Dec 19, 2023

View reviewed changes

packages/eas-cli/src/build/graphql.ts Outdated Show resolved Hide resolved

sjchmiela reviewed Dec 20, 2023

View reviewed changes

packages/eas-cli/src/build/build.ts Outdated Show resolved Hide resolved

khamilowicz force-pushed the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch 3 times, most recently from 83fe69c to 3af7343 Compare January 30, 2024 12:16

khamilowicz requested a review from sjchmiela January 30, 2024 12:17

khamilowicz force-pushed the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch from 3af7343 to 588127a Compare January 30, 2024 13:16

khamilowicz requested a review from szdziedzic January 31, 2024 10:32

sjchmiela reviewed Jan 31, 2024

View reviewed changes

khamilowicz force-pushed the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch from 71b75d4 to 87ccf9b Compare February 5, 2024 14:13

szdziedzic approved these changes Feb 7, 2024

View reviewed changes

CHANGELOG.md Show resolved Hide resolved

packages/eas-cli/src/build/build.ts Outdated Show resolved Hide resolved

packages/eas-cli/graphql.schema.json Outdated Show resolved Hide resolved

khamilowicz force-pushed the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch from 419a03c to bc2ef82 Compare February 12, 2024 15:09

khamilowicz requested review from szdziedzic and sjchmiela February 12, 2024 15:17

sjchmiela approved these changes Feb 12, 2024

View reviewed changes

szdziedzic reviewed Feb 12, 2024

View reviewed changes

packages/eas-cli/src/graphql/generated.ts Outdated Show resolved Hide resolved

khamilowicz force-pushed the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch from 09cbb45 to 6f3b4f3 Compare February 13, 2024 11:46

khamilowicz added 4 commits April 3, 2024 11:18

[ENG-10412] Upload projectMetadata file

1b32e86

Lint

8b87e55

[ENG-10412] Apply review suggestions

157b0a8

update graphql

7a40db5

khamilowicz force-pushed the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch from 6f3b4f3 to 7a40db5 Compare April 3, 2024 09:20

khamilowicz merged commit b346f96 into main Apr 3, 2024
9 checks passed

khamilowicz deleted the piotrekszeremeta/eng-10412-ui-to-inspect-the-contents-of-eas-build-archive-for-a-build branch April 3, 2024 09:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ENG-10412] Upload projectMetadata file #2149

[ENG-10412] Upload projectMetadata file #2149

khamilowicz commented Dec 13, 2023 •

edited

Loading

linear bot commented Dec 13, 2023

szdziedzic left a comment

szdziedzic left a comment

szdziedzic left a comment

github-actions bot commented Jan 30, 2024 •

edited

Loading

codecov bot commented Jan 30, 2024 •

edited

Loading

sjchmiela left a comment

szdziedzic left a comment

sjchmiela Feb 12, 2024

sjchmiela Feb 12, 2024

github-actions bot commented Apr 3, 2024

[ENG-10412] Upload projectMetadata file #2149

[ENG-10412] Upload projectMetadata file #2149

Conversation

khamilowicz commented Dec 13, 2023 • edited Loading

Why

How

Test Plan

Deploy plan

linear bot commented Dec 13, 2023

szdziedzic left a comment

Choose a reason for hiding this comment

szdziedzic left a comment

Choose a reason for hiding this comment

szdziedzic left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 30, 2024 • edited Loading

codecov bot commented Jan 30, 2024 • edited Loading

Codecov Report

sjchmiela left a comment

Choose a reason for hiding this comment

szdziedzic left a comment

Choose a reason for hiding this comment

sjchmiela Feb 12, 2024

Choose a reason for hiding this comment

sjchmiela Feb 12, 2024

Choose a reason for hiding this comment

github-actions bot commented Apr 3, 2024

khamilowicz commented Dec 13, 2023 •

edited

Loading

github-actions bot commented Jan 30, 2024 •

edited

Loading

codecov bot commented Jan 30, 2024 •

edited

Loading