Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for account format v2 in util check-storage #6977

Merged
merged 6 commits into from
Feb 6, 2025

Conversation

fxamacker
Copy link
Member

@fxamacker fxamacker commented Feb 4, 2025

Currently, util program's check-storage command (to check storage health) only supports account storage format v1. However, we need to also support the recently added v2.

This commit adds two flags to check storage health of account format (v1 or v2).

  • account-format-v1 flag should be on if state contains any accounts in v1 format
  • account-format-v2 flag should be on if state contains any accounts in v2 format

IMPORTANT: Both flags should be specified if both v1 and v2 are present in the state (e.g. during zero-downtime migration to account storage format v2 before all accounts finish migrating).

Currently, the `storage-health` command only supports
account storage format v1.  However, we need to
also support the recently added v2.

This commit adds two flags to check storage health of account
format (v1 or v2).
- "account-format-v1" flag should be on if state contains any
  accounts in v1 format
- "account-format-v2" flag should be on if state contains any
  accounts in v2 format

Both flags should be specified if both v1 and v2 are present
in the state.
@fxamacker fxamacker requested a review from a team February 4, 2025 22:27
@fxamacker fxamacker self-assigned this Feb 4, 2025
@fxamacker fxamacker requested a review from a team as a code owner February 4, 2025 22:27
@fxamacker fxamacker changed the title Update util storage-health for account format v1 & v2 Add support for account format v2 in util storage-health Feb 4, 2025
@codecov-commenter
Copy link

codecov-commenter commented Feb 4, 2025

Codecov Report

Attention: Patch coverage is 17.92453% with 87 lines in your changes missing coverage. Please review.

Project coverage is 41.11%. Comparing base (914f353) to head (740377b).

Files with missing lines Patch % Lines
cmd/util/cmd/check-storage/cmd.go 16.86% 69 Missing ⚠️
cmd/util/ledger/migrations/cadence_value_diff.go 25.00% 12 Missing ⚠️
...util/cmd/checkpoint-collect-stats/account_stats.go 0.00% 6 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##           master    #6977   +/-   ##
=======================================
  Coverage   41.10%   41.11%           
=======================================
  Files        2127     2127           
  Lines      186321   186278   -43     
=======================================
- Hits        76586    76581    -5     
+ Misses     103308   103275   -33     
+ Partials     6427     6422    -5     
Flag Coverage Δ
unittests 41.11% <17.92%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@turbolent turbolent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@fxamacker
Copy link
Member Author

I found old concurrency code in diff-states command isn't compatible anymore because new account format v2 uses a single map for all domains.

Fixed it with stop-gap code for now and will push the fix into this PR after testing it on gcp with account format v2.

I have a bad cold since early morning, so if test run on gcp doesn't finish soon (by around 9 PM Central) then I will push the fix tomorrow morning after confirming it worked.

The old concurrency code is incompatible with the new
account format v2 which uses a single map for account data.

As a stop-gap, this commit removes the incompatible concurrency code.
@fxamacker fxamacker changed the title Add support for account format v2 in util storage-health Add support for account format v2 in util check-storage Feb 5, 2025
@fxamacker
Copy link
Member Author

I have a bad cold since early morning, so if test run on gcp doesn't finish soon (by around 9 PM Central) then I will push the fix tomorrow morning after confirming it worked.

It ran fine on gcp so commit 740377b was pushed last night.

We can merge this PR (unless there is a bug). If needed, we can open a separate PR to optimize diff-states for speed.

@fxamacker fxamacker enabled auto-merge February 6, 2025 01:04
@fxamacker fxamacker added this pull request to the merge queue Feb 6, 2025
Merged via the queue into master with commit f0d7ec9 Feb 6, 2025
57 checks passed
@fxamacker fxamacker deleted the fxamacker/update-storage-health-check branch February 6, 2025 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants