Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(filter,cli): add remove future metrics in cli and discarding metrics written into the future #1012

Merged
merged 11 commits into from
May 7, 2024

Conversation

almostinf
Copy link
Member

@almostinf almostinf commented Apr 16, 2024

Add remove future metrics in cli and discarding metrics written into the future

Added ability to delete metrics via --cleanup-future-metrics command in cli with specifying cleanup_future_metrics_duration parameter in config, starting from which metrics written to the future will be deleted

Also added discarding metrics whose timestamp is greater than now() + dropMetricsTTL to avoid writing garbage to the database

And accelerated tests by 8 seconds by replacing time.Sleep with metricsCache.Flush.

@almostinf almostinf requested a review from a team as a code owner April 16, 2024 12:45
log.Info().Msg("Cleanup of outdated metrics finished")
}

if *cleanupFutureMetrics {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

тоже спрошу, не хотим ли мы все это сразу в отчистку метрик добавить, чтобы не плодить ключики?

Copy link
Member Author

@almostinf almostinf Apr 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Мне кажется, это не нужно, тк операция удаления метрик в будущее довольно тяжелая, тк требует прохода по всем метрикам, в идеале 1 раз удалить метрики, уже записанные, а все новые, пишущиеcя в будущее, и так будут откидываться

cmd/cli/main.go Outdated Show resolved Hide resolved
@almostinf almostinf requested a review from kissken April 17, 2024 10:05
@almostinf
Copy link
Member Author

/build

kissken
kissken previously approved these changes Apr 25, 2024
@codecov-commenter
Copy link

codecov-commenter commented Apr 27, 2024

Codecov Report

Attention: Patch coverage is 72.72727% with 12 lines in your changes are missing coverage. Please review.

Project coverage is 67.82%. Comparing base (1f2b4ac) to head (ea9e162).
Report is 6 commits behind head on master.

❗ Current head ea9e162 differs from pull request most recent head 4c8004d. Consider uploading reports for the commit 4c8004d to get more accurate results

Files Patch % Lines
cmd/cli/main.go 0.00% 10 Missing ⚠️
database/redis/metric.go 90.00% 1 Missing and 1 partial ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1012      +/-   ##
==========================================
- Coverage   67.92%   67.82%   -0.11%     
==========================================
  Files         212      212              
  Lines       12024    12098      +74     
==========================================
+ Hits         8167     8205      +38     
- Misses       3359     3388      +29     
- Partials      498      505       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

cmd/cli/main.go Outdated
cleanupLastChecks = flag.Bool("cleanup-last-checks", false, "Delete abandoned triggers last checks.")
cleanupTags = flag.Bool("cleanup-tags", false, "Delete abandoned tags.")
cleanupMetrics = flag.Bool("cleanup-metrics", false, "Delete outdated metrics.")
cleanupFutureMetrics = flag.Bool("cleanup-future-metrics", false, "Delete metrics from the database written to the future.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Мне кажется, сейчас неконсистентно с хелпами по другим ключам

Suggested change
cleanupFutureMetrics = flag.Bool("cleanup-future-metrics", false, "Delete metrics from the database written to the future.")
cleanupFutureMetrics = flag.Bool("cleanup-future-metrics", false, "Delete metrics with future timestamps.")

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Поправил

@almostinf almostinf requested a review from Tetrergeru May 6, 2024 10:21
@almostinf
Copy link
Member Author

/build

@almostinf almostinf merged commit ce13490 into master May 7, 2024
6 checks passed
@almostinf almostinf deleted the fix/remove-future-metrics branch May 7, 2024 08:46
Copy link

github-actions bot commented May 7, 2024

Build and push Docker images with tag: 2024-05-07.ce13490

2 similar comments
Copy link

github-actions bot commented May 7, 2024

Build and push Docker images with tag: 2024-05-07.ce13490

Copy link

github-actions bot commented May 7, 2024

Build and push Docker images with tag: 2024-05-07.ce13490

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants