Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db: download compactions are being counted towards the compaction concurrency #4299

Open
sumeerbhola opened this issue Jan 28, 2025 · 0 comments

Comments

@sumeerbhola
Copy link
Collaborator

sumeerbhola commented Jan 28, 2025

We add download compactions to the inProgress compaction map here https://github.com/cockroachdb/pebble/blob/master/download.go#L439 (we also add flushes to this map). Download compactions have their own concurrency limit specified by Options.MaxConcurrentDownloads(). For this reason these download compactions don't increment DB.mu.compact.compactingCount, and instead increment DB.mu.compact.downloadingCount. But in compactionPickerByScore.pickAuto we are using the length of the in-progress compactions (which excludes flushes but includes download compactions) to decide whether we are allowed to pick another compaction.

if n := len(env.inProgressCompactions); n > 0 {

This is incorrect -- download compactions should be excluded from this length calculation.

Jira issue: PEBBLE-336

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants