Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase number of capacity units #192

Open
wfvining opened this issue Mar 13, 2017 · 0 comments
Open

Increase number of capacity units #192

wfvining opened this issue Mar 13, 2017 · 0 comments

Comments

@wfvining
Copy link
Contributor

We need to be able to add more storage by adding a set of capacity units while still maintaining readability of the old data. We have discussed several schemes for this including distributing the objects uniformly at random on write, then stating all the capacity units in parallel to find the object for reading. This has the drawback that we can't easily re-balance objects across the available capacity.

I have discussed an alternative scheme based on consistent-hashing with Jeff that will allow for efficient background re-balancing (if wanted) and eliminates the need to stat every capacity unit. This scheme also facilitates the idea of "capacity targets" where an admin can decide we want to place 50% of the data on cap0 and 25% on cap1 and cap2 respectively.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant