Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HashMap reallocates even though capacity is not exceeded #134459

Open
e00E opened this issue Dec 18, 2024 · 0 comments
Open

HashMap reallocates even though capacity is not exceeded #134459

e00E opened this issue Dec 18, 2024 · 0 comments
Labels
A-collections Area: `std::collection` A-docs Area: Documentation for any part of the project, including the compiler, standard library, and tools C-bug Category: This is a bug. T-libs Relevant to the library team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.

Comments

@e00E
Copy link
Contributor

e00E commented Dec 18, 2024

HashMap::with_capacity's documentation states:

The hash map will be able to hold at least capacity elements without reallocating. This method is allowed to allocate for more elements than capacity.

This statement is incorrect as the following code demonstrates.

fn main() {
    let initial_capacity = 100;
    let mut a = std::collections::HashMap::<u32, ()>::with_capacity(initial_capacity);
    let mut current_capacity = initial_capacity;
    for i in 0..1_000 {
        let new_capacity = a.capacity();
        if new_capacity != current_capacity {
            println!("iteration {i}, new capacity {new_capacity}");
            current_capacity = a.capacity();
        }

        assert!(a.len() < initial_capacity);
        a.insert(i, ());
        if a.len() == initial_capacity {
            a.remove(&i).unwrap();
        }
    }
}

A possible output of this program is as follows. It is a possible output because the default hasher is randomly seeded. You can make it deterministic with a custom hasher.

iteration 0, new capacity 112
iteration 100, new capacity 111
iteration 101, new capacity 110
iteration 105, new capacity 109
iteration 109, new capacity 108
iteration 111, new capacity 107
iteration 125, new capacity 106
iteration 137, new capacity 105
iteration 138, new capacity 104
iteration 141, new capacity 103
iteration 165, new capacity 102
iteration 187, new capacity 101
iteration 188, new capacity 100
iteration 252, new capacity 99
iteration 253, new capacity 224

As you can see in the output the capacity jumps from the initial 112 to 224 in iteration 253. This means the HashMap has allocated more memory. However, as the assert shows, the HashMap never held more elements than the initial capacity. This violates the guarantee documented in with_capacity.

This is a bug in hashbrown. I've opened an issue there too rust-lang/hashbrown#602 .

@e00E e00E added the C-bug Category: This is a bug. label Dec 18, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Dec 18, 2024
@workingjubilee workingjubilee added T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Dec 18, 2024
@jieyouxu jieyouxu added A-collections Area: `std::collection` A-docs Area: Documentation for any part of the project, including the compiler, standard library, and tools and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-collections Area: `std::collection` A-docs Area: Documentation for any part of the project, including the compiler, standard library, and tools C-bug Category: This is a bug. T-libs Relevant to the library team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

4 participants