Can you confirm, and state clearly in the docs, that sampling from a Replay buffer is "Sampling With Replacement"

[jamartinh]

Can you confirm, and state clearly in the docs, that sampling from a Replay buffer is "Sampling With Replacement"?

[jamartinh]

It is said in the doc of the sample method in the class reference. However due to high significance, for statistical and theoretical properties, it is perhaps desirable to state it more clearly as a feature in the main docs.

[ymd-h]

@jamartinh
Thank you for your feedback.

A transaction can be contained multiple times in the same mini-batch. (aka. "Sample with Replacement")
This can be tested with small buffer with larger batch size sampling.

ReplayBuffer samples transactions independently.

cpprb/cpprb/PyReplayBuffer.pyx

Line 1384 in 0513b05

cdef idx = np.random.randint(0,self.get_stored_size(),batch_size)

PrioritizedReplayBuffer uses stratified sampling, so that transactions in the same mini-batch are not completely independent.
https://ymd_h.gitlab.io/cpprb/features/per/#technical-detail