Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Promote multiprocessing MC runner into mkl_random package #27

Open
samaid opened this issue Mar 22, 2023 · 0 comments
Open

Promote multiprocessing MC runner into mkl_random package #27

samaid opened this issue Mar 22, 2023 · 0 comments

Comments

@samaid
Copy link
Contributor

samaid commented Mar 22, 2023

Examples are very useful to comprehend a concept of running mkl_random using multiprocessing. However, writing a multi-processing orchestration code is labor intensive and error prone.

I am suggesting to design proper API for generalized MC runner with:

  • Capability to request independent random states/streams for a given number of workers. Follow NumPy recommended methods under the hood

Possible API:

[streams] = mkl_random.get_independent_streams(n_streams=1, root_seed=None, brng=MT2203, method="seed-sequencing", n_jump_ahead=1)

n_streams: The number of independent streams
brng: One of basic RNG available in mkl_random
method: One of the following "default", "seed-sequencing", "leapfrog", "jump_ahead", "key-counter". The "leapfrog" method is applicable only for brng that supports the Leap Frogging technique of sub-sequencing random streams. The "key-counter" method is applicable for counter-based brng. Respectively, "jump_ahead" will be applicable to only brng that supports the Jumping/Skipping Ahead technique. The "seed-sequencing" will be universally supported by all brng.

The "default" method is brng specific. It will use seed sequencing technique for brng like MT19937. For MCG59 it will use the "leapfrog". For counter-based brng it will use unique the "key-counter" technique. The "seed-sequencing" will be the default method for the rest of brng.

The function will return the list [stream[0], ..., stream[n_streams-1] of size n_streams containing independent streams which can later be accessed by respective process's worker_id.

  • Capability to spawn the required number of processes from multiprocessing.Pool and perform a map operation for a user-supplied function local_task(worker_id, stream, *args, **kwargs).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant