Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please create a new release for Julia 1.x support #118

Open
wkharold opened this issue May 11, 2019 · 10 comments
Open

Please create a new release for Julia 1.x support #118

wkharold opened this issue May 11, 2019 · 10 comments

Comments

@wkharold
Copy link

The SlurmManager fails under Julia 1.x because of a change to the replace() function. The master branch incorporates the necessary modification but has not been incorporated in a release.

@vchuravy
Copy link
Member

Yes =, we do need a new release. I have been hesitant to do so because we currently lack tests (#105) and without them it is very hard to gauge which managers are currently working an which patches fixed things.

I am currently traveling, but if you need a release maybe you could help with transitioning from REQUIRE to Project.toml

@jamitzky
Copy link

jamitzky commented Sep 4, 2019

I still get the replace bug in the 1.x Versions for SLURM. Any plans to fix that?

@kescobo
Copy link
Collaborator

kescobo commented Apr 12, 2020

Given julia 1.0 has been out for nearly 2 years, seems like a release supporting it would be worth doing. Yes, it's unfortunate that it lacks tests, but that doesn't seem worse than the current situation...

@vchuravy
Copy link
Member

Yes and no... Not having a release is a clear documentation that this package currently does not have a maintainer and I am kinda hoping that someone would like to help out with this. People relying on software that might not work is worse than them being aware of that it might not work... So yes if you are using ClusterManagers.jl it would be great if you could help out with maintenance and maybe setting up a test system...

@kescobo
Copy link
Collaborator

kescobo commented Apr 13, 2020

That's fair - I can definitely see the point. But I'd argue that something that might not work in the release is better than something definitely doesn't work. Especially if it's clearly stated in the README that it's poorly tested.

That said, I'd potentially be willing to take on a little bit of maintenance work if that would be helpful. In general, my only experience with distributed computing is using SLURM on Harvard's and MIT's clusters.

@bjarthur
Copy link
Collaborator

how would a test system work given that this pkg supports several different schedulers? perhaps it's time to break it up into smaller packages?

@vchuravy
Copy link
Member

@bjarthur yes that might be possible. I would also be more than happy to give people the commit bit if they want to contribute to ClusterManagers.jl

There is also https://github.com/JuliaParallel/MPIClusterManagers.jl

@DrChainsaw
Copy link
Collaborator

Hi,

I'm willing to put some effort into helping maintaining the LSF parts, preferably as a separate package. I don't own an LSF cluster but I don't know if thats a showstopper or if it is just a matter of needing an extra pair of hands.

I was thinking that one could install the trial version in a build script (maybe ask IBM first if this is ok). I have no idea how hard this will be though. Perhaps it has already been tried?

@vchuravy vchuravy mentioned this issue Oct 6, 2020
@vchuravy
Copy link
Member

vchuravy commented Oct 6, 2020

@DrChainsaw I gave you access/

@DrChainsaw
Copy link
Collaborator

@vchuravy Thanks alot.

What do people feel about mocking as a way to support testing? I think it has some merits given that one disadvantage of spinning up a real (but local) cluster is that one probably will see more issues with that than the code it is supposed to test.

I started drafting up a tiny PoC a few days ago but then I got some other things on my plate.

In a linux environment it could be as simple as just piping a few echo job 23432 submitted to a file called bsub (or whatever is needed for the other managers) and similar for other commands while spinning up local worker processes.

It seems to be slightly more annoying in windows as it does not execute files without extensions, and julia shell won't run e.g. bsub.bat if you type ;bsub. It does work for exe-files though and my 5 minute search for how to realiably create an exe in any windows environment lead to .NET and csc.exe which might be workable despite being non-ideal.

All this goes away ofc if it is possible to modify which command is being executed.

Worth pursuing?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants