Autoscaling

[bkmgit]

Slurm supports autoscaling which is very helpful for cloud deployments. Might this be something that can be included and made relatively easy to configure?

[ChrisDowning]

Hi @bkmgit - we have a "Cloud Working Group" tackling this right now. The goal is to have a cloud equivalent of the current on-premises recipes, but skipping the un-needed parts (Warewulf/xCAT) and instead dealing with automated scale up/down of compute nodes, as well as other considerations (which instance types make sense, what storage to use, etc).

I'll drop another message here when there is something for you to try out.

[bkmgit]

@ChrisDowning Thanks. Mailing list may be a helpful thing to have as indicated at openhpc/cloudwg#13
Hopefully community contributions on the design and implementation of the cloud equivalent will also be considered. This may possibly also be useful for HPC Carpentry hpc-carpentry/coordination#42

[sjpb]

@ChrisDowning I'd definitely be interested in hearing what happens too. We have done proof-of-concept work on Slurm autoscaling using OpenHPC before, although it's not ready for production.

[ChrisDowning]

@sjpb Great - will keep you in the loop. I've deployed auto-scaling using Slurm power-saving for customers a few times over the last ~18 months, just never using the OpenHPC build. Deploying the same basic functionality using the OpenHPC Slurm package is pretty trivial, so we need to just get it documented first then move on to the "best practices" and other considerations people might not be aware of if they are new to cloud.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.