My cell biology lab does a lot of image analysis (>100 TB) and our i7 linux machine is no longer up to the task. People tell me that I need a SLURM cluster to run Ilastik and CellProfiler efficiently. Hardware is not my specialty though. Can you advise me? Where can I buy a SLURM cluster (ideally turnkey, zero hassle)? How much do I need to budget? Sorry for the n00b questions, please be gentle on me!
SLURM cluster
- Thread starter aag
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
There is nothing turnkey about a slurm cluster, imo this sounds like the perfect cross discipline project for CS/EE grad students.
You need someone that knows the workflow demands of cellprofiler and llastik to know what your computational needs are.
Slurm is an opensource cluster job manager, I very quickly found a plugin for cell profiler for it. GitHub - sa2c/CellProfiler-RunOnCluster: A CellProfiler plugin for running on an HPC cluster
If you want a turnkey solution with Slurm you are going to be paying a considerable sum to a consultant to make a glove for you.
I may have laughed out loud and then read through your past posts to make sure you weren't trolling.
You need someone that knows the workflow demands of cellprofiler and llastik to know what your computational needs are.
Slurm is an opensource cluster job manager, I very quickly found a plugin for cell profiler for it. GitHub - sa2c/CellProfiler-RunOnCluster: A CellProfiler plugin for running on an HPC cluster
If you want a turnkey solution with Slurm you are going to be paying a considerable sum to a consultant to make a glove for you.
I may have laughed out loud and then read through your past posts to make sure you weren't trolling.
Simple Linux Utility for Resource Management
slurm.schedmd.com
possibly a machine with some MORE COREs can help you better for the moment.
Slurm Workload Manager - Overview
possibly a machine with some MORE COREs can help you better for the moment.
Workloads appear capable of gpu accel as well.Simple Linux Utility for Resource Management
Slurm Workload Manager - Overview
possibly a machine with some MORE COREs can help you better for the moment.
I am fully conscious of the naivety of my question, and I have apologized in advance - hence I will not offer additional apologies. I am a professor of neuropathology, not a computer scientist. If I were one, I might not need to ask any questions.
CS grad students rarely come to my lab for their MSc or PhD. Nowadays they all want a path towards fintech, which I cannot offer. It's not easy to do cross-disciplinary science; the incentives for the young people are often insufficient.
I would suggest you pick the brains of those who suggested slurm to you as they might give additional details to their setups.
As it stands, having read back to your storage help request I think I have an idea of your lab config. I didn't mean to be rude but slurm and turn-key was a bit amusing. I realize this is far from your expertise but you will still have to learn a new workflow if you utilize slurm.
I am also not sure slurm is right for you, it is good for scaling but I am not sure you need it, it can take you from 2-3 boxes all the way to 10,000 nodes. My expertise is in the hardware side of hyperscale/supercomputer systems.
Are you still using 2 ingest boxes per
??
You could tie the 2 boxes together with slurm, giving work to each other...you could use those 2 boxes to give jobs to a new 3rd beefy box.
Or you could just build 2 beefier boxes without the additional learning of slurm.
The current hardware specs, and the processing file size and time of current tasks would be ideal in helping.
Last edited: