12G SAS SSD boot drive and NVMe data volume

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Superpos

New Member
Jun 9, 2020
25
1
3
I'm thinking about getting a few cheap Dell rack nodes and filling them with M.2 NVMe drives via PCIe slots ideally for some form of clustered storage. Due to a lack of slots for an NVMe boot drive (CentOS 7), I'm considering using a 12G SAS SSD instead - probably a HGST Ultrastar SSD1600MR.

If it runs about 1GB/s throughput, is that going to be a significant bottleneck for the NVMe volumes? Especially if I start RAIDing them together? Or will the system memory cache make that a non-issue? I'll have at least 64GB RAM. The use case is mainly film post production/vfx, and possibly some machine learning. Thanks!
 
Last edited:

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
"Filling them" - How many M2 NVME is that? Which M2 NVME?

I don't know any drive that runs 1TB/s, which drive is this?

A high performance storage system will be much better off with >64GB RAM, but you don't mention what storage system you're running either?

Are you going to be using hardware RAID for the SAS3 or software RAID?

You may want to elaborate on your specific setup for accurate advice. (Drive make\model, quantity, configuration, OS, utilization, etc.)
 

Superpos

New Member
Jun 9, 2020
25
1
3
Thanks, that was a bit of a goof. I meant a SAS 12G SSD running ~1GB/sec read. I've fixed the original post. My simple point is a SAS 12G SSD boot drive is less than half the speed of even a single NVMe stick used as a data volume, and is that a problem? I would normally put the boot SSD in an x4 PCI slot but I'm trying to save slots for other things.

So I'm considering putting the system on the SAS SSD since there's an onboard SAS 3008 controller and I can use a 2.5" bay.

(Drive make\model, quantity, configuration, OS, utilization, etc.)
As mostly mentioned,
System boot drive possibly HGST Ultrastar SSD1600MR running CentOS 7.8
Utilization is film post production
Quantity and brand of NVMe drives doesn't matter because even one stick is more than twice as fast as the 12G SAS SSD.
NVMe configuration is possibly Raidix Era to create the volume for initially 4xNVMe sticks in an x16 PCIe adapter. The SAS SSD has no configuration, it would just be a single boot drive running XFS or EXT3.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
I don't know enough about video production, but something tells me you're not logging like a high usage DB where you'd want a high performance boot or additional location so that the DB performance is optimized.

You're not going to be running that NVME maxed out so a SAS3 boot should be more than enough.
 

Superpos

New Member
Jun 9, 2020
25
1
3
Right, no databases, just image file sequences. A single stream of 4k uncompressed playback for a 16 bit RGB file sequence would only be ~1.7GB/s, so it would work comfortably on a single NVMe stick. Multiple streams in a Resolve editorial timeline would be an example of where I would benefit from a quad NVMe RAID setup.

I mean is there a general rule about what is an acceptable speed deficiency for your boot volume compared to your high speed data storage? Emphasis on read speeds. I guess it's contextual to what the usage scenario is. The most read bandwidth I would use would probably be quad NVMe drives maxing out a single x16 slot, vs a SAS 12G boot drive @~1GB/s. People spend a lot more money configuring this sort of thing with obscenely expensive flash storage devices, but I'm of course trying to do it on the cheap.
 

i386

Well-Known Member
Mar 18, 2016
4,217
1,540
113
34
Germany
If you're not working on the boot volume even a sata dom could be used as a boot device. Once the os boots, most of the system relevant data is loaded into ram.
 

Superpos

New Member
Jun 9, 2020
25
1
3
Great. That's what I suspected, but I didn't know for sure. Thanks! I've got plenty of ram slots, so memory is not going to be an issue.