SAS / SATA spinners for ceph

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

skunky

New Member
Apr 25, 2017
24
1
3
43
Hi everyone,
I'm in the situation that i would need some advice in choosing a proper spinning hdd for ceph ( mostly used for rbd - proxmox vms ) . Looking for both sas and sata hdds.
At the moment i have very bad performance with Seagate 2.5" 5TB because they are SMR drives, which is pretty slow on random writes - which an rbd does a lot. So bassically i cannot use the sata pool without cache tiering in front of it. ( Ceph Luminous / Bluestore ), I end up with slow requests, and those Seagates get fried at a point ( about 1/month going down ) :( .
Can you please recommend me some spinning drives that a would suite for a rbd sata pool ? Should i definitely go for 10K hdds, or 7.2k would be good ? I'm just not sure what to buy, and i'll need about 25 of them.
Would it be some 2.5" "hi-end" hdds that i can buy or some "reasonably good" ones ?
Actual ceph networking is on 10gb/s, and on ssd pools i reach up to 8000 iops on rbd images.
Thank you !
 
Last edited:

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
2.5” SAS 10K would be ideal, server grade ones, but they are not so cool,running etc and 15mm thick. (Ok your 5tb ones will be also), they only come in upto 1.8tb per disk so if you need space you will need a lot more than 5tb disks.
 

MiniKnight

Well-Known Member
Mar 30, 2012
3,072
973
113
NYC
2.5" good drives are usually SAS.

3.5" spinnies are good for SATA

Cache is good to have regardless.
 

skunky

New Member
Apr 25, 2017
24
1
3
43
Thank you !
Can you point me out to a model for as example ?
I mean some particular 2.5 spinning model that you guys succesfully used for Ceph/Bluestore rbd pool.
 

skunky

New Member
Apr 25, 2017
24
1
3
43
Can't tell about the budget, i'll need to define the options first. So basically there are: 4 nodes, 1 x ssd + 3 x 5tb Seagate - each. There are other 3 nodes pariticipating to sata pool but those have classic 2TB WD Caviar. The plan is to take out the Seagate SMR drives first, before i kill them all in ceph. They would be good in other envs like write-once-read-many - wich is not the case with rbds in ceph...
So preferabilly would be to get as close as possible up to 5tb, but in a decent, acceptable write-intensive way, so the vms that have their hdds stored on sata rbd will be happy.
 

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
SATA or SAS and required 'usable' capacity?

I think the WD RED 2.5" drives were 'supposedly' decent, has NASWare thought they made a 2TB model but maybe I am imagining things.

Option 1 (SATA): WD10JFCX

If this helps at all: WD Red 2.5" 1TB HDD Review (WD10JFCX) | StorageReview.com - Storage Reviews

Option 2 (SAS/SATA variants):

https://www.seagate.com/files/www-c...se-capacity-2-5-hdd/en-us/docs/100751314g.pdf
https://www.seagate.com/files/www-c...se-capacity-2-5-hdd/en-us/docs/100751316e.pdf

EDIT: First things first, if you need more than 2TB 2.5" magnetic drives, good luck w/ anything performing it seems, better to go AFA/all-flash and just pop in larg'er' capacity SSD's but we all know that's $$$. Pick your poison you can't have it all WRT price/performance/capacity 99.999-infinite% of the time. :-D
 

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
WD have a blue/green 2tb model that was ok, 15mm think though, I have a dozen of them still. But they are old not, I could imagine there is better options, or maybe not since they were Moore platters and PMR.
WD20NPVZ or WD20NPVX
Honestly outside of the Seagate and HDS C10K1800 or similar 10k SAS drives at 1.8tb the rest of larger capacity is mostly consumer sata as far as I know.
 

skunky

New Member
Apr 25, 2017
24
1
3
43
Honestly outside of the Seagate and HDS C10K1800 or similar 10k SAS drives at 1.8tb the rest of larger capacity is mostly consumer sata as far as I know.
Thanks, i have that feeling too. I've seen those Seagate Exos at 2.4TB, but way too expensive. I'll try to restrict search area withing 1.8 10k sas ( hopefully my LSI2008 controllers will be happy with them as sas or advanced format ).