Intel DC S3500 vs Kingston HyperX for OS/SLOG drives

[zogthegreat]

Hi everyone!

So I've been slowly rebuilding/upgrading my home server. I've been buying a few parts every month and this month I'm looking at upgrading my 32gb SATA drives that run my OS, (Proxmox), with something a bit better. I will also be adding a SLOG drive for my ZFS array. After doing some research, I have two choices that fit my budget, the Intel DC S3500 300gb or the Kingston HyperX 240gb.

As a general rule, I avoid Kingston due to the fact that their warranties sucks, (I recently tried to RMA a 16gb USB stick and they wanted the original receipt from 4 years ago). But since I'm buying the drives used, the warranty isn't an issue.

Another choice that would fit my budget are some used 340gb Fusion ioDrive's. I have the PCI-e slots on my motherboard, (Asus Z9PR-D12), however I've never used these types of drives, so I'm a little wary of them.

Suggestions, comments and philosophical musing's would be appreciated!

Thanks!

zog

 

[i386]

Intel DC S3500 vs Kingston HyperX for OS/SLOG drives

This is an unfair comparison: the s3500 is an enterprise ssd with powerloss protection and the hyperx is a consumer ssd.
Why the ssds with plp wins against the consumer ssd: https://www.servethehome.com/what-is-the-zfs-zil-slog-and-what-makes-a-good-one/

I will also be adding a SLOG drive for my ZFS array.
Suggestions, comments and philosophical musing's would be appreciated!

Or questions?
How is the zfs array setup? Raidz2? Mirrors?
What are you doing with your homeserver? Is it just a fileserver?
Are you using 100GBE?

 

[gea]

Yes, a choice between a slow but working slog (dc 3500) and a worthless one (why do you want to enable slow sync write without proper slog protection of the rambased write cache in case of a crash?).

Either forget the high end ZFS feature secure sync write or use a decent/fast/with plp slog (dc3700, WD SS530, Optane - best 4801x or at least up from 800p)

An Slog is like a parachute only for emergency cases (Crash). Either you do not need/want or you use one made from silk and not paper.

 

[zogthegreat]

Hi @i386 !

Thanks for the information, you answered my question by pointing out that the Hyper X is consumer crap. I will go with the S3500 drives, (unless there is a stronger recommendation for the Fusion-IO drives).

My current zpool is raidz2 mirror setup with 6 x's 2TB drives that I will be upgrading this year to some used 4TB data center drives. I am, ahem, somewhat financially challenged these days, so used DC drives are all I can afford this year coming. This is the command that I used to configure the zfs pool:

zpool create rz7TB -o ashift=12 raidz2 /dev/disk/by-id/scsi-SATA_ST2000DL003-9VT1_5YD6FRG9 /dev/disk/by-id/scsi-SATA_ST2000DM001-9YN1_S240EP7J /dev/disk/by-id/scsi-SATA_ST32000542AS_5XW1GQ8L /dev/disk/by-id/scsi-SATA_ST32000542AS_5XW1GQDQ /dev/disk/by-id/scsi-SATA_ST32000542AS_5XW1Q81N /dev/disk/by-id/scsi-SATA_ST32000542AS_5XW1Q972

zfs set compression=lz4 rz7TB

(I am open to any suggestions on how I set this up).

Currently, I don't have a SLOG drive setup, but I want to get one configured and play around with it before I add the new 4TB drives.

As for what I use this for, I have Proxmox running Ubuntu Server for a local file samba server and I am also running three other VM's, one VM is running Mac OS X and the other two are running Windows 10. All of them have passthrough video and USB 3.0. I am also planning to setup a NextCloud server for my daughter, because a teenager takes A LOT of photos and I'm tired of paying Apple more money every month after I gave them a $1000's for my daughters iPhone!

Everything is on a 10/100/1000 network, because right now I don't need anything faster. I'm using a Dell PowerConnect 5012 that my son tossed to me, (if you raise your children right, they give you hardware in your old age! ). It's old, but does the job for me.

 

[T_Minus]

$50 FusionIO II or older SLC variant would be > than S3500 for sure
S3700 200GB (200gb not 100GB for better write perf. is a good SATA choice > than S3500 with minimal cost over.)

 

[zogthegreat]

Yes, a choice between a slow but working slog (dc 3500) and a worthless one (why do you want to enable slow sync write without proper slog protection of the rambased write cache in case of a crash?).

Either forget the high end ZFS feature secure sync write or use a decent/fast/with plp slog (dc3700, WD SS530, Optane - best 4801x or at least up from 800p)

I'm a beginner to zfs, that's why I'm asking questions. What I want is a home server that can act as a file server and also run about 5 - 6 VM's. I'm using an Asus Z9PR-D12 motherboard with an Asus PIKE2008 RAID controller that has been flashed into IT mode. It has 2 Xeon E5-2620's 2.0 ghz CPU's and 70gb DDR3 ECC. The CPU's have been doing what I need, so I haven't upgraded them yet.

I'm using 2 x's 32GB SSD drives for my OS, (mirrored), and 6 x's 2TB Seagate consumer drives, with 2 spares in case of drive failure. I am planning to upgrade the 2TB's with 8 used 4TB Western Digital RE datacenter drives. Based on my research, for the home setup that I will be using, these drives will be "fine".

An Slog is like a parachute only for emergency cases (Crash). Either you do not need/want or you use one made from silk and not paper.

Hmm, they never mentioned paper parachutes when I went through jump school! They did mention that a really strong parachute was a good thing, sooooo

From what I see in your answer, it would be better to avoid the SC3500 for a SLOG drive and look for an DC3700 drive instead? Would a Fusion IO drive fulfil the requirements? I've seen 345GB Fusion IO's for around $45 USD. What would be the best SLOG drive to get on a budget, in your opinion? Please remember that I have a teenager that likes Apple products!

(why do you want to enable slow sync write without proper slog protection of the rambased write cache in case of a crash?).

Would you happen to have any links that I could follow up on about this? It looks like something I should know before I set things up.

 

[gea]

Basically you only need to know about ZFS

Due Copy on Write ZFS can avoid a corrupt filesystem per design when it crashes even during a write (unlike older filesystems where you can expect a damaged filesystem that requires a fschk to fix after a crash without guarantee of success).

ZFS is high security not high performance. To overcome a performance degration due copy on write or the additional amount of data due checksum it uses superiour rambased read and write caches.

On a crash during a write, ZFS remains always intact but the content of the rambased write cache with commited writes is lost (can be several GB). If you want to be sure that all commited writes are on stable disks, you can enable sync write. This is save but really slow on disks (say 10% of normal nonsync write performance).

To avoid the massive performance degration of sync write you can use an Slog. This is not a write cache but a logging device to protect content of write cache. On a crash its content is written to pool on next reboot. During regular operation it is never read so not a cache device! Think of it like a cache+BBU protection on a hardware raid card. As the Slog must only protect ramcache (several GB), its size can be quite small (2x write ramcache, up from 8 GB). A normal SMB filer (mostly) does not need or want a slow sync write. Only VM storage and databases really need it.

While ZFS remains always valid, a VM filesystem like ext4 or ntfs on top is not save after a crash. If you want to avoid a corrupted VM filesystem you must enable sync and if you want a decent performance on disks you need an Slog.

An Slog needs powerloss protection, low latency and very high iops on steady write or it is useless.

As an Slog is only there to protect against a powerloss/crash it must be itself powerloss protected.

 

[itronin]

This is quite possibly the best worded most concise description of this topic that I have read! Thanks @gea !

 

[zogthegreat]

OK, thanks for the advice everyone, especially to @gea, nice explanation that you gave me there! After reading up on everything that all of you have suggested, I'm thinking of going with the following:

2 Intel SSD DC S3700 200GB (SSDSC2BA200G3) @ $40 USD each - for my OS drives, (mirrored). This will give me ample space for the OS, plus storage space for ISO's.

1 Fusion ioScale 320Gb MLC (FS1-004-320-CS-0001) @ $40 USD for my SLOG drive. I have plenty of unused PCI-e slots on my motherboard, so this will free up a SATA port on my system.

Thoughts?

 

[Magic8Ball]

You might want to double check compatibility because from memory I don't think the Fusion II drives work with the latest versions of debian/ubuntu, at least not without community developed patches. There's a massive thread on here about these drives and their different versions and compatibility etc.

 

[T_Minus]

You might want to double check compatibility because from memory I don't think the Fusion II drives work with the latest versions of debian/ubuntu, at least not without community developed patches. There's a massive thread on here about these drives and their different versions and compatibility etc.

Great point! I didn't think of this potential conflict.


@zogthegreat You'll be happy with that setup for sure I run the mirrored s3700 and mirrored s3500 (for os) on dozens of builds.

 

[zogthegreat]

You might want to double check compatibility because from memory I don't think the Fusion II drives work with the latest versions of debian/ubuntu, at least not without community developed patches. There's a massive thread on here about these drives and their different versions and compatibility etc.

Hi @Magic8Ball , do you have a link for the thread that you referenced? I did a forum search and ended up with a lot of advert's selling the drives, but not much info!

However, doing a quick Google search turns up that to use a Fusion drive with Proxmox, you need to compile your own drivers for the current kernel. Every time there is a kernel update, you need to recompile the drivers. Not that difficult, but kind of a PITA.

You'll be happy with that setup for sure I run the mirrored s3700 and mirrored s3500 (for os) on dozens of builds.

@T_Minus I'm thinking about what you said here and maybe it would be best to use one Intel S3700 to run my SLOG on. Based on what I have read, I won't need a L2ARC drive for my setup... (please feel free to correct my (mis)understanding of this! ). I can get a 400gb drive @ $60USD. Since the price difference between the S3500 240gb and the S3700 200gb is only $10, would it be better to get 2 S3700's? They are "newer" than the S3500, although with used drives, "newer" isn't really the thing to worry about, read/writes on the drives is what matters. However, the electronics should be better on the S3700 than the S3500. The $20 that I save could go to memory or an additional video card for passthrough or.... well you guys understand, $20 is $20!

Also I noticed that you said that you mirror your SLOG drive. Is this something that I need to do, or something that's just "good practice"?

 

[T_Minus]

If it's only $10 more for S3700 I would do that over S3500 due to the increased write performance but also the MUCH greater endurance of the drive itself. I don't think mirroring the SLOG is needed unless it's production.

I've used s3500 mirrored and s3700 for OS and really no difference unless you're after most performance, longevity and\or logging a lot then for sure the s3700, and for $10 difference I'd go S3700 I'm looking for some s3700 for some other members here, if I find any extra I'll shoot you a PM and price is always lower than ebay, aside from crazy ebay deals

 

[Magic8Ball]

Hi @Magic8Ball , do you have a link for the thread that you referenced?

Try here: Fusion-io ioDrive 2 1.2TB Reference Page | ServeTheHome Forums

I have a 3.2TB version running fine in Win 10 after I'd gone through a few initial steps to get it working. Back then there was some uncertainty about getting them to work on linux (eg here) but that may or may not have been resolved as I haven't followed that thread in a while. Personally I don't think it's worth the hassle to recompile the kernel - just more things to go wrong and maintain over time.

 

[zogthegreat]

Thanks @Magic8Ball . I agree that recompiling the drivers every time that there is a kernel update is to much of a PITA for a server that I want up 24/7. Although it would have been fun to play with new hardware, (full disclosure, I'm a "hardware whore" ), I think that I'll go with 2 S3700 200gb drives for my OS and a 400gb S3700 for my SLOG. Keep things simple.