Enterprise grade M.2 PCI-E SSDs that don't break the bank?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

msvirtualguy

Active Member
Jan 23, 2013
494
244
43
msvirtualguy.com
Hi all. First let me wish you Happy Holidays.

I'm starting to plan the final piece to lab conversion to Small Form Factor and that's my FreeNAS server.

I'm moving away from:

Supermicr CSE-825TQ-563LPB
Supermicro X9SRL-F
Xeon 2670 SR0KX
64GB DDR3 ECC 1333
SAS 2008
2 x 200GB Intel S3700
6 x 4TB Seagate NAS Drives
Intel X520-DA2


My new build will consist of:

Supermicro CSE-721TQ-250B
Supermicro MBD-X10SDV-4C-TLN2F-O
32GB DDR4-2400 (2 x 16GB)
4 x 4TB Seagate NAS Drives

I'm wondering, since i'll have more than enough RAM for Cache, I want to install a nice M.2 PCI-E drive for ZIL. Any out there that would be good for this use case? I only really see consumer grade drives out there.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
Could go with a m.2 adapter to run a 2.5" if you have room.
 

msvirtualguy

Active Member
Jan 23, 2013
494
244
43
msvirtualguy.com
I have 2 x 2.5 slots in that case so it's not about the room, just lookin for a smaller footprint, and take advantage of PCI-E. I wonder since i'm not writing a lot of data, if I can get away with a 950/960 PRO.
 

Terry Kennedy

Well-Known Member
Jun 25, 2015
1,140
594
113
New York City
www.glaver.org
I'm wondering, since i'll have more than enough RAM for Cache, I want to install a nice M.2 PCI-E drive for ZIL. Any out there that would be good for this use case? I only really see consumer grade drives out there.
The SM961 (OEM 960 Pro) would be a great drive and is under $100 for 128GB (and the price drops nicely as you go larger), but right now it experiences device errors / resets under FreeBSD (and thus, presumably, FreeNAS). I'm working with the developer on sorting this out, but it will likely be a few weeks (if you compile the kernel yourself) or potentially much longer for it to appear in a scheduled FreeBSD / FreeNAS release.

I don't know if the PM961 (OEM 960 Evo) has the same issue, but since it has the same controller it probably does.
 
  • Like
Reactions: Hal and Patrick

msvirtualguy

Active Member
Jan 23, 2013
494
244
43
msvirtualguy.com
Thanks Terry, great input here as I was looking at the PM961 128GB drive and was gonna pull the trigger. I'm in no rush for this project but like to get it done by end of January so maybe i'll shelf this and revisit. To be quite honest, I don't really know if i'm sticking with FreeNAS either.
 

sth

Active Member
Oct 29, 2015
379
91
28
isn't on device battery backup advised for ZIL's and if so, do M2 drives have any?

Also interested in what you are thinking of moving too Terry, if you do decide to leave FreeNAS.
 
  • Like
Reactions: Patrick

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
I was suggesting a m2 adapter so you could use a 2.5" NVME such as P3600 400gb.
 
  • Like
Reactions: Patrick

Terry Kennedy

Well-Known Member
Jun 25, 2015
1,140
594
113
New York City
www.glaver.org
isn't on device battery backup advised for ZIL's and if so, do M2 drives have any?
It is one of the characteristics reported in the Identify Namespace Data Structure, but the datasheets I have for the SM961/PM961 (unfortunately marked "Samsung confidential", so not posting here) don't specify if the bit is set. And the system I have the card in is at the office, powered down.

On the other hand, if my UPS blows up or both power supplies in the chassis fail, I'll have bigger problems than an incomplete write log.

By the way, hard drives suffer from power loss problems as well. You get "write splice errors" if the drive was writing when the power fails. At least modern controllers / drives give up cleanly and should only corrupt a single [physical, possibly 4K] sector. However, if the splice error extends past the user data portion of the sector, this may be an unrecoverable defect. In the "bad old days", you'd often get a nice spiraling pattern of corrupted sectors.
Also interested in what you are thinking of moving too Terry, if you do decide to leave FreeNAS.
I think you're confusing me with the original poster - I'm using FreeBSD and don't intend to change.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
@Terry Kennedy at this point I would imagine the bigger concern is the power distribution board.

@msvirtualguy the hard part of what you are asking is that there are still no great options due to m.2 physical space constraints.

My big suggestion is to over provision. I have seen 950 Pros fall over under heavy ZIL loads.
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
Please report some numbers if you settle on a drive:)
I had played with an 950 pro for Slog earlier this year and it was horrible, I ended up with a small S3700 (non nvme) b/c it was faster.
 

Terry Kennedy

Well-Known Member
Jun 25, 2015
1,140
594
113
New York City
www.glaver.org
@msvirtualguy the hard part of what you are asking is that there are still no great options due to m.2 physical space constraints.
Which options are being limited? The motherboard @msvirtualguy picked has M.2 2242/2280, and all of the SM961/PM961 are 2280 single-sided, from 128GB to 1TB (overkill for ZIL unless the speed advantage of the larger modules matters*). The lower capacity drives have a big empty "this space for rent" area instead of flash - I'm not sure why they didn't spread out the NAND for cooling purposes if they weren't going to shrink the form factor.

I'm a newcomer to M.2 (the SM961 is the first one I've tried, and in my case it is on a PCIe slot adapter card on an X8DTH-iF, so I'm not going to see top throughput anyway*), so I'm not familiar with the design trade-offs. But as a (mostly-retired) hardware designer, I don't know why they insisted on such a thin PCB (even compared to PCIe Mini) - a thicker board would allow a beefier ground plane for heat transfer. For that matter, none of the PCIe slot adapters except for the OCZ RD400A even bother with cooling vents on the bracket. Also, since there is a lot more flexibility in motherboard component placement and clearance, I would have considered designing things so the PLP capacitor could be adjacent to the M.2 slot on the motherboard. The PLP cap costs less than the M.2 connector, so it shouldn't have been a cost-based decision.

* The write speeds for the SM961 128/256/512/1024GB are quoted on the datasheet as 700/1400/1700/1800MB/sec. My array already does > 700MB/sec over the LAN (10GbE) with just spinning rust, so I'm really looking at write IOPS, spec'd as 170/280/300/320K. That's one reason to pick the SM961 over the PM961 - the respective numbers on the PM961 are 600/1100/1600/1700 (not bad) and 40/180/260/330 (yikes!, especially on the smaller models).
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
Limited in terms of options for having capacitors for the write cache. There are drives like the Seagate Nytro XM1440
which are M.2 and have capacitors but those are M.2 22110.

I do agree that I would love to see a 2280 drive with capacitors.
 

msvirtualguy

Active Member
Jan 23, 2013
494
244
43
msvirtualguy.com
I was suggesting a m2 adapter so you could use a 2.5" NVME such as P3600 400gb.
Ahh..sorry man...I think I may be limited on space due to thickness of those but i'd have to check.

FreeNAS 9.x is getting long in the tooth for simplicity and ease of use, not that I can't handle it but using Nutanix and comparing other platforms (yes I know FreeNAS is not on that level) they simply fall way short in the ease of use..etc.

Also..ZFS is not the easiest to expand or the most efficient, right? Like simply adding a single disk.

I don't know...maybe I will stick with the server and put it underneath the office piece, just don't know yet.

Do we have any ideas on FreeNAS 10.x GA?