$4K SLC SSD PCIe cards for $350 on eBay

[dba]

If money were no object, you'd buy flash storage with SLC memory instead of MLC. SLC flash lasts around 30 times longer than MLC, which means you can abuse it as much as you want and basically never have to worry about it wearing out. The ultimate ZFS write cache (ZIL)? SLC flash memory of course. The ultimate OS swap drive? SLC flash memory again. Unfortunately, SLC drives are shockingly expensive, especially high performance versions. Oracle would love to sell you a few model F20 PCIe cards from its million-dollar Exadata V2 data warehouse machine for $4,695 each. Ouch.

But, once again, eBay vendors come to our rescue. Right now I see some of these Sun F20 SLC flash PCIe cards for $350. Another vendor has them for $360. Of course if you prefer to buy new, there is that $4,695 option still available.

I'll take one for $350. For that money you get a PCIe card that presents four 24GB flash drives to your server. 24Gb is pretty tiny, but remember that these are SLC based drives, with the kind of reliability that you need for a ZFS ZIL write cache for example. Published specifications say that in aggregate these four drives are good for around 1.1GB/Second sequential reads and 567MB/Second sequential writes or 100K random read IOPS and 88K random write IOPS. Those are good numbers by any standard, but then you remember that this is SLC memory good for 100,000 write cycles whereas consumer-grade flash tops out at around 3,000 cycles.

I can't recommend these for general storage - at 4x24 GB they are simply too small. The also aren't the absolute fastest PCIe flash cards available - the latency is a little high by today's standards and other newer cards thrown down better benchmark numbers, albeit for much more money. But, for a high performance, high reliability ZFS ZIL made with SLC memory, $350 is an absolute steal. And as a bonus, the card has two SAS SFF-8087 disk ports built in and it acts as a SAS/SATA2 (3GB/S) host bus adapter. For my part, I'm picking one up as a server swap drive, where its reliability is also very useful.

Data Sheet:
http://www.oracle.com/us/products/servers-storage/storage/flash-storage/f20-data-sheet-403555.pdf

White Paper:
http://www.oracle.com/technetwork/a...re/flash-accelerator-f20-pcie-card-163877.pdf

 

[dba]

Sorry about the error in the title - they are $350 and not $250. Unfortunately, titles are not editable.

If money were no object, you'd buy flash storage with SLC memory instead of MLC. SLC flash lasts around 30 times longer than MLC, which means you can abuse it as much as you want and basically never have to worry about it wearing out. The ultimate ZFS write cache (ZIL)? SLC flash memory of course. The ultimate OS swap drive? SLC flash memory again. Unfortunately, SLC drives are shockingly expensive, especially high performance versions. Oracle would love to sell you a model F20 PCIe card from its million-dollar Exadata V2 data warehouse machine for just $4,695. Ouch.

But, once again, it's eBay vendors to the rescue. Right now I see some of these Sun F20 SLC flash PCIe cards for $350. Another vendor has them for $360. Of course if you prefer to buy new, there is that $4,695 option still available.

I'll take one for $350. For that money you get a PCIe card that presents four 24GB flash drives to your server. 24Gb is pretty tiny, but remember that these are SLC based drives, with the kind of reliability that you need for a ZFS ZIL write cache for example. Published specifications say that in aggregate these four drives are good for around 1.1GB/Second sequential reads and 567MB/Second sequential writes or 100K random read IOPS and 88K random write IOPS. Those are good numbers by any standard, but then you remember that this is SLC memory good for 100,000 write cycles whereas consumer-grade flash tops out at around 3,000 cycles.

I can't recommend these for general storage - at 4x24 GB they are simply too small. The also aren't the absolute fastest PCIe flash cards available - the latency is a little high by today's standards and other newer cards thrown down better benchmark numbers, albeit for much more money. But, for a high performance, high reliability ZFS ZIL made with SLC memory, $350 is an absolute steal. And as a bonus, the card has two SAS SFF-8087 disk ports built in and it acts as a SAS/SATA2 (3GB/S) host bus adapter. For my part, I'm picking one up as a server swap drive, where its reliability is also very useful.

Data Sheet:
http://www.oracle.com/us/products/servers-storage/storage/flash-storage/f20-data-sheet-403555.pdf

White Paper:
http://www.oracle.com/technetwork/a...re/flash-accelerator-f20-pcie-card-163877.pdf

 

[RimBlock]

Nice although someone else is selling Intel 313 SLC 24GB OEM drives for US$70 (US$80 BiN).

They are only SATA II but for $80... (only one drive available though).

Edit:

Hmm thinking about it, there are quite a few 20GB Intel 311s around for US$60. 8 of these on a M1015 striped for up to 2.4GB/s for around 128GB SLC storage @ around US$580 (assuming US$100 for the M1015). May be an interesting experiment for someone .

RB

 

[dba]

The eight Intels would be a fun experiment! It would likely beat the F20 in most benchmarks except for write IOPS, where the F20 would have a significant advantage. Personally I would not want to dedicate that much real estate to a ZIL - eight disk slots and eight cables plus a PCIe slot.

Nice although someone else is selling Intel 313 SLC 24GB OEM drives for US$70 (US$80 BiN).

They are only SATA II but for $80... (only one drive available though).

Edit:

Hmm thinking about it, there are quite a few 20GB Intel 311s around for US$60. 8 of these on a M1015 striped for up to 2.4GB/s for around 128GB SLC storage @ around US$580 (assuming US$100 for the M1015). May be an interesting experiment for someone .

RB

 

[Patrick]

Sorry about the error in the title - they are $350 and not $250. Unfortunately, titles are not editable.

Not sure if I can fix that as a global setting. I did make an edit for you though.

 

[Patrick]

Here's a search to pick up the ones that are out there: Sun 541-3731

One at $330 OBO right now (1 offer in)

ebay bucks on 1/3 :-/

 

[gigatexal]

its a sas hba too! interesting

 

[Aluminum]

its a sas hba too! interesting

Anyone have an idea what the controller chip is, LSI 1068 perhaps? The datasheet has nothing although the driver support means it must be something common.

Also with 4 drives onboard that means its half full, another chip or expander? Otherwise it doesn't make sense having two 4x ports.

Seller says 'like new', but even though they are SLC and if[/i] they have been used hard (like for nonstop busy database r/w) for the last few years they might not be so great. If they are actually just spare parts then thats great.

Real interesting find either way.


BTW for the DIY idea, don't forget to add the price of a bunch of msata adapters and cables etc.
Someone should make a PCB that takes 8 msata and has two SAS ports and a single power cable, would easily fit in a single 5.25" or maybe even 3.5" bay...I should plug that idea to HWtools considering all the other crazy stuff they make.

 

[dba]

Aluminum, the controller chip is an LSI SAS1068E - a rather older SAS controller chip but probably just good enough to handle the four flash modules on the F20. The 1068E supports eight SAS channels. In the F20, four of those attach to the four separate flash modules and four attach to a 12-channel expander chip, which breaks out the remaining eight channels to two four-channel SFF-808 ports. Of course if you do use the flash modules and the SAS ports at the same time, you will be loading up the 1068E chip enough to be significantly bottlenecked.


...and thanks for the pointer to HWtools, who seem to make some of the most obscure adapters that the world has ever seen. I can only speculate, but I think that their R&D organization consist of two large drums filled with technology words written on index cards. They select one index card from drum A and an second card from drum B, then - preso! - announce a new this-to-that adapter card. I wish that I knew about them earlier, because they make stuff that I could have used!

 

[Patrick]

Not sure but I bet the LSI 1068E would be able to handle for 3.0gbps generation small SLC SSDs.

The big question I have is what does the driver situation look like.

 

[dba]

Oracle specifies Windows 2008, RHEL 5.4+, SUSE, and of course Solaris compatibility. Since it is an LSI HBA with SSDs bolted on, I bet that it's broader than that.

Not sure but I bet the LSI 1068E would be able to handle for 3.0gbps generation small SLC SSDs.

The big question I have is what does the driver situation look like.

 

[mobilenvidia]

I saw these also the other day while searching for something else, nice to play with but would mean getting side tracked from other projects I've started but not finished

It being a SAS SLC SSD, very tempting now that I've had a closer look, would go very nicely with the 4TB WDC SAS HDD


Also looks to be a very early version for Fastpath/Cachecade, probably the device that started LSI in developing it into what we see now on the SAS2108/2208 controllers

 

[Patrick]

Oracle specifies Windows 2008, RHEL 5.4+, SUSE, and of course Solaris compatibility. Since it is an LSI HBA with SSDs bolted on, I bet that it's broader than that.

OK very interested now to see your results. Should be ridiculously good compatibility if there was no Sun firmware block.

 

[Patrick]

Prices from one seller dropped to $332.48 + 10 ship 5 left.

 

[RimBlock]

The eight Intels would be a fun experiment! It would likely beat the F20 in most benchmarks except for write IOPS, where the F20 would have a significant advantage. Personally I would not want to dedicate that much real estate to a ZIL - eight disk slots and eight cables plus a PCIe slot.

True but it could be used for a database storage space or the like. 128GB give a lot more options for use.

BTW for the DIY idea, don't forget to add the price of a bunch of msata adapters and cables etc.
Someone should make a PCB that takes 8 msata and has two SAS ports and a single power cable, would easily fit in a single 5.25" or maybe even 3.5" bay...I should plug that idea to HWtools considering all the other crazy stuff they make.

Yep although the 311 and 313 both come in SATA versions as well. Probably harder to find though. Syba do a mSATA to SATA adaptor for around US$10.

As they are bare cards you could use motherboard standoffs to stack them one on top of the other.

Having just checked the benchmarks for the 311 and 313 though, they are not that zippy speed wise .

Oh well .

RB

 

[Patrick]

Prices from one seller dropped to $332.48 + 10 ship 5 left.

Seller just counter offered at $298.

 

[gigatexal]

if this was a 2008 chipset i'd be all over this. How does ZFS say to know how to use this as a cache device? doesn't ZFS just cache almost everything it can in memory for files that are in high demand?

 

[dba]

I agree - a 2008 controller would be a dream! Of course that would make this pretty much an Intel 910 clone, a $2K current-model PCIe card that uses the same architecture but with cheaper MLC NAND and an LSI 2008 chip.

ZFS does make very good use of RAM to cache data for reads, but adding flash can be cheaper than adding RAM for such caching, an SSD L2ARC apparently speeds up de-duplication quite a bit, and of course you can't cache writes using volatile RAM like you can to a non-volatile flash device. Probably the best use of a small SLC flash drive is as a ZIL (fancy write cache).

BTW, Oracle puts four of these into each Exadata storage server. The late 2012 Exadata models use an updated version called the F40 I believe, a card that looks a great deal like an LSI Nytro, but everything earlier used these F20s.

if this was a 2008 chipset i'd be all over this. How does ZFS say to know how to use this as a cache device? doesn't ZFS just cache almost everything it can in memory for files that are in high demand?

 

[gigatexal]

what is needed to setup a ZIL with one of these? I am building a ZFS DAS and would love cache'd writes

just did some reading turns out ZIL is good for synchronus writes only not the tons of random writes I'll be doing

 

[mobilenvidia]

Seller just counter offered at $298.

Same for me, offered $250
Would be nice to have as a luxury but will side track me even further.
So will pass