Enterprise SSD "small deals"

[luckylinux]

The Chenbro NR400700 motherboard tray is essentially a full width (~17") 3U open chassis, since the power supplies sit "below" it, making the whole system 4U.

At one point, I had a stack of HGST SAS SSDs mounted next to the motherboard...

I guess I would end up with Split Systems for HDD and NVMe.

But IF (do NOT tempt me ) I would do something like that, I'd probably just put the Motherboard in an Open Frame on an Acrylic Plate or something, just to have the NVMe Drives via PCIe. A Chassis like the SuperChassis 213XAC-R1K05LP goes for 1200-1600 EUR so nope .

I don't want to hijack the Thread.

Nice Build though

 

[asgardthor]

This is getting above my head haha
I would say this my measurement is decently accurate, it's from the unifi PDU, and adding up both wattages

 

[ca3y6]

Samsung MZ-ILT3T2A 2.5" 3.2TB SAS 12Gb/s SSD $140

 

[luckylinux]

The thing is that the 48x spinning rust will still need to be connected to it. And the Chenbro is compatible with full height cards. To split this up, it'd end up taking 8U of rack space per SAN node...

I said **I** would, also because I don't have access to that cheap Chassis .

That being said I definitively have more experience in NAS than SAN. The single Point of Failure of Networking (even with dual NICs, dual switches + LAGG + Spanning Tree Protocol etc) makes it highly unpractical for storing VMs over iSCSI/NVMeOF, unless it's as Part of a CEPH Cluster or similar, which typically requires more than 5-7 Nodes, preferrably Symmetric / Balanced (no BIG.little ).

I haven't even started with that (NVMe) so I probably have a few Concepts that are somewhat wrong.

 

[jhartbarger]

Intel Optane 905p 480GB $63.99 QTY: 14

Pardon the Interruption

 

[nexox]

Intel Optane 905p 480GB $63.99 QTY: 14

Pardon the Interruption

I don't need those, I definitely don't need those, I have like three Optane drives I'm not even using yet. Maybe I need a couple of those.

 

[Cruzader]

How good are these Prices ? It seems VERY good to me. Where's the Catch ? Old Generation, Compatibility Issues, etc ?

The catch is that they are lower performance and endurance than the typical 1.6tb/3.2tb cards.

There are also still a bunch of surplus/new stock optane hitting the market, so you can get a 905p 280gb at 90-100$ while you tend to not know the wear of used cards.
For endurance there are also surplus/new P4800x 375gb u.2 drives at around 120-130$ with a nice 20.5PBW (and x4 cards at 180-190$).

 

[luckylinux]

It's really not. CEPH has its place, but an active-active SAN does make things fairly easy. I'm using Starwind SAN on these nodes (on Proxmox as the base/bare metal) and the rest of the Proxmox cluster is completely run over ISCSI (for boot and a few other things) and will run NVME-oF shortly (which is why this upgrade is taking place).

But even with that, a cluster aware file system is really not that critical in my case, because almost all compute in my application writes to the Postgres databases (and I'm moving to an active-active multi master for that as well). It's only the Postgres servers that need ultra fast, low latency disk access.

Without Cluster if your SAN goes down each and every VM on every cluster Node will crash/panic since it won't be able to write any longer, wouldn't it ?

Are you using the Free Version ?

https://www.starwindsoftware.com/starwind-virtual-san-free

Management via PowerShell . It's true Powershell can also be run on GNU/Linux but I have a bad Feeling should I try that. I don't use Windows at Home any longer for several Years.

 

[BackupProphet]

Intel Optane 905p 480GB $63.99 QTY: 14

Pardon the Interruption

Thanks, got 6. Hopefully they have some endurance left.

 

[BackupProphet]

Optane drives are great for cache, key-value persistent storage, streaming data. Applications like dm-pcache, rocksdb, kafka/pulsar
For ZFS log, its overkill.

 

[luckylinux]

That's why it's active-active... Two SAN nodes, real-time replication between them and ISCSI multi-path. If one goes down...the cluster doesn't even hiccup.

Yes, I'm using the free version, but you're a bit behind the curve. The KVM version of Starwind SAN has no restrictions...It allows HA LUN creation via the web-gui... No Windows needed (although they do use wine internally in the VM...which is ...well, meh.)

Thanks for the Explanation . I actually have 3 x Supermicro SC216 so up to 3x24 SATA/SAS SSD Bays there to put to good use .

Plus the Supermicro X10DRi with 3xIntel P4608 6.4TB (2x3.2TB) that I just assembled last Week. I'd need another good Deal on those 6.4TB Drives soon .

Of course everything will be Network Bottleneck like crazy in my Case, 10gbps Switch is the Max I got. And 100gbps Switches are too Noisy for my House I believe ...

 

[Markess]

Optane drives are great for cache, key-value persistent storage, streaming data. Applications like dm-pcache, rocksdb, kafka/pulsar
For ZFS log, its overkill.

…and boot/data drives on your kids gaming rig. Every little bit of speed helps. Especially when you run across brand new ones on eBay for less than $50. I’m still not sure what that seller was thinking. But I was his best customer…for about 5 minutes.

 

[Fritz]

…and boot/data drives on your kids gaming rig. Every little bit of speed helps. Especially when you run across brand new ones on eBay for less than $50. I’m still not sure what that seller was thinking. But I was his best customer…for about 5 minutes.

Yea, just like a lady of the night.

 

[josh]

Optane drives are great for cache, key-value persistent storage, streaming data. Applications like dm-pcache, rocksdb, kafka/pulsar
For ZFS log, its overkill.

Aren't optanes collectables now

 

[luckylinux]

Where will you be consuming that storage from? Might be worth doing server to server 56gbps or 100gbps connections, all you need is 3 cards with double ports, and each server is connected to each others.

Short Answer is: mostly (> 70% ?) in the same Room.

I got 4 x ConnectX-4 Dual Port 100gbps I could use for Point-to-Point, but definitively not for many Hosts.

I also have an Infiniband Switch & Infiniband ConnectX-2 I believe 40gbps HCA I bought a long Time ago and never used because the Fans on that Switch is absolutely horrendous.

EDIT 1: Switch is Voltaire 4036 if anybody was wondering. Still pretty sure I disassembled the Fan Cage to try to silence the Beast around 10 Years ago and never managed to go forward with that Project .

 

[Wasmachineman_NL]

…and boot/data drives on your kids gaming rig. Every little bit of speed helps. Especially when you run across brand new ones on eBay for less than $50. I’m still not sure what that seller was thinking. But I was his best customer…for about 5 minutes.

I use a 380GB M.2 Optane as my VRCCache drive and couldn't be happier tbh, wish it was bigger though.

 

[Fritz]

Aren't optanes collectables now

Dunno, but I'll as at the next National Nerd Convention.

 

[Markess]

Aren't optanes collectables now

My 480GB HHHL cards are PCIe 3.0, so yeah they aren't the fastest anymore. But back in 2020 when I got a handful of them new for $44 each, they were a pretty OK deal IMHO. Same for the "new", well unused at least, 1.2TB SSD 750s for $65. And everything is still chugging along. Since they're in desktops that get turned off, POH hours are low, and the sun will go dark before they hit the write endurance at the rate they're going.

Dunno, but I'll as at the next National Nerd Convention.

Make sure you're in the "I'm too cheap to buy current" breakout session. I went to the "I like new" breakout last time and the stuff they were talking about cost more than my house.

 

[percoseth]

Intel Optane 905p 480GB $63.99 QTY: 14

Pardon the Interruption

Damn! I missed them. I've been looking for a good deal on some optanes.

 

[luckylinux]

They accepted my offer for 2 at 130 per, total 260.

I offered 80 / Piece for 6 Pieces so I guess I'm automatically rejected .

 

[EasyRhino]

EDIT 2: first they say they will take a Picture and make 100% sure this Time that they send the correct Article over, then all of a sudden they Issue a (complete) Refund. So I'm left with 3 x 800gb OLD Intel P3500 Drives for which I paid Shipping & 75 USD VAT ... so basically roughly 30 USD / Piece.

You could probably push back harder against the seller or go through ebay. For instance, rebating shipping should be automatic since they shipped the wrong item. Ebay would probably let you insist on an actual full return, although you'd probably have to pay return shipping.

 

[ca3y6]

Another Lenovo / Micron 3.84TB 5210 ION SSD $150

 

[luckylinux]

That's why it's active-active... Two SAN nodes, real-time replication between them and ISCSI multi-path. If one goes down...the cluster doesn't even hiccup.

Yes, I'm using the free version, but you're a bit behind the curve. The KVM version of Starwind SAN has no restrictions...It allows HA LUN creation via the web-gui... No Windows needed (although they do use wine internally in the VM...which is ...well, meh.)

Just to be sure: which one of these are you using ?
- Virtual SAN Free (VSAN Free)
- Virtual HCI Appliance Free

It looks that I need to register with all my (full) Contact Details either Way .

 

[jhartbarger]

LENOVO/Micron 5300 Pro 3.84TB SATA 6Gb/s 2.5" Non-SED SSD MTFDDAK3T8TDS, Seller accepted $150 per drive but no idea on drive's POH or TBW but they accept returns if not satisfied. Drives have a MFGR date of 5/2022 so hopefully they have low(er) use on them.

Pardon the Interruption QTY: 5 Left as of this posting.

 

[Fritz]

10 competing Offers currently going on .

Well ... I wouldn't offer above 80 USD anyways without even knowing the Status of the Drive so ... Pass .

There are very few drives listed on eBay that show the status for whatever reason. I recently bought 2 14TB drives after asking for test results and promptly receiving them. I thought the seller had something to hide but this proved not to be the case. Bottom line is it never hurts to ask.

 

[jhartbarger]

There are very few drives listed on eBay that show the status for whatever reason. I recently bought 2 14TB drives after asking for test results and promptly receiving them. I thought the seller had something to hide but this proved not to be the case. Bottom line is it never hurts to ask.

Most sellers with something to hide don't accept returns, I decided to gamble and grab some of them even though they don't have any health info provided since they take returns. Best case I get drives that were barely used and worst case it costs $10 to send them back.

 

[ca3y6]

Another Lenovo / Micron 3.84TB 5210 ION SSD $150

And more…