or...
Question the need for them in the first place.There's a time and place for hardware upgrades...and then there's a time for perfecting your software/admin/automation skills...
Enterprise SSD "small deals"
- Thread starter ca3y6
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Question the need for them in the first place.There's a time and place for hardware upgrades...and then there's a time for perfecting your software/admin/automation skills...
Even though I spotted this deal, I am banning myself from additional storage purchases until I can reach 10% usage of my current stackor...
There's a time and place for hardware upgrades...and then there's a time for perfecting your software/admin/automation skills...
Get it done my man! I know it's a slog...lots of F-bombs and "ahhh screw it, I'll do it tomorrow" type of reactions, but once done, it's...glorius.
Two racks...lost of cursing while getting them completed. Here's a funny story. In the compute rack (right rack in the picture above), I thought I had "finalized" my hardware config, built the sleds, mounted them in the rack, was gonna use iSCSI boot for everything (which did work as such), but...due to many different things, plans changed...
I ordered a whole bunch of those adapters that can hold one m.2 sata SSD and one NVME, so I can have SATA boot and one NVME in each sled. Then...had to unmount ALL of the sleds...one by one...add the adapter with SSD and sata cable...mount the sled back...
And there's a shit load of them...fun fun fun...
I've given up on Starwind. I realized I didn't really need high availability per se, a 15 minute rsync is more than enough to give me the data security I need (This is a mirror of my prod, so the data is already duplicated per se). So, once that requirement went away, I realized, well, why Starwind?
And it still doesn't do NVME-oF with HA, actually even without HA as well, still is stuck on the 20.04 kernel...no feature parity between different flavors (bare metal vs Proxmox VM vs Windows VM vs ...).
Moved to Truenas and it seemed to fit the bill...but then their RDMA restrictions kicked in...and the frustration began anew...
Been testing RHEL (specifically Almalinux) with Cockpit and surprisingly it works great! The installer is flawless, configuration is straightforward, the documentation is top notch, every flavor of RDMA under the sun, NVME-oF works OOTB...
I might stick with this, once I've put everything through its paces.This reminds me of the first clusters i built that just used 1meter long threaded rods through all the mobo screwholes and nuts to lock them in place.
Was oh so fun when needing to remove a mobo half.
Then moved onto something closer to your shelf with screwing it all onto plates, slightly less wobbly.
Container host...that's the difference. I thought we were talking about a storage server? i.e. You want to use Starwind for a container host?Yeah, but unfortunately you know ... Priorities
That's somewhat of an off Choice (IMHO). I'm mostly using Debian, Proxmox VE and Fedora (the latter for Podman Containers). I gave a very short try to Alpine/Almalinux/OpenSUSE as Container Hosts but quickly gave up on them.
I still also need to put the 24x AMD Ryxen 3500X/3600X Systems into Operation. Funnily enough I was thinking of buying some SSDs but then the Prices started Skyrocketing (it was a while back, I know). So I guess I'll need to settle to some 1 x 128GB or 2x128GB NVMe Drive or something low-cost like that
I'm using Proxmox as well, for the container host side, but the storage is separate, connected under the hood with NFS over RDMA, and then to Proxmox, it's just a Directory.
This actually worked out quite well. Part of my problem was noise and cooling, and to a lesser degree, finding the right chassis for a varied configuration of motherboards, serving different purposes.
In the end, I separated compute and storage. That left rack is ALL storage...like ~200 LFF bays worth.
The right rack is all compute, with those sleds. That right rack has six 14 inch fans cooling it, and these are pretty quiet. Not like "silent" (the amount of airflow itself makes a fair amount of noise), but given the size of the fans, it's a low frequency "hum", rather than screaming fans.
(Excuse the mess in the picture, was still in the building stage).
Now, everything is on a 56gb backbone...and life is...good.
Agreed, that's just a pain.
No. "RHEL", not Fedora. Yes, I know the differences (and similarities) between them. Almalinux is essentially identical to RHEL, without the licensing. Yes, there's a few minor differences, but they really are minor.
My reason for choosing Almalinux at home? I foresee moving to RHEL for my prod side, and pay for support. This way both environments are identical. My compute needs, apart from a couple of Proxmox servers running various containers and VMs, are essentially running a monolithic app on each Linux server. This app is my own code and part of my business.
Im also on Alma since that is what i use at work.
My impression is that alma "won" the enterprise side not paying for RHEL when centos as we knew it died and there was the alma or rocky decision on who to go with now.
My impression is that alma "won" the enterprise side not paying for RHEL when centos as we knew it died and there was the alma or rocky decision on who to go with now.
Last edited:
Not sure I would move to paid RHEL when I could pay for Alma support. I'm moving some things over to openSuse now as I'm learning other things that have Suse as a base. Alma would be next if I had to move and Debian is still mostly what I use for real things.
wouldn't register as a good deal 6 months ago but in those times Toshiba 3.84TB 2.5" 12Gb/s Enterprise SAS SSD PX05SRB384 $185
So Harvester and Rancher that's a little more mature. Harvester is moving fast, good and bad things about this. Will be nice when it just merges both parts to make life easier, but I'm guessing sales got in the way and forces them to remain distinct products. Harvester already has a bunch of Rancher stuff in it, seems like it shouldn't be hard to put the rest in place and be done. It will require a more stout host to run it all, but in most of the industry, this should be a problem.
Thats a cool rack. Do you have more pictures?Get it done my man! I know it's a slog...lots of F-bombs and "ahhh screw it, I'll do it tomorrow" type of reactions, but once done, it's...glorius.
Two racks...lost of cursing while getting them completed. Here's a funny story. In the compute rack (right rack in the picture above), I thought I had "finalized" my hardware config, built the sleds, mounted them in the rack, was gonna use iSCSI boot for everything (which did work as such), but...due to many different things, plans changed...
I ordered a whole bunch of those adapters that can hold one m.2 sata SSD and one NVME, so I can have SATA boot and one NVME in each sled. Then...had to unmount ALL of the sleds...one by one...add the adapter with SSD and sata cable...mount the sled back...
And there's a shit load of them...fun fun fun...
I've given up on Starwind. I realized I didn't really need high availability per se, a 15 minute rsync is more than enough to give me the data security I need (This is a mirror of my prod, so the data is already duplicated per se). So, once that requirement went away, I realized, well, why Starwind?
And it still doesn't do NVME-oF with HA, actually even without HA as well, still is stuck on the 20.04 kernel...no feature parity between different flavors (bare metal vs Proxmox VM vs Windows VM vs ...).
Moved to Truenas and it seemed to fit the bill...but then their RDMA restrictions kicked in...and the frustration began anew...
Been testing RHEL (specifically Almalinux) with Cockpit and surprisingly it works great! The installer is flawless, configuration is straightforward, the documentation is top notch, every flavor of RDMA under the sun, NVME-oF works OOTB...
It all started when I was trying to upgrade my storage...
Ended up with two racks and compute and storage separated.
MZ-ILS15T0 Samsung PM1633a 15.36TB SAS 12Gbps 2.5in SSD EMC 118000556 - BIG SALE | eBay
The MZ-ILS15T0 Samsung PM1633a 15.36TB SAS 12Gbps 2.5in SSD is a high-performance internal SSD designed for use with PCs. With a storage capacity of 15.36TB, this drive combines both HDD and SSD storage formats to provide fast and efficient data access. The black color and compact 2.5 inch form...
www.ebay.com
MZ-ILS15T0 Samsung PM1633a 15.36TB SAS 12Gbps 2.5in SSD EMC 118000556 - new $398 - zero feedback seller
I see that pattern quite often. New sellers with zero feedback selling dirt cheap stuff. It's obviously scam but never understood why they do that.
