I'm kind of regretting not buying some of those used 3.84TB sata SSDs when they were regularly $160.
Enterprise SSD "small deals"
- Thread starter ca3y6
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Welcome to the Club
. I only have like 8-10 x 3.84TB SAS3 + 8-10 x 3.84TB SATA3 of those.
If I hadn't just gotten a good deal on a pair of brand new 1.5TB p4800x drives this would be irresistible.
The latency difference is enough to make regular tasks noticably faster, kernel builds and running
find, that sort of thing, and lately prices are such that I would be looking at QLC SSDs to get 5x the capacity at the same cost, I haven't seen anything with comparable write life per dollar in the last 6-8 months. (edit: except maybe those drives that keep popping up in this thread, but they're never in stock when I look. edit 2: even those are 28PBW vs the 164PBW for my p4800x drives.)I admittedly don't have any huge NVMe drives to compare, but my sx350s roughly match the p4800x specs in most aspects (and where they fall short of the Optane drives they're well ahead of TLC,) yet have 10x the read latency, they're noticably slower for any workload that hits lots of small files or a bunch of filesystem metadata.
Back from my Gentoo Days I would always build on
tmpfs (RAM).I'm not sure why you would use a (comparatively) much slower NVMe Drive. Just build in RAM, create an Archive, then move to persistent Storage.
I will probably have to look into the same with some CI Server (Forgejo Runner). Right now I'm just doing my "Default" Podman Setup.
I often rebuild after minor changes so persisting all the intermediate build files speeds things up, also
tmpfs isn't always that fast, I run containers with tmpfs volumes for bazel builds with a network cache (so I don't need to persist files locally) and just cleaning up a 64GB tmpfs volume can take 20-30 seconds - if I had Optane drives at work I would not be surprised if they built faster over all.Nice, the ones I bought in China just arrived in the warehouse. Cost about $18 to reship.
NVMe faster than RAM ? Weird. Also that should only be the Case if you are running out of RAM and are rebooting the Server often.
RAM is faster, the tmpfs implementation just has some slow parts, plus you're still copying the files from NVMe to tmpfs before you can do anything and then at least some files back again when you're done.
I don't know how much RAM you have but a kernel build will occasionally hit my cgroup limit of 128GB on its own if there are a lot of options enabled and I run one
make job per.core, that doesn't leave a ton of memory to waste on long term storage of intermediate build artifacts (and at work it's all ephemeral instances anyway.)Fair Point
. Or to/from SATA for me. Just installing Kernel Headers on Linux takes forever for me .I just installed Proxmox VE on the Hyve Servers I bought a while ago (on an NVMe Mirror out of 800GB / 960GB / 1.2TB / 3.2TB / 3.84TB, depending on what I had available) and now I'm basically out of NVMe, except 5-6 x consumer Crucial P5 Plus 2TB. The SAN Server will basically be only SAS 12G / SATA 6G plus maybe 3 x (6.4TB PCIe, though that is actually 2 x 3.2TB). No more NVMe available. Prices are currently crazy and Wallet is empty
.It depends on the Host.
32GB on some Supermicro X10SLL-F/X10SLM-F Motherboards
64GB on Supermicro X11SSM-F/X11SSL-F / AMD Ryzen 3600/3700X Boards.
All the Way up to 512GB DDR4 on some AMD EPYC 7642 Systems (Hyve).
To stress test a CPU and check Thermals I typically just build the Linux Kernel, with default Options.
I never had a Problem building the Linux Kernel (6.12.23) on 16GB tmpfs on one of the smallest Systems.
128GB RAM for a Kernel Build
? Holy smokes !That was me who bought 2 drives out of 3 available. I don't need more, won't snipe it again.