Enterprise SSD "small deals"

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
L

luckylinux

Well-Known Member
Mar 18, 2012
1,757
585
113
nexox said:
I admittedly don't have any huge NVMe drives to compare, but my sx350s roughly match the p4800x specs in most aspects (and where they fall short of the Optane drives they're well ahead of TLC,) yet have 10x the read latency, they're noticably slower for any workload that hits lots of small files or a bunch of filesystem metadata.
Click to expand...
Back from my Gentoo Days I would always build on tmpfs (RAM).

I'm not sure why you would use a (comparatively) much slower NVMe Drive. Just build in RAM, create an Archive, then move to persistent Storage.

I will probably have to look into the same with some CI Server (Forgejo Runner). Right now I'm just doing my "Default" Podman Setup.
 
N

nexox

Well-Known Member
May 3, 2023
2,081
1,046
113
luckylinux said:
I'm not sure why you would use a (comparatively) much slower NVMe Drive. Just build in RAM, create an Archive, then move to persistent Storage.
Click to expand...
I often rebuild after minor changes so persisting all the intermediate build files speeds things up, also tmpfs isn't always that fast, I run containers with tmpfs volumes for bazel builds with a network cache (so I don't need to persist files locally) and just cleaning up a 64GB tmpfs volume can take 20-30 seconds - if I had Optane drives at work I would not be surprised if they built faster over all.
 
  • Wow
Reactions: luckylinux
L

luckylinux

Well-Known Member
Mar 18, 2012
1,757
585
113
nexox said:
I often rebuild after minor changes so persisting all the intermediate build files speeds things up, also tmpfs isn't always that fast, I run containers with tmpfs volumes for bazel builds with a network cache (so I don't need to persist files locally) and just cleaning up a 64GB tmpfs volume can take 20-30 seconds - if I had Optane drives at work I would not be surprised if they built faster over all.
Click to expand...
NVMe faster than RAM ? Weird. Also that should only be the Case if you are running out of RAM and are rebooting the Server often.
 
N

nexox

Well-Known Member
May 3, 2023
2,081
1,046
113
luckylinux said:
NVMe faster than RAM?
Click to expand...
RAM is faster, the tmpfs implementation just has some slow parts, plus you're still copying the files from NVMe to tmpfs before you can do anything and then at least some files back again when you're done.

luckylinux said:
Also that should only be the Case if you are running out of RAM and are rebooting the Server often.
Click to expand...
I don't know how much RAM you have but a kernel build will occasionally hit my cgroup limit of 128GB on its own if there are a lot of options enabled and I run one make job per.core, that doesn't leave a ton of memory to waste on long term storage of intermediate build artifacts (and at work it's all ephemeral instances anyway.)
 
L

luckylinux

Well-Known Member
Mar 18, 2012
1,757
585
113
nexox said:
plus you're still copying the files from NVMe to tmpfs before you can do anything and then at least some files back again when you're done.
Click to expand...
Fair Point :). Or to/from SATA for me. Just installing Kernel Headers on Linux takes forever for me :p.

I just installed Proxmox VE on the Hyve Servers I bought a while ago (on an NVMe Mirror out of 800GB / 960GB / 1.2TB / 3.2TB / 3.84TB, depending on what I had available) and now I'm basically out of NVMe, except 5-6 x consumer Crucial P5 Plus 2TB. The SAN Server will basically be only SAS 12G / SATA 6G plus maybe 3 x (6.4TB PCIe, though that is actually 2 x 3.2TB). No more NVMe available. Prices are currently crazy and Wallet is empty :(.

nexox said:
I don't know how much RAM you have
Click to expand...
It depends on the Host.

32GB on some Supermicro X10SLL-F/X10SLM-F Motherboards

64GB on Supermicro X11SSM-F/X11SSL-F / AMD Ryzen 3600/3700X Boards.

All the Way up to 512GB DDR4 on some AMD EPYC 7642 Systems (Hyve).

To stress test a CPU and check Thermals I typically just build the Linux Kernel, with default Options.

I never had a Problem building the Linux Kernel (6.12.23) on 16GB tmpfs on one of the smallest Systems.

nexox said:
but a kernel build will occasionally hit my cgroup limit of 128GB on its own if there are a lot of options enabled and I run one make job per.core, that doesn't leave a ton of memory to waste on long term storage of intermediate build artifacts (and at work it's all ephemeral instances anyway.)
Click to expand...
128GB RAM for a Kernel Build :oops: ? Holy smokes !
 
C

ca3y6

Well-Known Member
Apr 3, 2021
942
957
93
No they only had one in stock. The UI lets increment the quantity but it only gets enforced against stock when you try to add to the basket
 
You must log in or register to reply here.
Top Bottom