Search results

Only 4x NVMe per node isn't too bad, but don't underestimate how much CPU the NVMe OSDs will use. Another option if you need more capacity is to use spinners for OSDs, with DB/WAL on a partition of NVMe.

If you really want 4TB RAM + GPU, X11DPU in 829U can do it pretty affordably. 24x DIMM for up to 6TB. If you want to stick with DDR3, you'd have to scale out to multiple nodes (which is probably a good thing for your application in the long term anyway).

SATA PWDIS strikes again!

OK 4TB of RAM is an unusual need; that certainly narrows the field. X11DP can do that, but the DDR4 cost will start to become noticeable. What CPU/compute needs? Cores over clocks? AVX512?

How about we approach this by asking what your needs are and what your budget is? Do you need the PCIe lanes and/or RAM capacity of 2011-0? Is the use case a CAD desktop workstation, or a ML batch server, or Plex transcoding, etc.?

VGA for compatibility with KVM console servers, e.g., Avocent. Sometimes IPMI arrives set to static IP, and it's easier to connect a monitor to flip it to DHCP than to sniff the network. But this is all just for console work; we're not gaming on the BMC graphics, we don't need high resolution.

I know we're all about the overkill here at STH, but my little opnsense box for gigabit fiber is just a $35 m73 tiny (Haswell) plus a $10 mPCIe RTL8111 NIC. Runs a bit hot but works just fine including wireguard, haproxy, and DNSBL; no suricata though. I try to design my network to minimize...

I echo the sentiment that there is just not enough information yet to know if your bottleneck is PCIe bandwidth to the GPUs, or raw compute for the games, or NUMA/cache issues with the two CCDs in the 5950X, or even I/O wait with the other workloads collocated on the system.

Yes, but even with all the 10GbaseT NICs disabled it still isn't exactly low power compared with more modern systems.

Another option is a $20 passive 4x8643 card, no retimer so you need to watch the signal loss. Whether to use 8643 vs 8654 SlimSAS vs 8642 OCuLink also depends on cabling, which might not be cheap. And whether you're using a backplane.

Yes, the H200 (OEM LSI 2008) will work when flashed to IT. The PERC 6/IR (OEM LSI 1068) is ancient and SAS1-only; I wouldn't bother with them.

Are you using the H710 for hardware RAID? Pop the drives onto an HBA (or a RAID card flashed to IT) to sg_format to 512 or 4k, then move the drives back to the R720, and the H710 should see them. If using the HBA doesn't show the block devs under the OS, try the SCSI generic devices (/dev/sg*...

Glad you got it sorted!

I'm happy with my Tyan S8030 from tugm4470 (H11SSL and H12SSL are good, too) in an 826 with 00053 rails.

Chassis sounds like an 826, which would fit any boards of normal form-factor (ATX, SSI-EEB, EE-ATX). If you don't care about ECC and IPMI, consumer 1151-2 boards are dirt cheap nowadays. Or consumer AM4, e.g., B550.

For such a small cluster, stick with a single NAS and have the compute nodes mount over NFS or iSCSI. Your life will be happier. Ceph is great, but overkill for this use case, very complex, and you will not get the performance you might be expecting. Gluster is dying out. Longhorn has come a...

sg3-utils

sg_format -Fs 4096?

Is the context of the poll limited to PVE? Would rook/k8s not be an option?

I'm not quite clear on your anticipated workload, but what you describe isn't particularly heavy. An older X11SCH-F (C246, 1151-2) may suffice: x16, x8, 2280, 22110. I'm not sure why you'd need to be transcoding AV1, and QSV can easily handle low-res transcodes. The total cost of the whole...