Recent content by pipe2null

  1. P

    M.2 NVMe's on Proxmox: pool vs array vs insanity?!?

    1. NVMe's, plain ol' ok-ish consumer grade M.2 SSDs: Western Digital Blue: SN570 500GB Western Digital Black: SN750 500GB Samsung NVMe... Came with my Lenovo laptop, not positive exact model, but looks like a PM981a? inxi model number "MZVLB512HBJQ-000L7" 2. Testing in other systems is not...
  2. P

    M.2 NVMe's on Proxmox: pool vs array vs insanity?!?

    NOTE: I solved Issue #2 "poor perf from inside VM". Now VM io perf is on par with host side perf. CrystalDisk running inside VM is getting 8.6GB/s read and 6.8GB/s write when qcow2 drive file is hosted on mdadm raid0 array of 3x NVMe's. Updated OP. Poor perf with Zfs on nvme's still an issue.
  3. P

    M.2 NVMe's on Proxmox: pool vs array vs insanity?!?

    Supermicro X10DRU-i+ NVMe's on Asus 4x slot M.2 NVMe adapter board - No bridge, but mobo supports PCIe-x16 bifurcated to 4x PCIe x4's
  4. P

    M.2 NVMe's on Proxmox: pool vs array vs insanity?!?

    No difference when I reran some of the perf tests with sync=disabled on zfs datasets. I just tried another approach with a zpool with only a single NVME drive. Creating a zpool with exactly one of my NVMe drives has roughly the same perf as a zpool with all 3 NVMe drives?!? So, here is the...
  5. P

    M.2 NVMe's on Proxmox: pool vs array vs insanity?!?

    The fio perf tests I was running are "supposed" to be running async io (fio --ioengine=libaio)... But I'll rerun a short test with sync disabled on the pool and see if the numbers are any different. Thanks for the suggestion!
  6. P

    M.2 NVMe's on Proxmox: pool vs array vs insanity?!?

    I did a PCIe passthough to a windows VM and checked the crystaldisk for a single nvme, and those numbers roughly matched fio perf numbers obtained on the host, and somewhat close to what people have posted within amazon reviews of that specific nvme drive. In that configuration I did not see...
  7. P

    M.2 NVMe's on Proxmox: pool vs array vs insanity?!?

    I've been wrestling with insanity on this one. I'm trying to find the optimum configuration for raw performance for 3x M.2 NVMe SSDs on my Proxmox server to be used for VM images and containers. For the life of me, I cannot understand the performance numbers I'm getting for various zfs and...
  8. P

    NVMe - make the most of your PCI-e Slots (How-to config Supermicro boards for AOC-SLG3-2E4T et al)

    As an afterthought, if your drive setup is supposed to be hot swappable (for an NVMe), you might try booting windows and THEN inserting the drive to backplane to see if windows detects it and/or it is working correctly. If windows can use it just fine when inserted after system boot, it might...
  9. P

    NVMe - make the most of your PCI-e Slots (How-to config Supermicro boards for AOC-SLG3-2E4T et al)

    My bios suggestion was just a shot in the dark, especially if you're booting your system off of a satadom. But if bios is not handling the drive correctly or only partially handling it instead of just ignoring it, I suppose it could cause problems handing the device off to the booting OS...
  10. P

    NVMe - make the most of your PCI-e Slots (How-to config Supermicro boards for AOC-SLG3-2E4T et al)

    I have successfully booted various M.2 NVMe's on X10DRU-i+ motherboards, which is not supported with stock bios. These booting drives are NOT on any compatibility list I am aware of, but maybe they are?: Samsung 970Evo 2TB, Western Digital WD Black SN750 500GB, Western Digital SN730 256GB. I...
  11. P

    Is there any practical advantage to installing a connectx3 card on a router?

    I have a supermicro X10SLH with 6x10G RJ45s currently running pfsense as my home/lab router. Soon I will be reinstalling router with OPNsense, so it is a good time to go ahead and populate the router's single PCIe slot with additional interface(s). At the same time, I am also finally adding a...
  12. P

    What's your favorite ONIE-installable NOS?

    Feel free to skip the long story and jump to the "What's your favorite ONIE-installable Network Operating System?" part. A little while back I decided to take the plunge and start learning the enterprise side of servers and networking, so I picked up some hardware from ebay that was cheapish...
  13. P

    Best Practices? for PCIE slot selection to optimize perf with dual/multi proc motherboards...

    I have been kicking around this question in my head for a while, but I don't know enough of the low level details to know one way or another and I have limited hardware at the moment so ability to experiment is also limited. So, I thought I'd see what more knowledgeable people think: The...