Search results

  1. E

    Help identify faulty hardware

    I had very similar issues with a H11DSi mobo and two 7f52 CPU’s purchased from tugm4470. However, after much trial and error I was able to resolve the issue. In a nut shell, I think it was a cooling problem. Disabling SMT was a quick fix that allowed me to improve cooling. I eventually got the...
  2. E

    Legit AMD Epyc or no?

    Thank you for your insights! So let me make sure I fully understand your comment. Let's say we use the H12DSi-NT6 Supermicro MOBO for this example. Since this MOBO was designed for Rome and a single upgrade step to Milan, Milan is end of the line and and there is less incentive for data centers...
  3. E

    Legit AMD Epyc or no?

    For the 7302 and 7f52 Epyc processors, which are great for distributed CFD computingthe new versions are 5-10 more than used equivalents as sold by tugm4470. I do sometimes get a little worried about used MOBO’s from any vendor in terms of security. I usually pay extra for new MOBO’s for...
  4. E

    What are your experiences with NEMIX RAM?

    Here is a quick update on this thread. I decided to use NEMIX for most of my project and tested dozens of 16GB sticks (~$32-35 per stick). I have not encountered any memory issues even while running very high workloads over multiple days and performance seems comparable to my Samsung RAM with...
  5. E

    Legit AMD Epyc or no?

    I bought around a dozen used Epyc Rome CPUs from tugm4470 on eBay. All came with this stamp, have worked very well and are equivalent as far as I can tell to CPUs purchased new from US vendors. If you clean your CPU with alcohol before applying thermal paste this stamp will come right off. I...
  6. E

    Anyone running Linux on Supermicro EPYC H12?

    I had spontaneous restart issues with popOS on a machine with a Supermicro H11DSi mobo and two high frequency 16 core Epyc 7f52 processors. A quick fix was to disable Simultaneous Multithreading (SMT) in the BIOS. Ultimately, the issue was resolved long term and I was able to activate SMT by...
  7. E

    RAM for EPYC Rome (4Rx4 supported?)

    I have tested dozens of sticks @Newegg:MEM-DR416L-HL01-ER32 16GB Memory Compatible With Supermicro by NEMIX RAM MEM-DR416L-HL01-ER32 16GB Memory Compatible With Supermicro by NEMIX RAM - Newegg.com Quality is better than expected given the cost ($35). I have not encountered any memory issues...
  8. E

    Mellanox SX6036 fan mod

    I am playing around with fan speed and temperatures (thank you for getting me here!) using the following workflow in the console interface: enable config terminal fae mlxi2c set-fan /FAN/FAN 1 <percent> fae mlxi2c set_fan /PS1/FAN 1 <percent> fae mlxi2c set_fan /PS2/FAN 1 <percent> The results...
  9. E

    Help with connecting serial port on Supermicro MOBO (H11DSi) to Mellanox switch (SX6036)

    Putty on Ubuntu is easy to install, and the default inputs worked without modification (dev/ttyS0, baud rate = 9600 etc.) so the tool showed me how to make this work. I can get minicom and screen to work now but Putty just made sense to me. I am playing with ltrace now and trying to understand...
  10. E

    RoCE v1 implementation (SX6036 heatsink/silence mod running log!)

    I got in. Here are the details.
  11. E

    Help with connecting serial port on Supermicro MOBO (H11DSi) to Mellanox switch (SX6036)

    Update: I made it into the switch! I had to add my username to the tty and dialout groups: sudo usermod -a -G tty <username> sudo user mod -a -G dial out <username> I also used putty which made things a bit easier to follow for noobs. We will see how far I get from this point.
  12. E

    RoCE v1 implementation (SX6036 heatsink/silence mod running log!)

    I just got my SX6036 and am having some trouble accessing the console through the serial port on my mobo. I am using a RJ45-DB9 cable I bought from eBay and am wondering if this cable is incompatible. Can you share the specs on the RJ45-USB cable you referenced above so I can give that a try?
  13. E

    Help with connecting serial port on Supermicro MOBO (H11DSi) to Mellanox switch (SX6036)

    I need some guidance on connecting a Mellanox SX6036 switch to the serial port of a Supermicro H11Dsi. My SX6036 is used and did not come with a RJ45-DB9 serial console cable. I bought this one on eBay. However, I have not been able to access the switch. OS: PopOS (Ubuntu 22.04) I tried the...
  14. E

    Epyc Rome 7F52 Upgrade from 7302P?

    …and at $300 the 7f52 has a very good performance over price ratio (so does the 7302 at $120).
  15. E

    Epyc Rome 7F52 Upgrade from 7302P?

    The actual performance improvement magnitude will depend on the specifics of your calculation but you can‘t go wrong with a higher clock rate and more cache as long as the higher TDP is worth it for you. If you do go with the higher TDP 7f52 make sure you have plenty of airflow across the board...
  16. E

    Epyc Rome 7F52 Upgrade from 7302P?

    My machine with a H11DSi mobo and two 7302’s runs idle at around 96W. My other machine with a H11DSi mobo and two 7f52‘s runs idle at around 114-135W. For my scientific calculations, which involve shared memory parallelization, I see around a 15-20% boost in performance with the 7f52 relative to...
  17. E

    AMD EPYC 7302p+ Supermicro H11SSL-i version 2

    If you are going with PC cases instead of server chassis to house the H11SSL-I you may want to consider the Fractal Meshify Lite case ($69 new on Amazon last time I checked) with a Noctua NH-U14S TR4-SP3 heatsink. I also upgraded the fans to Noctua-A14 PWN 40mm Premium fans to ensure adequate...
  18. E

    AMD EPYC 7302p+ Supermicro H11SSL-i version 2

    To minimize potential quality issues I would stick to RAM on the tested memory list located on the mobo website here. That said I have had a very good experience with 3200 Supermicro compatible RAM from NEMIX on the H11SSL-I, which runs around $35 for 16GB: MEM-DR416L-HL01-ER32 16GB Memory...
  19. E

    RoCE v1 implementation (SX6036 heatsink/silence mod running log!)

    Yes, I bet any integrated benefit peaks at 3-4 nodes after which a switch makes sense in terms of performance and cost. FYI, your post inspired me to get a SX6036. We’ll see if it is worth the effort for my use case. But cost is so low so it should be worth it for the learning experience.
  20. E

    RoCE v1 implementation (SX6036 heatsink/silence mod running log!)

    The no-switch approach for a small cluster was an idea that I got from @gb00s in this post. For the 3 node setup each machine had dual port ConnectX-3 cards connected via DAC’s. The ports on each card got the same IP but I used routes with a different subnet mask to guide traffic from each port...