Thanks for the post been eying this seller for a long while to replace a dual E5-2690 with DDR3 in a SC329. But was wondering how he did with Aus. Though id be goign for the 2RU HSF obviously.
AMD EPYC 7302p+ Supermicro H11SSL-i version 2
- Thread starter adamswebsiteaccount
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I had a great experience with TUGM5570 @ebay tugm4470 . He gave me a good discount on an EPYC 7453. I ended up purchasing it directly from him via Paypal. The processor was well packaged and the Fedex shipment arrived within a few days.
I ordered an Epyc 7543 from tugm4470 after reading all the positive feedback on these forums.
Not going to lie, I was concerned about a Chinese seller of a high ticket item, but this tugm4470 store really came through.
I tested the living daylights out of it on my desk for a week and a half (memtest86, memtest86+, mprime mixed), and it seems exactly as it should be.
I'm ready to call this thing stable, and leave my positive reviews on eBay.
I included the promotional code @gb00s provided in the message to seller, and they came through, Free Fedex piority shipping. Got from Shenzhen to New England (northeast U.S.) in 4 calendar days, which is pretty impressive.
Looks like tugm4470 is a major used parts broker and integrator, not just some guy, but the experience was good, and I'd definitely buy there again based on this experience.
Also, atechcomponents on eBay where I got the RAM was also stellar.
Both of them are quality sellers.
I opted to buy the motherboard (Supermicro H12SSL-NT in retail pack) new on Amazon, as I didn't want to risk there being any funny business flashed to the firmware.
So now my new server is stability tested and ready for the drop in upgrade in place of my 2x E5-2650 v2 with 256GB of RAM in a Supermicro SC846 chassis as follows:
- EPYC 7543 (32C/64T, base 2.8Ghz, boost 3.7Ghz)
- Supermicro H12SSL-NT
- 512GB (8x 64GB Registered ECC Hynix DDR4-3200)
I'll probably do it between Xmas and New Years when I have plenty of time to get it up and running when no one needs it.
Wish me luck! And thanks to @gb00s for reaching out and coordinating the code/promotion.
Not going to lie, I was concerned about a Chinese seller of a high ticket item, but this tugm4470 store really came through.
I tested the living daylights out of it on my desk for a week and a half (memtest86, memtest86+, mprime mixed), and it seems exactly as it should be.
I'm ready to call this thing stable, and leave my positive reviews on eBay.
I included the promotional code @gb00s provided in the message to seller, and they came through, Free Fedex piority shipping. Got from Shenzhen to New England (northeast U.S.) in 4 calendar days, which is pretty impressive.
Looks like tugm4470 is a major used parts broker and integrator, not just some guy, but the experience was good, and I'd definitely buy there again based on this experience.
Also, atechcomponents on eBay where I got the RAM was also stellar.
Both of them are quality sellers.
I opted to buy the motherboard (Supermicro H12SSL-NT in retail pack) new on Amazon, as I didn't want to risk there being any funny business flashed to the firmware.
So now my new server is stability tested and ready for the drop in upgrade in place of my 2x E5-2650 v2 with 256GB of RAM in a Supermicro SC846 chassis as follows:
- EPYC 7543 (32C/64T, base 2.8Ghz, boost 3.7Ghz)
- Supermicro H12SSL-NT
- 512GB (8x 64GB Registered ECC Hynix DDR4-3200)
I'll probably do it between Xmas and New Years when I have plenty of time to get it up and running when no one needs it.
Wish me luck! And thanks to @gb00s for reaching out and coordinating the code/promotion.
If I could ask, which specific memory did you get? My initial purchase only included 32GB of RAM (4x8), and now I'm feeling that 128GB (4x64 or 8x32) would be much nicer!
I got 8 of these, Hynix HMAA8GR7CJR4N-XN.
These are 64GB modules though. I'd imagine if you don't want 512GB you should probably get smaller modules, or you'll be giving up the added bandwidth of the 8 channel memory configuration.
Does anyone else have the -NT variant? (H12SSL-NT)
What kind of MB_10G_LAN Temps are you seeing in BMC (or reported in lm-sensors)? Mine are a little disconcertingly high, despite decent airflow in the case.
I suspect the heatsink back near the PCI slots is the one for the 10G Broadcom Ethernet. I installed a bunch of PCIe cards in the first few slots, and I think they might be disrupting airflow across the thing. I'm not even loading up the network, and it's reporting 72C-74C...
Still "green" in the BMC though. Not sure at what temp it turns "red".
Switched the fans from "Optimal Speed" to "Full Speed" and now it has dropped to 59C, but that still seems a bit high... (and now the server can also wake the dead...)
What kind of MB_10G_LAN Temps are you seeing in BMC (or reported in lm-sensors)? Mine are a little disconcertingly high, despite decent airflow in the case.
I suspect the heatsink back near the PCI slots is the one for the 10G Broadcom Ethernet. I installed a bunch of PCIe cards in the first few slots, and I think they might be disrupting airflow across the thing. I'm not even loading up the network, and it's reporting 72C-74C...
Still "green" in the BMC though. Not sure at what temp it turns "red".
Switched the fans from "Optimal Speed" to "Full Speed" and now it has dropped to 59C, but that still seems a bit high... (and now the server can also wake the dead...)
Last edited:
Room ambient 28C, inside the rack propably well over 30C. MB_10G_LAN 59C (idle). The MB_10G_LAN heatsink has a pretty direct airflow from nearby PCIe-slot-cooler. 60C for this component is nothing to worry about. BMC shows "High CT" -treshold of 100C.
A slightly different solution:
Evercool SB-F1 Fox-1 Slot Mounted Cooling Fan
SB-F2 Fox-2 Slot Mounted Cooling Fan
Thank you for that. So it's not just me.
I did some iperf tests, and heavy use of the networkinterfaces does not appear to impact temperatures at all. 74C at idle. 74C at load. For what its worth, it also does not appear to throttle. (Maybe it would if I used a lot of the fancy features, but for basic throughput it appears fine)
I suspect my three big 16x slot to four M2 cars are what are making mine a little worse.
The old Intel dual port 10gig adapter right next to them probably isn't helping either.
Problem is those big adapter cars only fit in those first three 16x slots. Because they are so long, there is "stuff" in the way if I put them in other slots.
I suppose I could move the intel NIC to the last slot, and put some sort of fan in there for direct sideways airflow, but I hadnt originally done this, as I was concerned about crowding my LSI 9305-24i, as it tends to get pretty hot under load as well.
I'll have to think about it.
Lol, I remember fans those from trying to keep my 3DFX Voodoo1 cool in the 90's
I think they work best in exhaust mode though, especially since in a server case there is strong front to rear flow, and that probably won't increase the airflow much over the Broadcom NIC heatsink.
All of that said, if the Broadcom NIC is happy up until 100C, maybe I don't have to do anything.
I'm currently operating in my 21.2C office. Once it goes back into the rack, ambient temps will go up a bit, and they will go up a bit more come summer, but they will not go up by more than 18.8C.
Fans are also operating at about 45% at these temps. I presume that once it gets hotter those fans will get faster as well, up to their 3000rpm max speed putting out over 100 CFM each. (Noctual industrial fans are awesome, if a little bit loud at full speed)
So maybe I am fine. It just looks a little disconcerting.
Last edited:
I was concerned about that too, since there is not much space between the m.2 adapters in this configuration (there was much more space in the old server board, as its PCIe slots were laid out with 16x and 8x slots in an alternating fashion always leaving a gap between them)What kind of temps do those M2 drives have?
In practice - however - I have not had to worry.
The little fan on the board, coupled with the server case airflow across the aluminum exterior does a surprisingly good job for its diminuative size, and low amount of space for airflow.
There are only a few mm between each of those cards, but it seems to be enough for the little fans to pull enough air over the aluminum heatsinks.
Gen3 m2's tend to sit at idle at about 26C-28C
Gen4 m2's tend to sit at idle at about 35C-41C depending on the model
The two m'2's directly on the motherboard, with those low profile heatsinks installed idle at about the same temp as the equivalent gen4 models in the adapter cards, and I gave those their own little 40mm Noctua fan to blow directly on them because I was worried they might get too hot. (it's visible on the left of the big PCIe cards in the pic.
For comparison, the two Intel Optane 900p u2 drives over on the right in that makeshift 2.5mm drive cage I taped to the side using 3m mounting tape idle at about 40C.
Most of these nvme drives (15 out of the 16) are used in ZFS pools, so to give them some load, I did a ZFS scrub and monitored their temps. Load temps seem to go up 5-6C over idle.
So yes, the concern about the airflow over the m.2 drives was real, and I was concerned as well, but testing has shown it to not be a problem. I even considered using some adhesive backed little rubber feet between them to force them apart just a tiny bit to make sure the air could get in, but looking at the numbers, I just don't have any reason to mess with them. They are doing their job.
Those Asus Hyper m.2 Gen4 cards may be consumer products, but as I understand they were based on enterprise designs. I was going to buy the famous Dell cards instead that are actually enterprise products, but when I was shopping for them years ago I couldn't find any in stock anywhere, and I was working against a deadline, so the Asus ones had to do. They are surprisingly well made for consumer products, and rely on PCIe bifurcation just like an enterprise model would, so the signal path is relatively simple. They are also fully powered by the slot, so no need for power cables.
I have been very happy with them, and wouldn't hesitate to recommend them even to enterprise users. Just have to keep an eye on those little fans. No idea how long those bearings are rated for.
Last edited:
Another question, if anyone knows.
I'm a little bit confused regarding the network interfaces on this board.
Using ip a and removing address details I have the following:
(bear with me here, after decades of using ifconfig, for some reason this distribution no longer includes it, and I am not quite comfortable with the "ip" command yet.)
So, enonp0 and enonp1 (number 2 and 3) are the on board broadcom 10gig ethernet devices.
enp68s0f0 and enp68s0f1 (numbers 4 and 6) are the 10gig SFP+ ports on my discrete Intel NIC.
There is always a local device, so no problem with #1.
But what on earth is #5? enxbe3af2b6059f?
The MAC address (be:3a:f2:b6:05:9f) appears to be in the locally administered range, so it is of no help. I did an lspci -vvv, but this device name does not show up in the connected PCIe devices.
lsusb finds this which might be related:
Is this the BMC interface somehow shining through to the operating system? If so, why would it do that?
Some googling regarding RNDIS suggests that it might be inherently insecure. I am considering just blacklisting the USB driver module that loads it. Any treason I shouldn't do that?
Edit:
I found this from Supermicro:
Apparently it is used by the BMC to perform in-band firmware updates.
Still, it is somewhat disturbing to have mystery network interfaces on your server, especially when there are suggestions that NDIS is inherently insecure.
I'm a little bit confused regarding the network interfaces on this board.
Using ip a and removing address details I have the following:
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
2: eno1np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
altname enp73s0f0np0
3: eno2np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
altname enp73s0f1np1
4: enp68s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
5: enxbe3af2b6059f: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
6: enp68s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000So, enonp0 and enonp1 (number 2 and 3) are the on board broadcom 10gig ethernet devices.
enp68s0f0 and enp68s0f1 (numbers 4 and 6) are the 10gig SFP+ ports on my discrete Intel NIC.
There is always a local device, so no problem with #1.
But what on earth is #5? enxbe3af2b6059f?
The MAC address (be:3a:f2:b6:05:9f) appears to be in the locally administered range, so it is of no help. I did an lspci -vvv, but this device name does not show up in the connected PCIe devices.
lsusb finds this which might be related:
Code:
Bus 005 Device 004: ID 0b1f:03ee Insyde Software Corp. RNDIS/Ethernet GadgetSome googling regarding RNDIS suggests that it might be inherently insecure. I am considering just blacklisting the USB driver module that loads it. Any treason I shouldn't do that?
Edit:
I found this from Supermicro:
Apparently it is used by the BMC to perform in-band firmware updates.
Still, it is somewhat disturbing to have mystery network interfaces on your server, especially when there are suggestions that NDIS is inherently insecure.
The Broadcom 10G Ethernet runs about 74C no matter how much load is on the network even in a large, well ventilated chassis. The temperatures are the same across all Supermicro H12 boards.
Thank you. Appreciate the "this is normal" confirmation.
I bought the H11SSL-NC, and I am trying to plug my Power Switch, Power LED+, Power LED-, Reset Switch, and HDD Led.
According to the manual, using the JF1 headers is the way to go (Page 38).
However, this is my first time making a computer. All wire pairs have a white (+) (neutral/ground) and black (-) (power to).
This is my current plan:
Is this correct?
According to the manual, using the JF1 headers is the way to go (Page 38).
However, this is my first time making a computer. All wire pairs have a white (+) (neutral/ground) and black (-) (power to).
This is my current plan:
Is this correct?
The polarity matters for the LEDs, but not for the power and reset switch.
If you get the polarity wrong for the LEDs, they won't light up. Just reverse them if that happens.
In my experience the HDD LED does not light up when the NVME M2 drive is active, just the SATA drives.
If you get the polarity wrong for the LEDs, they won't light up. Just reverse them if that happens.
In my experience the HDD LED does not light up when the NVME M2 drive is active, just the SATA drives.