And so it begins... First AMD Ryzen AM4 server motherboard.

[EffrafaxOfWug]

In doing some more research, I put ASRock's MAC util through strings and it's become apparent that it's just issuing raw commands to the IPMI using the bundled asrripmi binary. Through some trial and error (I'm new to the whole reverse engineering thing) I was able to identify the commands sent and I can verify they work in regular ipmitool.

# get current bonding status
ipmitool raw 0x32 0x72 0x01 0x00 0x00
# don't know what all the numbers mean, but:
# bonding on returns 01 00 01 00 00 00 01
# bonding off returns  00 00 01 00 00 00 01

In my original situation, the problem was that the MAC addrs for eth0 (dedicated IPMI NIC) and eth1 (combo ethernet shared with mobo NIC via NCSI) were the same. These can also be read and set as follows:

# get eth0
ipmitool raw 0x3a 0xa1 0x00
# set eth0 MAC to de:ad:be:ef:ba:be
ipmitool raw 0x3a 0xa0 0x00 0xde 0xad 0xbe 0xef 0xba 0xbe

# get eth1
ipmitool raw 0x3a 0xa1 0x01
# set eth1 MAC to be:ef:f0:0d:ca:fe
ipmitool raw 0x3a 0xa0 0x01 0xbe 0xef 0xf0 0x0d 0xca 0xfe

Even better, ASRR posted a FAQ entry listing even more of their raw commands for fan control, most interestingly the ability to change the fan curves (temps vs. duty cycle) without needing to boot in to the BIOS. Their explanation lacks finesse so here goes:

# get fan duty cycles - returns a list of 12 duties (0x00 = 0%, 0x64 = 100%)
ipmitool raw 0x3a 0x06 0x01 0x00
 00 2d 32 37 3c 41 46 4a 4e 52 56 5a
# get fan temp tables - returns  a list of 12 temperatures (degrees C in hex values)
ipmitool raw 0x3a 0x06 0x02 0x00

# set a new duty cycle table with 11 values in it - looks like the raws accept decimal values so that makes it easy
ipmitool raw 0x3a 0x05 0x01 0x00 30 55 60 65 70 75 80 85 90 95 100

# set a new temp table, again 11 values needed and decimal works
ipmitool raw 0x3a 0x05 0x02 0x00 45 50 55 60 65 70 74 78 82 86 90

# from previous findings, get current settings for all 6 fans
ipmitool raw 0x3a 0x02

# set all 6 fans to various thresholds (last two values always 0x00)
	0x00 - use smart fan tables vs. duty cycle values
	0x04 through 0x64 - manually set duty cycle 0-100%
# the following sets FAN1 to smart fan, FAN2 and FAN3 to 50%, FAN4 to 100%, 5 and 6 on smart fan
ipmitool raw 0x3a 0x01 0x00 0x32 0x32 0x64 0x00 0x00 0x00 0x00

The FAQ entry suggests that the IPMI settings page will at some point contain a GUI for this but it's not present on mine at least.

I suspect these raw values will hold true for all of their current IPMI implementations. I've tried looking through the IPMI image to see if there's any sort of a list or dictionary of IPMI raw values within it but sadly didn't find anything yet. If anyone knows of a method to see what the calls from the web UI actually do under the hood I'd love to hear about it.

 

[soupyfrood]

Anyone want to speculate if this board will get Zen3 support in the BIOS?

 

[Egbert]

I suspect these raw values will hold true for all of their current IPMI implementations. I've tried looking through the IPMI image to see if there's any sort of a list or dictionary of IPMI raw values within it but sadly didn't find anything yet. If anyone knows of a method to see what the calls from the web UI actually do under the hood I'd love to hear about it.

I did some reverse engineering of the web UI a few months back to automate switching of the SSL certificate. The web UI basically makes REST web service calls to the IPMI interface. Unfortunately, the calls go against an ASRock-specific variant of the Redfish API schema, so I was only able to figure out what each function in the web UI does by using Fiddler to capture the HTTP traffic.

 

[EffrafaxOfWug]

Anyone want to speculate if this board will get Zen3 support in the BIOS?

Probably not - grapevine says that only X470 boards with 32MB ROMs will be able to run the 5000 series (presumably due to needing space for all the different sets of microcode) and this board only uses a 16MB chip.

If so, unless ASRR introduce a 5000-series-only BIOS variant or offer replacement BIOS chips I think this'll be stuck with the 3000s only.

 

[llowrey]

Probably not - grapevine says that only X470 boards with 32MB ROMs will be able to run the 5000 series (presumably due to needing space for all the different sets of microcode) and this board only uses a 16MB chip.

If so, unless ASRR introduce a 5000-series-only BIOS variant or offer replacement BIOS chips I think this'll be stuck with the 3000s only.

That's too bad. It looks like all 3 X570 boards have 32MB chips so now I suppose we can speculate about 5000-series support for those. I have a X570D4U-2L2T on order from Provantage but I think I'd rather speculate about when the board will get 5000-series support rather than about when I'll actually receive the board.

 

[Deleted member 30475]

Hello, after spending fair amount of time with the X470D4U i have come to realize that there is a UEFI firmware bug, specifically related to manual fan control and applies to both the 3.30 release, as well as v3.37 beta version of UEFI firmware. Here are my findings so far:
FAN1 in UEFI controls the header #1 on the board (as it should) *
FAN2 in UEFI controls the header #3 on the board (instead of the header 2) **
FAN3 in UEFI controls the header #4 on the board (instead of the header 3)
FAN4 in UEFI controls the header #5 on the board (instead of the header 4)
FAN5 in UEFI controls the header #6 on the board (instead of the header 5)
FAN6 in UEFI does not control any.... possibly wrecking something else.

The above pattern looks like a typical ‘copy & paste’ (a classic), and it is entirely possible, or at a minimum should not be ruled out that a variable FAN6 is written to the wrong (adjacent) address space, potentially causing corruption of data in that address space. The consequence of this can be anything (Example, if the affected address space contains data used by network stack, like table of MAC addresses for instance, writing to this space will disrupt the operation of the network stack).

--------------
* FAN1 (CPU) is the only one that is properly controlled.
** I believe that the rotation speeds of the fans connected to headers # 1,3, 4, 5, and 6 accordingly are reported correctly in the UEFI. A fan, connected to the header # 2 on the board with the control switched to ‘manual’ cannot be controlled, however runs at a constant speed.



The steps I used to reproduce / demonstrate the bug:

1. With all six fans connected, change FAN control mode in the UEFI of the board to Manual mode, and set all fans to “Full On”. Save changes, exit and reset.

2. For each fan (one at a time):

2.1 Access UEFI, reduce the speed of any one fan to “Level 1”. Save changes, and exit.
2.2 Access UEFI. Observe the impact of the changes you made on the speed of the target fan as well as all other fans. Restore the target fan to “Full On”. Save changes and exit.
2.3 Repeat the steps in section 2.2 for each fan one at a time.


--
I have been sitting on this for a couple of weeks now, and have not yet reached out to ASRock for comment.
At this point I'm interested to hear some feedback from you guys on this finding


Cheers,

 

[EffrafaxOfWug]

Interesting, I'll have to confirm with my board at some point; IIRC I'm only using fans 1, 2 and 3 but I'm pretty sure I've been able to modify the duty cycles correctly via IPMI and see the corresponding rpm changes from the PWM tach.

However, I've been modifying direct through the IPMI raws rather than booting in to the BIOS so as you sat perhaps it's a bug in the UEFI -> IPMI fan mappings rather than a bug in the IPMI itself. I assume you're running the 1.90 BMC firmware?

 

[Deleted member 30475]

Yes, the BMC is v1.90. I haven't tried raw IPMI commands - only BIOS or IPMI server management tool v2.4 by Asrock. The board rev 1.01 came from the latest batch it appears (amazon was accepting pre-orders on boards it did not yet have in it's warehouses. Took about 10 days to arrive).

It looks like the fan issue was already reported in October of 2019 here:

...
Fan connector FAN2 only works correctly in smart mode (ie temperature following mode). When set to a fixed speed the fan will only spin at the minimum. The same fan connected to either FAN3 or FAN4 works as expected when set to a fixed speed (eg full on). Perhaps FAN2 is intended for coolers with dual fans. I did not test FAN5 or FAN6.

So... i finally pulled a trigger yesterday and submitted a bug report on this fan issue to a reresentative @asrockamerica.com. Let's see what comes back.

With regards to IPMI ... I have counted a total of 4 MAC addresses on 3 physical interfaces. Only 3 of the MACs found on the factory labels, but the 4th is not. In addition, the MAC on the physical interface and the label on that interface do not match. Smoke & mirrors is how I would describe it. At his moment I'm just trying to figure out what questions to ask when i contact AsRock for support.

 

[EffrafaxOfWug]

I've not studied the labels much, but OotB my board (also labelled as HW rev 1.01 BTW despite being bought over a year ago) came with two different MAC addresses on labels. The two i210 NICs both had sequential MACs, and the IPMI NIC had the same MAC as the NCSI i210 NIC, so that got reset - as such it now doesn't match the label printed on it. But I could use the IPMI raws to set that MAC address if I wanted to).

Are you sure the label definitely corresponds to a MAC address? I have two labels on top of my i210 ports, one of which is the MAC address and the other a serial number of some sort (it's not valid hex so it can't be a MAC).

I think it'll be worth experimenting with the IPMI raws for fan control if convenient but probably best to deal with ASRR support first. FWIW I've had good results from the Dutch guys.

 

[Deleted member 30475]

I have not encounter myself but do remember reading some people discussed interfaces on their board loosing MAC addresses. Are there any other reasons to use IPMI raws besides changing the MACs and Fan duty cycles? For example, I would be interested in reducing the TCP Windows size on the BMC, if at all possible.

Onto the labels: There are three MAC labels on my board are white yellow and green, each with one MAC. There is another white label as you said – but it has some irrelevant serial number.
My problem was that I took the labels too seriously when used them as a reference. Missing MAC label on the second i210 interface and the bond which is enabled by default only added to the confusion. It took some time to sniff which MAC correspond to which interface, but i think i finally have figured it out. As a result of all the time spent with the board I may be submitting a couple more minor bug reports to ASRock it looks like.

Also the good news came from William and the BIOS team at ASRock Rack. The Fan controls were fixed in the new beta firmware L3.39, and I have got the download link from ASRR support. Not sure whether it is appropriate to post the firmware on this forum - the new beta firmware was not posted on the X470D4U home page). I can share the new beta if there is an interest.

 

[EffrafaxOfWug]

Are they still using the FTP site at ftp.asrockrack.com/x470d4u/ for the beta downloads or did they provide you with a regular link that didn't need any creds to access...? I've seen a couple of references here and there regarding the L3.39A beta so wondered if other links or leaks had gone out, or if someone had divulged the new credentials...

As for the TCP window size, think it'd be a matter of whether there's an ability to actually change it from within the IPMI commands themselves. I've not seen such an option in the firmware itself so chances are it's hard-coded and not user-editable directly.

 

[Deleted member 30475]

I don't have credentials to ftp, just a link to dropbox with X470D4U L3.39 beta , and instruction to use instant flash.
Fan controls are fixed (verified) in v3.39, that's all i know so far. Also found BMC Beta 2.05 which was leaked on Asrock forums three weeks ago. From what i gather, AsRock Rack lacks a clear release process.

 

[EffrafaxOfWug]

FWIW it looks like BIOS and IPMI updates have just landed for the X470D4U and friends; BIOS is at v3.50 and includes ComboPI 1.0.0.6, IPMI at 02.20.00 and supposedly includes a fix for GPU temp sensor. I'll likely update my BMC some time in the next day or so assuming I can find my brave pills.

Corresponding updates for the X470D4U2-2T are v3.40 and 01.80.00 respectively, although the IPMI changelog there says there's a fix for "...BMC error caused by holding down UID for 4 seconds".

 

[DavidB]

One thing when updating the BMC: if you have previously disabled bonding the update re-enables this even with "keep network configuration" checked and you will lose access on any custom IP to the BMC.

 

[EffrafaxOfWug]

One thing when updating the BMC: if you have previously disabled bonding the update re-enables this even with "keep network configuration" checked and you will lose access on any custom IP to the BMC.

Ha! My brave pills came earlier than expected (before your post in fact) so I've just spent the last half hour restoring the network config; I've had bonding disabled since day 1 so this required a re-wire to a different switch and VLAN. The admin user was also reset back to the default password so I think it must have just wiped the entire config rather than just resetting the networking config.

Not terribly encouraging for ASRR's QA department...

 

[masimus]

My lab setup has been stable for quite some time (BIOS 3.37 and BMC L2.05). But for some reason I jumped on the upgrade train (BIOS 3.50 and BMC P2.20)...

Setup still seems to be stable, so no worries there. But I see a jump up in idle power by ~12-15W (measured from UPS, not very granular but consistent). Of course still same HW as before (3950X + 4x32GB Samsung ECC DIMMs + 2 SATA SSDs + 2 NVMe SSDs). From what I can see, BIOS settings are identical as before as well. Can anyone else confirm/deny this behavior?

 

[masimus]

Downgraded to BIOS 3.37 (kept BMC P2.20). And idle power dropped back again... Maybe some settings has changed default values?

 

[i386]

I would try the newest bios version, reset to "optimal/default settings" and see if that changes something

 

[EffrafaxOfWug]

I've not tested the updated BIOS yet (after my experience with the BMC I figured I'd hold off) and don't have a power meter that'll work on kettle leads but might be worth collecting some before and after temperature/voltage readings to see if there's more juice going to the CPU somehow. 15W is a pretty substantial rise during idle considering you're not running a lot of extra peripherals.

What version of AGESA does 3.37 come with?

 

[masimus]

>> What version of AGESA does 3.37 come with?

3.37 is the version has been available on the official download site (under "Beta Zone") for some time. Descriptions says "Update AGESA PI to 1.0.0.6". Does that differ from the "AGESA Combo-AM4_1.0.0.6" for 3.50? I am not well versed in AMD nomenclature. Previously only using Intel for a loooong time .