Dual Xeon-D

berny81

New Member
Dec 30, 2020
2
0
1
hi all, i have buyed one of this board. i'm finding some documentation of this board, but it look like a top secret board. Someone have understand the motherboard jumper? i want to know if it is possible to power on/off node individually. it is possible to power off a node by OS, but after that the only way to power on is disconnecting and reconnecting power, but if the other node is running this is not a good thing
 

bob_dvb

Active Member
Sep 7, 2018
147
67
28
Not quite London
www.orbit.me.uk
hi all, i have buyed one of this board. i'm finding some documentation of this board, but it look like a top secret board. Someone have understand the motherboard jumper? i want to know if it is possible to power on/off node individually. it is possible to power off a node by OS, but after that the only way to power on is disconnecting and reconnecting power, but if the other node is running this is not a good thing
Hi Berny,

I asked Akamai if I could have access to any documentation on this motherboard and despite the fact that my company spends a huge amount of money with them and I have an NDA with them, they said they couldn't share anything about it or even acknowledge it. I never found a way to manage the power of each node separately, but I was looking at the LPC Port 80 connector and thinking someone could build something to manage the power states?

The Intel LPC specification includes LPCPD which signals power states to devices, so that could be an answer.
 

berny81

New Member
Dec 30, 2020
2
0
1
Hi Berny,

I asked Akamai if I could have access to any documentation on this motherboard and despite the fact that my company spends a huge amount of money with them and I have an NDA with them, they said they couldn't share anything about it or even acknowledge it. I never found a way to manage the power of each node separately, but I was looking at the LPC Port 80 connector and thinking someone could build something to manage the power states?

The Intel LPC specification includes LPCPD which signals power states to devices, so that could be an answer.
thanks bob_dvb, using LPC is a good idea. In a old asus motherboard i have a diagnostic board with 7segment display and power and reset button, it fit in tpm connector. Try to find where is and try to adapt the pinout to make it working with this board
 

craig5571

Member
May 31, 2020
60
6
8
now this is interesting... and only 40$ plus shipping. m.2 vga card...
thats the one thing this board doesn't have..

Linus tech tips talks about the product..

Where I ordered it..

A pdf about it...

The marketing bs from Asrock..

It looks to be a great fit for this board...
 

int0x2e

Member
Dec 9, 2015
88
61
18
41
now this is interesting... and only 40$ plus shipping. m.2 vga card...
thats the one thing this board doesn't have..

Linus tech tips talks about the product..

Where I ordered it..

A pdf about it...

The marketing bs from Asrock..

It looks to be a great fit for this board...
I do hope I'm wrong, but I believe you ordered the BGA chip at the heart of the Asrock m.2 vga board - you can't just plug it into an m.2 connector...
 
  • Like
Reactions: craig5571

craig5571

Member
May 31, 2020
60
6
8
I do hope I'm wrong, but I believe you ordered the BGA chip at the heart of the Asrock m.2 vga board - you can't just plug it into an m.2 connector...
yes that would suck... I'll dig in and find out.. yep I screwed up.. lol..

although I did find this



who knows if it even exists...
 
Last edited:

melozo

New Member
Jan 15, 2021
1
0
1
i did a scripted install of esxi, its very easy.. i just modified the ks.cfg file i used this for a reference


with the scripted install, you just insert the usb, and wait.. it works automagically..

thank you so much for the help..

I do have another question , would one of these nodes make a good TRUEnas 12 NAS?
the horsepower is there , just wondering if there was any gotchas?

and is anyone utilizing the m.2 slot in addition to a SAS OCP card ... i have an asrock rack m3008x8 sas card and it installs just above the m.2 slot. my only concern with using both of them is the heat from the SAS card.. but with proper cooling it should be ok i think..

and i actually use a box fan on low setting on top of the case.. this makes the case ultra quiet and provides great cooling..
Script installation prompts the error, please advise.
<6>UEFI Secure Boot is not enabled
<6>Shutting down firmware services...
 

Attachments

Zalouma

New Member
Aug 5, 2020
29
18
3
I was not able to get anywhere with this one, Found something cool if someone want to give it a try without the need to install a M.2 GPU

There is a web console that can be installed on raspberry pi and connect both USB-com to the raspberry and you have a remote console as well as the option to remote power on/off and reset each individually


I am going to give this a last shot, if I got nowhere would be selling mine if someone would be interested to buy it... It just this project need alot of time and I dont have it

I would be selling it for $500 (paid around $690 with shipping from Canada to US), I added a link to the listing I purchased from with pictures... Hit me up if you are interested
 

koyetsu

New Member
Feb 28, 2021
12
7
3
Has anyone been able to get ESXi on this? i keep getting the tpm error and when i turned it off on node 0 the option for it disappeared from bios and i can't re-enable it, and that didn't fix the error
 

Zalouma

New Member
Aug 5, 2020
29
18
3
Selling mine for 500 OBO excluding shipping (Shipping with FeDEX ground)

This is where i bought it and how it looks

Let me know if you are interested
 

fake-name

Active Member
Feb 28, 2017
165
124
43
70
So my build with 2 boards is done:

_MG_7542_s.jpg

It works fine, though the power supply situation is kind of ridiculous:

_MG_7536_s.jpg

That's 2 Supermicro PWS-351-1H power supplies shucked and stacked, with the fans moved into a 3D printed bracket. I had to shuck the power supplies to get them to fit vertically and for airflow reasons.

There's also switches to power-off each PSU, as well as remote relays to power cycle the nodes electronically.

Finally, there's a raspberry pi running ConsolePi to control the relays and provide access to the 4 USB TTYs, though due to an outstanding issue I mostly just use picocom for the actual serial ports.

Also yes, that is a random 5V power brick floating zip-tied to the various power cables. It's running the Pi, and I got lazy so it's connected by wires literally just soldered to the AC plug tines. I did at least heatshrink the AC connections so it's fairly finger safe.

Not pictured is I also replaced some of the power supply molex connector cables on both power supplies with SATA power cable strings scavenged from a dead power supply, so I can run SATA drives. That gives me the ability to put decent drives in the box without the concern that is molex to sata adapters.

I generally save power supplies that would otherwise be tossed, and at minimum scavenge the wires and connectors from them, because you can usually reuse those in projects if you're OK with doing some hackery. I have a number of power supplies where I've replaced unused power cable strings with more useful ones (like removing all molex connectors in favor of sata, etc...)

IMG_4688_s.jpgIMG_4684_s.jpg

Anyways, all 4 nodes are now happily part of my proxmox cluster:

1616998508432.png

My only remaining issue is I need more RAM for the upper node pair, and DDR4 ram prices have just gone stupid in the last few months. I paid $300 for 128GB DDR4-2133 ECC a year ago, and now it's nearly twice that.

Anyone know a decent place to buy ECC DDR4?
 

Attachments

Last edited:

fake-name

Active Member
Feb 28, 2017
165
124
43
70
So some fun drama with these.

Of the 4 nodes I'm running, I had 3 SSD variants. 1 SK Hynix Gold P31, one Samsung SSD 970 evo, and 2 Kingston A2000.

Both of the nodes running the Kinsgton A2000 SSDs would fall over under any IO loading. The other two nodes survived fine. I could repeatedly cause the drive to crap itself within 30 seconds with just `stress-ng`.

Anyways, the two A2000 drives were in the same motherboard, so I either had a bad mobo, or a bad interaction with the SSDs. I took a gamble, and bought another of the SK Hynix drives, copied the drive contents across, and when placing the new drive in the previously failing node, I couldn't get it to fail anymore, so I've replaced both of the two kingston drives.

I thought it was an interesting failure mode.

I'm now looking at adding some hot-swap SATA drive bays so I can add additional storage.
 
  • Like
Reactions: Zalouma and itronin

n17ikh

Member
Jul 12, 2019
53
53
18
So some fun drama with these.

Of the 4 nodes I'm running, I had 3 SSD variants. 1 SK Hynix Gold P31, one Samsung SSD 970 evo, and 2 Kingston A2000.

Both of the nodes running the Kinsgton A2000 SSDs would fall over under any IO loading. The other two nodes survived fine. I could repeatedly cause the drive to crap itself within 30 seconds with just `stress-ng`.

Anyways, the two A2000 drives were in the same motherboard, so I either had a bad mobo, or a bad interaction with the SSDs. I took a gamble, and bought another of the SK Hynix drives, copied the drive contents across, and when placing the new drive in the previously failing node, I couldn't get it to fail anymore, so I've replaced both of the two kingston drives.

I thought it was an interesting failure mode.

I'm now looking at adding some hot-swap SATA drive bays so I can add additional storage.
The A2000 has an issue with PCIe power saving.


Not sure if it's the issue you're seeing or not. If you're on Linux, try the kernel parameter nvme_core.default_ps_max_latency_us=0