Weird eBay find: any clue what a Supermicro X10SDV-IBQ-AM04 is?

[Scott Laird]

I was looking for something else, and ran across this listing on eBay for a Supermicro X10SDC-IBQ-AM04, and it's weird enough that I decided to ask if anyone here had any idea what it is. It's presumably a custom board for someone that SM made, with a Xeon D-1567, but the form factor and a bunch of small details are just confusing me. It's shaped roughly like the MB for a 4-nodes-in-2U server, except there are no connectors at the back of the board for plugging into a chassis. OTOH, there's a SFP port hanging off the side of the board facing the wrong direction. And also a QSFP+ port on the back panel, presumably connected to a 40G NIC chip hiding under one of the heat sinks. Plus it has a TPM installed.

Was this for some weird hyperscaler project or something? It looks like the board has space for a SAS chip + ports, but they're not populated here. This form-factor works well enough for a storage server board, but without the SAS chips it has ~no I/O and only room for 2 DIMMs, which means it's not a general purpose system. I don't even see any place to put a local boot device, unless there are more connectors on the back. With the weird internal SFP+ connector, presumably it's intended to connect to another network device (or switch) in the same chassis, but why have 40+10 Gbps on the outside and only 10 Gbps on the inside?

Huh, the MAC stickers on the board imply that they're actually SFP28s, not SFP+. Which is weird--the Xeon D 1500 series has 2x 10G interfaces in the SoC, which would mean that the designer added 2 extra NIC chips, one for 40G and one for 2x25G. But that'd mean that they had 25G internal and 25/40G external, which would be useful when connecting to different generations of network hardware with at least 25G of bandwidth. That could make this some sort of firewall-like device, where they didn't really trust whatever was connected to the internal SFP.

Anyone else love a good mystery? Or better yet, recognize the board?

Disclaimer: it's not my eBay listing.

 

[BlueFox]

It's an custom OEM product for a network appliance. Supermicro makes hardware for plenty of other companies, including ones you wouldn't expect (like HPE and IBM).

 

[FrankTL]

I bought this board from a seller in the UK. It was advertised as in working condition and with a Xeon D-1581. Will keep you informed on my progress to get it up and running once I receive it.

From the photos in the listing, it looks like the power connector closely resembles HP backpane 10-pin power plugs (also found on some Lenovo servers) although not exactly identical.

There are around 4 'extra' chips on this board; the SAS chip which isn't populated on the IBQ variant, a chip which appears twice on the IBQ and once on the CIBF variant, and another chip above it. From the video from the Japanese user (see below for the url), one IC probably is the Mellanox ConnectX-4 lx controller, one that only is present on the IBQ I assume to be a 40GBs infiniband controller and one the PHY for the Intel 10GBps SOC LAN.

Once I receive the board I'll remove the heatsinks to confirm with the markings.

Note there's a different variant with the SAS ports populated, the X10SDV-CIBF-AM041 still up for sale: SUPERO X10SDV-CIBF-AM041 Mother board with Intel Xeon D-1581 CPU | eBay

edit: Japanese user who found this same model motherboard and manage to get it to boot as well as figure out the KVM plug VGA pinout. The OS shows there are two interfaces on a Mellanox ConnectX-4 lx :

 

[FrankTL]

update: just received the board. The 40G MAC is definitely for infiniband; there's a jumper labeled as such.

I'm wondering why the second 25G interface isn't present on the X10SDV-CIBF-AM041 because I'd assume it has the same Mellanox ConnectX-4 lx controller.

The Japanese user also demonstrates the PCIe slot working on the X10SDV-IBQ-AM041, with a NVMe M2 storage card. Similarly he has VGA hooked up - I intend to avoid the hassle of soldering that cable by going the BMC route.

heatsinks on the infiniband and Mellanox controllers seem to be glued; they don't come unstuck with the little effort I tried.

Next:
- figure out power connector pinout using any or combination of: contact japanese user, contact SM support, use multimeter.

 

[FrankTL]

with the DMM it looks like pins 1-5 could be COM and 6-9 as +12V.

Pin 10 is not obvious to me.

Some remarks after watching the video from the Japanese user:
* It was bought from a seller in the Kanagawa prefecture (not from US as OP or UK like mine).
* Quite different S/N from mine, I'd say it's unlikely they originate from the same batch. As other suggested, probably some sort of network appliance that was sold all over the globe rather than for a hyperscaler project.
* The previous owner has (unwittingly?) changed the heatsink orientation. The fins are not aligned for front-to-back airflow, unlike the board I got.
* The PSU was sold alongside the board. I didn't yet manage to get a good look at the model or even make in the video.
* The JKVM1 header looks like to be meant for use with AOC B1SA4-F-KVM or something very similar.

 

[adaptivesystems]

with the DMM it looks like pins 1-5 could be COM and 6-9 as +12V.

Pin 10 is not obvious to me.

Some remarks after watching the video from the Japanese user:
* It was bought from a seller in the Kanagawa prefecture (not from US as OP or UK like mine).
* Quite different S/N from mine, I'd say it's unlikely they originate from the same batch. As other suggested, probably some sort of network appliance that was sold all over the globe rather than for a hyperscaler project.
* The previous owner has (unwittingly?) changed the heatsink orientation. The fins are not aligned for front-to-back airflow, unlike the board I got.
* The PSU was sold alongside the board. I didn't yet manage to get a good look at the model or even make in the video.
* The JKVM1 header looks like to be meant for use with AOC B1SA4-F-KVM or something very similar.

picked up that other board, and found your post.it hasnt arrived yet, but looks specwise great for a home vm / kubes lab using low power.

so: the psu is x.com as he stated on twitter.

i suspect that its just atx-12vo, having looked up that psu it appears to be a 24 pin, and if you lok closely in the video you can see where its just been spliced manually...annoying that the guy hasnt wrote any pinouts etc.

suspect this might be the legit front usb cable CBL-CUSB-0983 - although thats probably the easiest to work out pinouts from, soldering stuff at that size is annoying as hell...also saw a post about wiping the kvm password if that ends up being a thing.

 

[adaptivesystems]

oh, and i figured out who the OEM is.

Amazon.

 

[FrankTL]

Yeah the Ebay seller told me Amazon, but I'm not 100% convinced yet until I can find some documentation.
Let's see if the guy responds on Twitter/X to our requests for more details on pinout.

I'm not sure about ATX12VO as it would require 12V standby and in his YouTube video, pin 6-9 seem 12V while ATX 12VO states pin 7-10 for 12V (with pin 7 12Vsb). Here are some pictures for various 10-pin connectors: the ATX 12VO one, a Lenovo 10-pin and the mystery SuperMicro one for the X10SDV-.-AM041.

They are all different, but I suspect the SuperMicro one is more similar to the propriety one(s?) used by large OEMs such as Lenovo (and maybe HP Proliant HDD backplanes).

edit: looking at the video there's a frame somewhere around 3:56 where the power connector comes in focus. It clearly shows the style 10-pin connector used by e.g. Lenovo, not a ATX 12VO connector:


The other relevant detail is connector JP1, left of the JPWR1 power connector and the USB0/1 header. This JP1 contains pins for PSU ON and PSU PWR OK, both of which are part of the ATX 12VO 10-pin spec (pins 1 and 6 respectively).

Based on the wiring seen in the still image at 5:47 in the video, it's pretty safe to conclude that connector « P3 » of the Delta PSU contains the green PS_ON# wire and shows that PIN 10 of the connector plugged into JPWR1 is purple (which would be 12 V SB if the colour coding is consistent with ATX 12VO or 5V SB if consistent with ATX 12V 2.x), while PIN 5 is black (GND/COM).

As PS_ON# is on the 10-pin Lenovo connector, this SuperMicro board as well as the Delta PSU clearly don't use the Lenovo layout either.

Note that Pin 8 of the connector in JPWR1 has a yellow wire with green stripe, not sure what that could be. Maybe just to denote a different 12V rail?

edit2: main open points at this point are:
- identification of the yellow-green wire (Pin 8 on the JPWR1 connector)
- identification of the purple wire (Pin 10 on the JPWR1 connector)
- understand why JTBT1 header is wired to the PSU
Speculation: if the motherboard is using an ATX 12VO-like power, then Pin 10 would be +12 V SB and Pin 8 could be the +12 V voltage sensing pin.
edit3: scrap that, the close up photo of the PSU on X/Twitter clearly shows no +12 V SB, but does show +5 V SB

 

[FrankTL]

just to confirm these motherboards do come from an AWS Snowball device (first gen Snowball Edge, introduced around AWS re:Invent 2016).

I just found a positive identification on an old YouTube video on AWS SnowBall. In the shot linked, the X10SDV-IBQ-AM041 is mounted upside-down at the bottom of the unit with the PSU sitting on top of it.

so the X10SDV-IBQ-AM041 would logically come from the compute edition, and the X10SDV-CIBF-AM041 from the storage edition.

 

[adaptivesystems]

finally got the board, and i think pin 10 is 5v. as the first place it appears to go into is this chip, via https://www.onsemi.com/pdf/datasheet/fds8858cz-d.pdf

but it may be 5vsb so the board can use power states...

 

[FrankTL]

+5Vsb would make most sense, as it also is in accordance with the ATX color coding for wires and matches the capabilities of the Delta PSU.

That kind of leaves the ambiguity around pin 8. Why the green stripe?
* a 12V sensing pin?
* a dedicated rail (in that case, why only one pin, as I'd expect a more even distribution between both rails)?
* +12Vsb (doesn't match the Delta PSU capabilities)?

Meanwhile I contacted SuperMicro for info, but it seems from their reaction that they have an NDA in place with Amazon that prohibits disclosing any data on these -AM041 motherboards. Delta appears to have similar NDA in place for custom contracts (referring to the reseller for more info).

 

[adaptivesystems]

oh, and no, the ps-on goes to ground. if you ground that pin, the psu turns on - i've done quite a few pc psu mods for repurposing in my time -


you can see at 1:48 on his vid, he's using a paper clip to directly ground it. so we can essentially ignore the jfp bit for now.

this is what i tihnk it looks like.


power

1-5 GND - have checked with MM connecting to the usb grounds and various other obvious ground plane
6-9 12V - these 4 are connected, whatever they are - must all be 12v...
10 5v sb




was hoping to jut be able to get it to power on enough to get to ipmi, and check power from there, but SM ipmi isn't as extensive as Dell/HP's.

 

[FrankTL]

finally got the board, and i think pin 10 is 5v. as the first place it appears to go into is this chip, via https://www.onsemi.com/pdf/datasheet/fds8858cz-d.pdf

but it may be 5vsb so the board can use power states...

The +5Vsb on the delta supplies 2A, which should be more than enough for the Aspeed BMC. Might actually be a good idea to try!

 

[FrankTL]

update
I've bought some cables and did some soldering to build a convertor from standard ATX 24-pin + EPS connectors to the custom connectors on this motherboard.

The good news is that it seems to work with the pin-out and voltages suggested in previous posts in this thread. Power consumption as reported by a Technoline Ltd. "Cost Control" unit is 9W for the IPMI only and about 45-50W when the system is up and running in pre-OS boot phase (no RAM, no connected peripherals other than a network cable with the IPMI). I'm not sure how accurate my measuring device is, but if anyone else wants to test, I'd advice a PSU that does at least 2A on the +5Vsb line. Note that auto start after power failure is enabled.

The bad news is that the IPMI doesn't listen on port 80 and 443; only port 22 (SSH) is accessible and of course the default password doesn't work.

The reset procedure for the Supermicro IPMI seems to imply booting from a rescue disk to start the ipmicfg tool.

That sounds easier said than done...
- I'll get some RAM because I doubt the board will boot up anything without access to memory
- will have to determine if the current BIOS settings allow for things like booting from bootp/PXE, USB or PCIe add-in card (about the only options I see)
- if that works, I'll try and boot a Linux OS which uses DHCP and has sshd running, or with some auto-start commands to capture the output of ipmicfg to file to inspect the current configuration, and then try to add a new admin user with known password or reset the current admin password.
- crossing my fingers that the above line works - if it doesn't I'll need to have more visibility of what's going on, and since I don't have the Supermicro KVM-break-out-cable, I'll first try with a serial console with a Linux boot that listens on ttyS0 or ttyUSB0
- if the serial console route doesn't work, I'll probably need to go hunt for the B1SA4-F-KVM or copy the Japanese guy and solder a custom VGA cable for the KVM port (crossing my fingers that I don't need to go this route).
- last resort would be to try a PCIe display card, but it doesn't seem that was working out-of-the box for the Japanese guy

 

[adaptivesystems]

meanwhile, i'm still waiting on bits n bobs from aliexpress/ebay etc to turn up for it. yaay - customs!

judging by the name on the file, this looks like its standard across all x10 series.

Firmware Resources | Support

Firmware Resources | Support Download Supermicro resource.

www.supermicro.com

i'll try that out at some point, as i have a chip programmer with a clip thingy.

 

[FrankTL]

I was searching the web for more details on the Infiniband controller on my X10SDV-IBQ-AM041 and deduced that the « IBQ » probably stands for a IB QDR adapter, likely provided by the Mellanox ConnectX-3 QDR controller such as on this add-on board: AOC-CIBQ-m1 | Add-on Cards | Accessories | Products - Super Micro Computer, Inc.

With that logic, the X10SDV-CIBF-AM041 would be sporting a Infiniband FDR controller such as the ConnectX-3 FDR on this board: AOC-CIBQ-m1 | Add-on Cards | Accessories | Products - Super Micro Computer, Inc.
... except that this doesn't correspond with the pictures in the ebay listing, as the cage clearly resembles SFP28 more than the QSFP used by Infiniband HDR and the MAC also clearly mentions « 25G ». I'm a bit at a loss why this particular board would disregard Supermicro naming conventions like that...

Looking at the SAS controllers historically used by Supermicro, it does seem to me that the X10SDV-CIBF-AM041 likely uses a SAS3008 or SAS3108 controller. Note that there also are 4 SATA ports on the X10SDV-CIBF-AM041 (which should go directly to the SOC - no idea why they aren't connected on my X10SDV-IBQ-AM041 ; similarly I don't get why the onboard 10GBe and USB 3.0 controllers aren't wired on the X10SDV-CIBF-AM041 while they are on mine).

Wrt a serial port connector and cable for the COM header with DMT layout on Supermicro, there is CBL-0010L which can cheaply be ordered directly from the Supermicro web store for delivery in the US; Unfortunately there doesn't seem to be a cheap way to get it where I live. There seem to be cheaper options, for example when you search for FPUSB813 you can get lucky and find second hand CSE-PT40L or CSE-PT40L-B0 for very good prices.

If I can get this thing to boot from network/USB media, and is in proper working order, the next step will be to mount it in a safe chassis; the open air/open frame cases for ATX size on Amazon seem to be a good fit as the standoff position can be adjusted (this thing does not seem to use anything near to standard positions for the motherboard mounting holes).

 

[FrankTL]

Update:
- received the RAM - unfortunately 2 non-identical 32GB sticks of Samsung 2Rx4 ECC RDIMM despite ebay adds showing the same picture. One is rated for 2133 MHz the other for 2666 MHz.
- tried booting with one stick in slot DIMMA1
- didn't see any packets when sniffing the 10GBe port
- tried with a bootable persistent linux USB stick in the USB3 port, it doesn't seem to boot the stick (tried both with MBR and GPT/UEFI-compatible boot device, neither seems to do anything).

The serial console is unlikely to do me any good, as I'd expect by default that bios access is redirected to the BMC.

Things I haven't tried yet, in the order of easy to hard:
- boot with dimms in boths slots
- try booting from the USB2 port (rear connector)
- try sniffing the ConnectX-4 ports (SFP28) for bootp/PXE
- investigate attack surface of IPMI (although ssh-only port access is not encouraging)
- solder a VGA cable like our Japanese friend did, as there seems to be little hope of finding a B1SA4-F-KVM

 

[sam55todd]

IMV despite price often being relatively low - getting random unknown OEM boards tends to be quite a time-consuming masochist-worth gamble with very doubtful rewards (if any) , those frequently have issues with form-factors (and customizing case for it), non-standard connectors (sometime even finding power supply pinout is a challenge), drivers/bios routines and other device identification/compatibility problems. Moreover people might need to buy extra cable-connectors/adapters/devices fitting specifically for that equipment (and having no much use for applications in regular PCs, therefore if board turns out to be useless and you can potentially return it then you end up with excessive expenditures you can't get refund because it's often has no basis since all additional things are fully functional and do match specification). Not even mentioning completely dark areas related to updates against vulnerabilities and exploits.

 

[FrankTL]

IMV despite price often being relatively low - getting random unknown OEM boards tends to be quite a time-consuming masochist-worth gamble with very doubtful rewards (if any) , those frequently have issues with form-factors (and customizing case for it), non-standard connectors (sometime even finding power supply pinout is a challenge), drivers/bios routines and other device identification/compatibility problems. Moreover people might need to buy extra cable-connectors/adapters/devices fitting specifically for that equipment (and having no much use for applications in regular PCs, therefore if board turns out to be useless and you can potentially return it then you end up with excessive expenditures you can't get refund because it's often has no basis since all additional things are fully functional and do match specification). Not even mentioning completely dark areas related to updates against vulnerabilities and exploits.

I call most of what you describe 'having some fun'.

 

[adaptivesystems]

well, i got some the connectors, and soldered them up. currently it appears to turn on, and hasn't released any magic smoke - so i'l call that a win.

currently not gt all of them, so cant really do much with it - waiting on the breakout board for vga, but have the flat ribbon fr it, and the actual vga port.

yay, more waiting.