New Fujitsu D3348-B23 Motherboard (LGA2011-3), $200 OBO ($140 accepted), Ebay

[Markess]

Short Version: New, bulk packaged with I/O Shield and Driver disk. Mine came well packed and double boxed. These are C612 chipset & support E5-v3 & v4. They don't have IPMI but do support S3 sleep, so probably best used as a desktop. One PCIe slot supports bifurcation (although I haven't tested that myself yet) (EDIT: Bifurcation tested to work, see Post #9 below). Seller accepted my initial BO of $140. Listing says shipping to U.S. only. Fujitsu D3348-B23 GS 2 Motherboard | eBay

More detail for those still interested, and since Fujitsu isn't really a household name in the U.S.:

Taking advantage of falling CPU prices, I got an E5-2680v4 (https://forums.servethehome.com/index.php?threads/ebay-intel-xeon-e5-2680v4-168.29838/) to build a desktop around. I guess I wasn't the only one with this idea though, because Ebay prices for used (and new) LGA 2011-3 motherboards seem to have gone up steeply in the last couple months. I was seeing used worktstation boards going for as much as they went for when they were new. And because the socket is "obsolete" new supply is scarce, although Provantage still shows stock for X10SRA at $280.00 if anyone is looking for a SuperMicro WS board.

Fujitsu's enterprise computing corporate offices are in Germany, so they're probably not as well known in the U.S. as they are in Europe. But, I"d had quite a lot of Fujitsu equipment when I worked in Europe some time back, and was very satisfied. So, when I saw this listing I thought I'd give it a go. For the price, I think its a much more mainstream and better documented option than the "no name" X99 boards that have been coming out recently.

Documentation, Drivers, BIOS, Certifications, Tested Memory list, etc.are all available via ftp site: ftp://ftp.kontron.com/Products/Motherboards/EoL_Motherboards/EoL_ExtendedLifeCycle/D3348-B/ . The most recent BIOS is from February 2020. Mine came with an August, 2018 BIOS and I updated it to current without issue. Of note for anyone checking out the ftp site: these boards boards are the B2 version, which has E5 v4 support.

I'm currently testing mine with Ubuntu 20.04 as host and Windows 10 in a guest VM with GPU passthrough (QEMU/KVM). No issues so far other than all the USB ports being in the same IOMMU group, complicating USB passthrough. Each PCIe slot has its own dedicated IOMMU group though, which simplified GPU and NVMe passthrough, so I guess it balances out.

Anyone knowing of other deals on LGA 2011-3 motherboards, please post them! I know there's folks out there looking.

 

[Lambdariver]

I use Fujitsu hardware quite a bit at work and have been very pleased with their initial and long term failure rate. I was sad when my sales rep told me they’re exiting the North American market for enterprise hardware.

 

[j_h_o]

What heatsink are you using on this board, and what case do you have it in? I'm considering a few options... wanted some input

 

[chinesestunna]

Great find! X99/C612 board prices are really silly for the longest time, makes it hard to take advantage of the v3/v4 Xeon prices. There's another thread on Chinese X99 boards and that seems to be a booming market as there's obviously market demand.
Too bad HP went with a proprietary PSU solution with X99/C612 gen motherboards, their stuff can be picked up for pennies: HP Z440 motherboard

 

[Markess]

Great find! X99/C612 board prices are really silly for the longest time, makes it hard to take advantage of the v3/v4 Xeon prices. There's another thread on Chinese X99 boards and that seems to be a booming market as there's obviously market demand.

True! I'd looked at the Chinese boards, and there's a lot of You Tube reviews to draw from. The most "legit" ones with decent build quality were all in the $140-150 price range. Which is how I chose a $140 BO bid on this to start with.

What heatsink are you using on this board, and what case do you have it in? I'm considering a few options... wanted some input

Previously, I had the "rack" one one side of my home office, the desktop PC on the other, and my 7 foot desk/work table in the middle. The old desktop is now upstairs with the kids, and the new "desktop PC" is in the rack, so all the noisy bits are in one place:

D3348-B23 Motherboard w/E5-2680 v4
4x16GB RDIMM
Noctua NH-U9DX I4
Chenbro RM41300
Corsair AX760i
Quadro P620 (Windows) & Radeon Pro WX2100 (Linux)
a pair of NVMe M.2 drives in PCIe slots and a pair of 2.5" SSD.
One additional 120mm & one 80mm PWM case fans

The case and PSU were part of a complete system I picked up for $40. The drive bay configuration is odd (3x5.25 & 4x3.5) so I doubt I would have chosen it if I were paying retail, but it works well for this use case and the price was great.

In testing, there was definite audible fan noise under load, but in normal use its pretty silent. There's room for more fans, but temps were pretty low, so I'm running it with just the two case fans for now. The PSU is a semi-silent model & the fan usually only kicks on if the VM is running and I'm doing something with both operating systems at the same time, and even then its still pretty silent. I've got noise throttled down on all the other gear, and this fits right in.

 

[Prof_G]

Do you know if you put a pci NVMe card can you use bifurcation on it?

 

[Markess]

Do you know if you put a pci NVMe card can you use bifurcation on it?

I've got a Supermicro AOC-SLG3-2M2 that I can test with and report back.

 

[Prof_G]

Oh and I know you got the 2133 memory but Newegg.com has some 2400mhz 16Gb Curcial
$50 with some free cheap SD card as today’s daily deal.

 

[Markess]

Do you know if you put a pci NVMe card can you use bifurcation on it?

The short answer is yes, it works.

The long answer:
- Bifurcation is available only for Slot 3.
- I needed to set bifurcation FIRST (I used x4x4x4x4) and then install the AOC-SLG3-2M2 SECOND after allowing the system to boot once with bifurcation set. When I tried to do it all in one boot cycle, installing the card first and setting bifurcation on the subsequent boot, booting failed.
- I was able to boot from both drives I tested with: Intel 665p (Ubuntu 20.04) and Samsung PM951 (Win 10).
- The system put each drive into their own unique IOMMU group.

 

[hmartin]

Too bad HP went with a proprietary PSU solution with X99/C612 gen motherboards, their stuff can be picked up for pennies: HP Z440 motherboard

It's not ATX, but adapters exist: HP Server Z440 Main Power 24 and 8 Pin to 18 and 12 Pin Adapter Cable

I can't find the PSU pinout with my Google-foo, but the above adapter appears to be passive, so if the pinout is known (as it is for the Z#20 series) it is trivial to build your own adapter for much less than Mod DIY would like to charge you. You can already buy Z#20 series adapters for a pittance on eBay/AliExpress, or build your own with public information about the pinout.

Just checked and you can buy Z440/Z640 adapters on AliExpress for ~$15. Search terms like "ATX 24Pin to 18Pin Adapter Converter Power Cable Cord for HP Z440 Z640" will find them.

 

[Markess]

Just checked and you can buy Z440/Z640 adapters on AliExpress for ~$15. Search terms like "ATX 24Pin to 18Pin Adapter Converter Power Cable Cord for HP Z440 Z640" will find them.

I researched trying this a while back. As I recall from reading about other folks' adventures trying to transplant their HP Z440 boards into ATX cases, there's also issues with proprietary sensors and maybe front panel connectors. For example, the temperature sensor that controls fan speed for the case fan headers is part of the power switch harness. That harness has a proprietary connector of course. So, if you want fan control you either need to buy the switch harness, or make your own with the right sensor. If you don't already have a board, the costs for adapters and harness(es) make it less appealing as a cheap alternative. On the other hand, barebones units complete with heatsinks and power supplies are hitting the secondary market in volume now and go for around $200 shipped in the US. Drop in CPU, RAM and disks and you're ready to go.

Dell's LGA 2011-3 workstation boards are similarly cheap, but have issues as well, just different ones. Power & connectors are less of an issue, but the mounting hole pattern is non-standard and the PCIe slots are set back in relation to the I/O connectors. So if you drill your case for standoffs that put the I/O in the right spot, the PCIE slots are too far back in the case to mount cards. If you're not using any PCIe or are using cards that don't require external connectors, you can probably work around it. But as with the HP, its going to require some effort.

 

[chinesestunna]

It's not ATX, but adapters exist: HP Server Z440 Main Power 24 and 8 Pin to 18 and 12 Pin Adapter Cable

I can't find the PSU pinout with my Google-foo, but the above adapter appears to be passive, so if the pinout is known (as it is for the Z#20 series) it is trivial to build your own adapter for much less than Mod DIY would like to charge you. You can already buy Z#20 series adapters for a pittance on eBay/AliExpress, or build your own with public information about the pinout.

Just checked and you can buy Z440/Z640 adapters on AliExpress for ~$15. Search terms like "ATX 24Pin to 18Pin Adapter Converter Power Cable Cord for HP Z440 Z640" will find them.

Haha read my mind, I searched while working on recent X99 build but only found a mod-pc version for $40 so gave up. Eventually found a good deal on Asus X99 board and called it a day

 

[hmartin]

As I recall from reading about other folks' adventures trying to transplant their HP Z440 boards into ATX cases, there's also issues with proprietary sensors and maybe front panel connectors.

Yes, I did that with a Z420 motherboard. You have to use some jumper wires or solder across several pins (USB/IEEE1394/FP-Audio) to fool the board into thinking it has all the normal front-panel connectors. The temperature sensor (at least on the Z420) is just an NPN transistor across 3 pins. BIOS will require confirmation during boot if you don't have all the system fans connected and reporting an RPM though.

So, yes, it's somewhat of a pain, but HP workstation boards are nice for the money (though the Z420 doesn't appear to support bifurcation, which is a bummer). They support S3 too, so if you don't want to expend 40W constantly you can put them to sleep where they only consume 10W. WoL doesn't work on my Z420 with the modded PSU connection though.

 

[itronin]

@Sleyk you made me go look.

found the manual (DE/EN edition) and the specsheet (EN)

hmmm lots of phys x16?

this is what I see in the specsheet and manual

pcie x8 gen 3 (4 lanes)
pcie x16 gen3 (16 lanes)
pcie x16 gen2 (4 lanes)
pcie x16 gen3 (4 lanes)
pcie x16 gen3 (16 lanes)
pcie x8 gen2 (1 lane)
pcie x8 gen2 (1 lane)

sooo
6 lanes total of gen2 (chipset)
40 lanes total of gen3 (cpu)

I do not see a PLX in the manual's block diagram though.

question might be though can you use bifurcation to reassign to that x16 (4 lanes) the other x8 (4lanes) and have
3 x16 physical, 2 with x16 data and one with x8 data all gen 3? its not in the manual or specsheet.

the physical layout and slot sizing (rather than open ended) give this board a lot of flexibility though.

 

[Markess]

@Sleyk and @itronin :

@itronin has it right. There's 40 Gen3 lanes from the CPU. The others are Gen2 lanes from the PCH. @itronin's list is in the correct order of slots, where #7, which is closest to the CPU, is at the TOP of the list, and Slot #1, which is furthest from the CPU at the BOTTOM of the list:

---I/O & CPU Are Here --
Slot 7: pcie x8 gen 3 (4 lanes) (CPU)
Slot 6: pcie x16 gen3 (16 lanes) (CPU)
Slot 5: pcie x16 gen2 (4 lanes) (PCH)
Slot 4 pcie x16 gen3 (4 lanes) (CPU)
Slot 3: pcie x16 gen3 (16 lanes) (CPU) (Supports Bifurcation)
Slot 2: pcie x8 gen2 (1 lane) (PCH)
Slot 1: pcie x8 gen2 (1 lane) (PCH)
-- Edge of the board is Here -

Slot 3 is the ONLY ONE that supports Bifurcation. I assume that's where the Riser goes in their turn-key 2U servers. Bifurcation control is in the PCIe managment submenu in the BIOS and you get the usual choices: x16, x4x4x4x4, and all the combinations of X4 & x8 in between. I have confirmed it works...at least for me and my 2 drive Supermicro NVMe expansion card with bifurcation set in BIOS to x4x4x4x4.

Alogn with the four slots that are x16 physically, the slots that are physically X8 are open ended which makes it easy if there's a need to fit a card that's physically got an x16 connector. However, only slots 2 and 7 actually have the clearance to the rear of the slot to do this. Not enough room behind Slot #1 due to the SATA ports.

Since I have the board here, and I'm still testing (I'll be doing a fresh OS install later), let me know if there's anything else you'd like tested/checked out.

 

[hmartin]

In case anyone in the EU is looking for such a deal, you can find the D3358-A13 (2011-3 dual socket) for around 120€:

Fujitsu D3358-A13 GS2 Mainboard NEU Sockel LGA2011-3 Celsius R940 S26361-D3358 | eBay

Finden Sie Top-Angebote für Fujitsu D3358-A13 GS2 Mainboard NEU Sockel LGA2011-3 Celsius R940 S26361-D3358 bei eBay. Kostenlose Lieferung für viele Artikel!

www.ebay.de

 

[gb00s]

In case anyone in the EU is looking for such a deal, you can find the D3358-A13 (2011-3 dual socket) for around 120€:

Fujitsu D3358-A13 GS2 Mainboard NEU Sockel LGA2011-3 Celsius R940 S26361-D3358 | eBay

Finden Sie Top-Angebote für Fujitsu D3358-A13 GS2 Mainboard NEU Sockel LGA2011-3 Celsius R940 S26361-D3358 bei eBay. Kostenlose Lieferung für viele Artikel!

www.ebay.de

Just look at the power delivery. This is not as easy to adjust as the D3348-B23.

Corsair AX760i

When I see the power delivery layout, how did you manage to connect it to the standard connectors?

 

[Markess]

When I see the power delivery layout, how did you manage to connect it to the standard connectors?

Not sure if you're asking about the D3358, the dual processor board that at @hmartin posted the link to, or the D3348 that I linked to in the original post? The D3348 that I originally posted about has a standard 24 pin ATX connector and 8 Pin CPU connector. So, the AX760i, which is an ATX PSU, just plugged right in. I'm not familiar with the other motherboard.

 

[gb00s]

Ok. I just realized there is an A12 version out there with the same layout but different power connectors.

 

[gb00s]

I just found more versions of this board with very similar layout and specs, but again totally different power connectors ... Versions are A13 + A21/23.

Image of the A21/23 with Fujitsus common power connectors >> 16/12




Fujitsu D3348-A13 with Fujitsus common power connectors >> 16/12



I found another eBay listing for a B23 version with the regular connectors 24/8. Seems like there's a significant difference between A & B power connector wise. Take care what you buy.