High core count v3/v4 E7 CPUs (as well as v2 for that matter) are often available at relatively low prices. A big reason the prices are low is that compatible boards are scarce, as they're quad socket parts that were only found in relatively exotic and high end systems.
One of the most widely available boards is the Supermicro X10QBI. The X10QBI is an unusual board: it has four 2011-1 (E7 compatible) sockets and 96 dimm slots. The 96 dimm slots are accomplished by the ram being placed on daughter boards, with part-number X10QBI-MEM. The daughterboard based ram creates a lot of confusion because there are multiple versions of the memory boards: MEM1 rev 1.01, MEM1 rev 2.0, and MEM2 rev 1.01.
Mem1 rev 1.01 uses the Intel Jordan Creek 1 chip, the other to boards uses Jordan Creek 2. Mem2 is a DDR4 board (The JC2 chip can do either) while both of the Mem1 boards use DDR3.
X10QBI also has the BMC, nics, and VGA on a daughterboard. According to the manual the board will not boot with the BMC board. For some reason electronics recyclers remove and sell the BMC board separately.
According to the supermicro docs, v3 and v4 cpus are supported in X10QBI but only with Mem1 rev 2 and Mem2. So, contrary to common belief in 2011-1 boards it is possible to run v3/v4 CPUs with DDR3 ram, and it's even a supported configuration.
However, Mem1 rev 2 boards don't appear to be common. I have some, but have seen fairly few show up in the surplus market.
There are also at least two revisions of the x10qbi itself. I have rev 1.01 and rev 1.2a. Both seem equivalent so far.
Getting a v3/v4 cpu to boot in x10qbi requires a new-ish bios. None of the ones I've obtained except for a sys-4048b-tr4ft that had mem2 boards have had a recent enough bios. The bios can be updated from the BMC, though it does the typical supermicro licensed feature thing. There is a one-liner shell keygen that uses openssl for sha1, if you find yourself stuck in the middle of the night trying to fix one of these things.
Now as I mentioned before, the common mem1 rev 1.01 does not officially support v3/v4 cpus. But I have personally found that they work. Some support matrix documents I receive indicate that for JC1 chip intel performs only limited on electrical and operational tests and did not validate w/ the full matrix of supported memory configurations. So YMMV.
One challenge, then, is that systems based on X10QBI are somewhat finicky. There are a LOT of parts. If any memboard is even somewhat mis-seated, the boot process will fail with inscrutable post codes which only sometimes indicate which board is the issue. The 2011 sockets are fragile-- no more so than other modern sockets-- but it's easy to get bent pins when you're working with a *bunch* of CPUs in a crowded system. I would highly recommend that anyone who wants to experiment here obtain a set of v2 cpus to use for testing or as a fallback in case v3/v4 do not work. (I have a bunch of 2.8GHz E7-4890 v2s that I'd be happy to sell for the ~$50/ea I paid for them, if anyone is interested)
These systems boot fairly slowly. Making it all the way to bios takes about 5 minutes, and when you change hardware the system will sometimes reboot and go through the self-tests again-- so 10 minutes to try a configuration is pretty common. If you're inserting memboards one a time it can easily take two hours (including the time it takes to install the ram itself).
Ram compatibility is also at least somewhat complicated: I never got these systems to post with 4GB ECC rdimms. Full performance from these systems require 32x 1333+ MHz (for MEM1) or 32x 1600 MHz ram. The JC controllers do an interesting trick where the 4-way memory interface of the e7 cpu effectively becomes 8-way per socket by running it at the full 3200 MHz while running the ram at half that speed and pairing up two chips at a time. Supermicro has memory compatibility lists, a number of chips on it are widely available.
The systems can be booted with dimms on any memboard, for testing, however. CPUs without ram will be inactive. Some of the JC docs also suggest that 64x dimms can have better performance depending on the chips ranks due to rank interleave.
X10QBI uses a somewhat unorthodox powersupply which has more than the usual number of cpu power connectors, otherwise its an atx psu. Be aware if you're thinking of trying to get a motherboard and use it directly. I'd recommend against that-- esp with the memboards being particular about how well they're seated.
I currently have 6 x10qbi systems each running with four E7-8880 2.2GHz ES cpus each. Three of the systems are Mem1 rev 1.01, two of them are mem1 rev 1.2, and one of them uses mem2 (w/ DDR4). After much tweaking and cursing, dealing with a dead power distribution board, a bad cmos battery, some bent 2011 pins, and @#$@ heatsink compound getting on the cpu pins... all of them are up and appear stable under high load.
If anyone else wants to try out v3/v4 cpus in an unsupported configuration I'd be glad to lend what knowledge I've gained. Understand that I might not have the complete picture and it might not work with some hardware, some ram, or might not actually be stable (though it appears to be for me). Supermicro appears to know nothing about v3/v4 cpus working in these older systems, and certainly doesn't support it themselves. Because these systems are heavy and a pain to ship, you might be able to obtain them much less expensively than the going rate on ebay (I did).
One challenge I had with the BMC board is that 33% of the boards I got from ebay had a password set, and if a password is set and you need the BMC to upgrade the bios before you boot-- you're out of luck. I got around this by using an unpassworded BMC to bring a system up, then after it was working swapped BMCs and used ipmitool to reset the password. I tried using a network exploit against older BMC firmware, but none of mine were vulnerable. If someone here runs into this problem I might be able to help you out by doing the same trick for you in exchange for snack money.
I only discovered the v3/v4 apparently-compatibility myself because I obtained several systems, some of which had the newer mem boards and were supported so I had v4 cpus for them and I tried the cpus in the unsupported systems.
One of the most widely available boards is the Supermicro X10QBI. The X10QBI is an unusual board: it has four 2011-1 (E7 compatible) sockets and 96 dimm slots. The 96 dimm slots are accomplished by the ram being placed on daughter boards, with part-number X10QBI-MEM. The daughterboard based ram creates a lot of confusion because there are multiple versions of the memory boards: MEM1 rev 1.01, MEM1 rev 2.0, and MEM2 rev 1.01.
Mem1 rev 1.01 uses the Intel Jordan Creek 1 chip, the other to boards uses Jordan Creek 2. Mem2 is a DDR4 board (The JC2 chip can do either) while both of the Mem1 boards use DDR3.
X10QBI also has the BMC, nics, and VGA on a daughterboard. According to the manual the board will not boot with the BMC board. For some reason electronics recyclers remove and sell the BMC board separately.
According to the supermicro docs, v3 and v4 cpus are supported in X10QBI but only with Mem1 rev 2 and Mem2. So, contrary to common belief in 2011-1 boards it is possible to run v3/v4 CPUs with DDR3 ram, and it's even a supported configuration.
However, Mem1 rev 2 boards don't appear to be common. I have some, but have seen fairly few show up in the surplus market.
There are also at least two revisions of the x10qbi itself. I have rev 1.01 and rev 1.2a. Both seem equivalent so far.
Getting a v3/v4 cpu to boot in x10qbi requires a new-ish bios. None of the ones I've obtained except for a sys-4048b-tr4ft that had mem2 boards have had a recent enough bios. The bios can be updated from the BMC, though it does the typical supermicro licensed feature thing. There is a one-liner shell keygen that uses openssl for sha1, if you find yourself stuck in the middle of the night trying to fix one of these things.
Now as I mentioned before, the common mem1 rev 1.01 does not officially support v3/v4 cpus. But I have personally found that they work. Some support matrix documents I receive indicate that for JC1 chip intel performs only limited on electrical and operational tests and did not validate w/ the full matrix of supported memory configurations. So YMMV.
One challenge, then, is that systems based on X10QBI are somewhat finicky. There are a LOT of parts. If any memboard is even somewhat mis-seated, the boot process will fail with inscrutable post codes which only sometimes indicate which board is the issue. The 2011 sockets are fragile-- no more so than other modern sockets-- but it's easy to get bent pins when you're working with a *bunch* of CPUs in a crowded system. I would highly recommend that anyone who wants to experiment here obtain a set of v2 cpus to use for testing or as a fallback in case v3/v4 do not work. (I have a bunch of 2.8GHz E7-4890 v2s that I'd be happy to sell for the ~$50/ea I paid for them, if anyone is interested)
These systems boot fairly slowly. Making it all the way to bios takes about 5 minutes, and when you change hardware the system will sometimes reboot and go through the self-tests again-- so 10 minutes to try a configuration is pretty common. If you're inserting memboards one a time it can easily take two hours (including the time it takes to install the ram itself).
Ram compatibility is also at least somewhat complicated: I never got these systems to post with 4GB ECC rdimms. Full performance from these systems require 32x 1333+ MHz (for MEM1) or 32x 1600 MHz ram. The JC controllers do an interesting trick where the 4-way memory interface of the e7 cpu effectively becomes 8-way per socket by running it at the full 3200 MHz while running the ram at half that speed and pairing up two chips at a time. Supermicro has memory compatibility lists, a number of chips on it are widely available.
The systems can be booted with dimms on any memboard, for testing, however. CPUs without ram will be inactive. Some of the JC docs also suggest that 64x dimms can have better performance depending on the chips ranks due to rank interleave.
X10QBI uses a somewhat unorthodox powersupply which has more than the usual number of cpu power connectors, otherwise its an atx psu. Be aware if you're thinking of trying to get a motherboard and use it directly. I'd recommend against that-- esp with the memboards being particular about how well they're seated.
I currently have 6 x10qbi systems each running with four E7-8880 2.2GHz ES cpus each. Three of the systems are Mem1 rev 1.01, two of them are mem1 rev 1.2, and one of them uses mem2 (w/ DDR4). After much tweaking and cursing, dealing with a dead power distribution board, a bad cmos battery, some bent 2011 pins, and @#$@ heatsink compound getting on the cpu pins... all of them are up and appear stable under high load.
If anyone else wants to try out v3/v4 cpus in an unsupported configuration I'd be glad to lend what knowledge I've gained. Understand that I might not have the complete picture and it might not work with some hardware, some ram, or might not actually be stable (though it appears to be for me). Supermicro appears to know nothing about v3/v4 cpus working in these older systems, and certainly doesn't support it themselves. Because these systems are heavy and a pain to ship, you might be able to obtain them much less expensively than the going rate on ebay (I did).
One challenge I had with the BMC board is that 33% of the boards I got from ebay had a password set, and if a password is set and you need the BMC to upgrade the bios before you boot-- you're out of luck. I got around this by using an unpassworded BMC to bring a system up, then after it was working swapped BMCs and used ipmitool to reset the password. I tried using a network exploit against older BMC firmware, but none of mine were vulnerable. If someone here runs into this problem I might be able to help you out by doing the same trick for you in exchange for snack money.
I only discovered the v3/v4 apparently-compatibility myself because I obtained several systems, some of which had the newer mem boards and were supported so I had v4 cpus for them and I tried the cpus in the unsupported systems.
Last edited: