ASrockrack 2U4N-F/X200

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

onsit

Member
Jan 5, 2018
98
26
18
33
Recently was following a blog by lukminer, after he did the record for biggest hashing power under 1 machine (4u build with multiple Phi cards). Was pleased to find a more recent update about a pre-built solution that uses socket baesde PHIs as opposed to the Cards. ASrockrack is ASrock's taiwanese based small business solution department that doesn't seem to have a website for all the gear, and relies of manual invoicing and talking with clients.

Anyway. I was intrigued by the small form factor, and how much hashing power these units put out. Especially since my current 24U server rack is filled to the brim with OLD hardware and only does 12 KH/s. And a lot of power is being turned into heat due to the Gold rated PSUs, and lots of them. Switching to just 1 4U unit, with platinum power would make for a much more efficient overall hashing system without generating as much heat (factoring this in based on Gold PSUs stepping down 240v to 12v, still a bit hot running at 50% capacity).

Here are the details.
  1. $3000, won't ship until April/May
  2. 4U, 4 node sub modules, with 1 socket for a X200 processor
  3. Comes with chasis, 2 platinum 1600w, 4x Xeon Phi KNL 7210, No ram, No Risers
  4. You don't need ram as the chip has DIMMs on the SoC from what I understand, just need to load linux up on a usb, and the SoC should be able to run the OS on the chip.
  5. 2800 H/s per Phi 7210
  6. ~12-11 KH/s @ 1200-1100 watts
  7. Comes out to... about ~ 4H/$ and 10H/watt (Essentially VEGA 64 cost and efficiency, before the price hike).
I attached the spec sheet for the chassis that I got as part of the email from their sales department.

I've tested a few systems so far that I found to be around the same investment cost @ 4H/$ and only about 4H/watt in efficiency.
  • OC nodes with E5-2660 V1 (investement cost was about 2.7H/$1, and power consumption is about 5.3H/watt)
  • RX 550 with bios edits (2.7H/$1, and about 8H/watt)
  • E5-2450L (4H/$1, and about 3H/watt)
This setup obviously requires MUCH more money down. However, it is vastly efficient, compact, and could have some decent resale to the Machine Learning community.
 

Attachments

Last edited:
  • Like
Reactions: leebo_28 and Marsh

polyfractal

Member
Apr 6, 2016
33
27
18
Are you sure about those OC prices? $4/H sounds very high. I semi-recently expanded my OC cluster with an additional 8 nodes and it worked out to this:
  • x4 Quanta servers @ $175ea == $696
  • x16 E5-2650L V2 @ $94ea == $1,519
  • x8 320GB drives @ 19ea == $152
  • 24gb ECC RAM * 8 nodes @ $1.25/gb == $240
  • Hash Rate: ~610 H/s per node
Total Cost: $2607
Total Hash: 4880/s
Cost-to-Hash ratio: $0.53/H

Maybe prices have changed a lot since I built? I know RAM is way more expensive (I repurposed some RAM from other machines), but even at $4/gb you could throw a stick or two in each machine and have the same price. Not sure about CPU prices right now... I personally would have preferred something other than L version, but best I could do at the time.

Now, it's definitely not a compact setup, definitely much larger than a 4U system. :) I don't have hard power numbers (haven't had time to rewire my energy consumption monitor yet), but I'm guessing my entire OC cluster pulls around 2300W. That includes an additional eight E5-2670 V1s, does around 8kH/s, and puts my overall efficiency somewhere around 3-4 H/W. So not as efficient either.

All that aside, that's a pretty neat setup. I really want to like the Phis, but I've heard they are a huge pain to code against. It's a shame Intel hasn't done a better job pushing/marketing/developing them... I think they are a great multipurpose compute platform compared to GPUs.

Edit: Oh hey onist, we chatted briefly about this ^^^^ OC setup on reddit, regarding V2s :)
 
  • Like
Reactions: Joel

onsit

Member
Jan 5, 2018
98
26
18
33
Woops, yeah it's meant to be 4Hashes per 1 Dollar. Which seems to be the common targeted ratio for return atm from all the optimal solutions. Vegas, Cpu builds, Budget GPU builds. 4 Hashes a dollar, haven't seen quite anything with better return for the dollar spent.
 

its

New Member
Mar 14, 2017
10
2
3
56
Recently was following a blog by lukminer, after he did the record for biggest hashing power under 1 machine (4u build with multiple Phi cards). Was pleased to find a more recent update about a pre-built solution that uses socket baesde PHIs as opposed to the Cards. ASrockrack is ASrock's taiwanese based small business solution department that doesn't seem to have a website for all the gear, and relies of manual invoicing and talking with clients.

Anyway. I was intrigued by the small form factor, and how much hashing power these units put out. Especially since my current 24U server rack is filled to the brim with OLD hardware and only does 12 KH/s. And a lot of power is being turned into heat due to the Gold rated PSUs, and lots of them. Switching to just 1 4U unit, with platinum power would make for a much more efficient overall hashing system without generating as much heat (factoring this in based on Gold PSUs stepping down 240v to 12v, still a bit hot running at 50% capacity).

Here are the details.
  1. $3000, won't ship until April/May
  2. 4U, 4 node sub modules, with 1 socket for a X200 processor
  3. Comes with chasis, 2 platinum 1600w, 4x Xeon Phi KNL 7210, No ram, No Risers
  4. You don't need ram as the chip has DIMMs on the SoC from what I understand, just need to load linux up on a usb, and the SoC should be able to run the OS on the chip.
  5. 2800 H/s per Phi 7210
  6. ~12-11 KH/s @ 1200-1100 watts
  7. Comes out to... about ~ 4H/$ and 10H/watt (Essentially VEGA 64 cost and efficiency, before the price hike).
I attached the spec sheet for the chassis that I got as part of the email from their sales department.

I've tested a few systems so far that I found to be around the same investment cost @ 4H/$ and only about 4H/watt in efficiency.
  • OC nodes with E5-2660 V1 (investement cost was about 4H/$1, and power consumption is about 4H/watt)
  • RX 550 with bios edits (4H/$1, and about 6.5H/watt)
  • E5-2450L (4H/$1, and about 4H/watt)
This setup obviously requires MUCH more money down. However, it is vastly efficient, compact, and could have some decent resale to the Machine Learning community.
 

onsit

Member
Jan 5, 2018
98
26
18
33
Are you sure about those OC prices? $4/H sounds very high. I semi-recently expanded my OC cluster with an additional 8 nodes and it worked out to this:
  • x4 Quanta servers @ $175ea == $696
  • x16 E5-2650L V2 @ $94ea == $1,519
  • x8 320GB drives @ 19ea == $152
  • 24gb ECC RAM * 8 nodes @ $1.25/gb == $240
  • Hash Rate: ~610 H/s per node
Total Cost: $2607
Total Hash: 4880/s
Cost-to-Hash ratio: $0.53/H

Maybe prices have changed a lot since I built? I know RAM is way more expensive (I repurposed some RAM from other machines), but even at $4/gb you could throw a stick or two in each machine and have the same price. Not sure about CPU prices right now... I personally would have preferred something other than L version, but best I could do at the time.

Now, it's definitely not a compact setup, definitely much larger than a 4U system. :) I don't have hard power numbers (haven't had time to rewire my energy consumption monitor yet), but I'm guessing my entire OC cluster pulls around 2300W. That includes an additional eight E5-2670 V1s, does around 8kH/s, and puts my overall efficiency somewhere around 3-4 H/W. So not as efficient either.

All that aside, that's a pretty neat setup. I really want to like the Phis, but I've heard they are a huge pain to code against. It's a shame Intel hasn't done a better job pushing/marketing/developing them... I think they are a great multipurpose compute platform compared to GPUs.

Edit: Oh hey onist, we chatted briefly about this ^^^^ OC setup on reddit, regarding V2s :)
Your quantas are a bit cheaper than my wiwynns. Here is my spread sheet, and yes my math was a bit wrong until I looked at what my spreadsheet said.

 

its

New Member
Mar 14, 2017
10
2
3
56
Shipping is not included. It can get pricey shipping such a system from Taiwan.
 

onsit

Member
Jan 5, 2018
98
26
18
33
Shipping is not included. It can get pricey shipping such a system from Taiwan.
Shipping would only be about $200, DHL. I bought a few things this large before from ali express - and shipping is usually pretty cheap...
 

coinstash

New Member
Mar 16, 2016
14
4
1
Brisbane
I enquired about this and was told that power supplies had been downgraded to 1200W units due to long lead times from the manufacturer. More likely reason is that the seller is trying to squeeze the maximum possible profit from the deal. Considering that Lukas has measured the consumption of a 4 x 7210 system at 1328 watts this just seems like cheapass dodgy engineering to me. It also leaves no room for adding memory or HDDs for other uses of the system.

I'll pass.

Oh, and if you think Xeon Phi machines have any resale value, I have 38 X100 copprocessor cards that I'd love to sell you. :)
 
Last edited:
  • Like
Reactions: pgh5278

spfoo

Member
May 23, 2017
102
16
18
Recently was following a blog by lukminer, after he did the record for biggest hashing power under 1 machine (4u build with multiple Phi cards). Was pleased to find a more recent update about a pre-built solution that uses socket baesde PHIs as opposed to the Cards. ASrockrack is ASrock's taiwanese based small business solution department that doesn't seem to have a website for all the gear, and relies of manual invoicing and talking with clients.

Anyway. I was intrigued by the small form factor, and how much hashing power these units put out. Especially since my current 24U server rack is filled to the brim with OLD hardware and only does 12 KH/s. And a lot of power is being turned into heat due to the Gold rated PSUs, and lots of them. Switching to just 1 4U unit, with platinum power would make for a much more efficient overall hashing system without generating as much heat (factoring this in based on Gold PSUs stepping down 240v to 12v, still a bit hot running at 50% capacity).

Here are the details.
  1. $3000, won't ship until April/May
  2. 4U, 4 node sub modules, with 1 socket for a X200 processor
  3. Comes with chasis, 2 platinum 1600w, 4x Xeon Phi KNL 7210, No ram, No Risers
  4. You don't need ram as the chip has DIMMs on the SoC from what I understand, just need to load linux up on a usb, and the SoC should be able to run the OS on the chip.
  5. 2800 H/s per Phi 7210
  6. ~12-11 KH/s @ 1200-1100 watts
  7. Comes out to... about ~ 4H/$ and 10H/watt (Essentially VEGA 64 cost and efficiency, before the price hike).
I attached the spec sheet for the chassis that I got as part of the email from their sales department.

I've tested a few systems so far that I found to be around the same investment cost @ 4H/$ and only about 4H/watt in efficiency.
  • OC nodes with E5-2660 V1 (investement cost was about 2.7H/$1, and power consumption is about 5.3H/watt)
  • RX 550 with bios edits (2.7H/$1, and about 8H/watt)
  • E5-2450L (4H/$1, and about 3H/watt)
This setup obviously requires MUCH more money down. However, it is vastly efficient, compact, and could have some decent resale to the Machine Learning community.
Thank you for this. You made me feel like a hero.
 

onsit

Member
Jan 5, 2018
98
26
18
33
I enquired about this and was told that power supplies had been downgraded to 1200W units due to long lead times from the manufacturer. More likely reason is that the seller is trying to squeeze the maximum possible profit from the deal. Considering that Lukas has measured the consumption of a 4 x 7210 system at 1328 watts this just seems like cheapass dodgy engineering to me. It also leaves no room for adding memory or HDDs for other uses of the system.

I'll pass.

Oh, and if you think Xeon Phi machines have any resale value, I have 38 X100 copprocessor cards that I'd love to sell you. :)
It's obviously a liquidation effort from Intel's heavily discounted older generation chips, and the few manufacturers that made rackable units for them.

One thing to note - is this 2U comes with 2 PSUs but they are not hot-swap, redundant either. So it's 1200W x2 as both PSUs need to be functioning to power the full system. I'd imagine it's 2 nodes per PSU.

Which will leave you plenty of power for SSD, and RAM. But yes - resale value of a PHI based system is going to be abysmal.
 

coinstash

New Member
Mar 16, 2016
14
4
1
Brisbane
My point was that with two 1600W power supplies - which would be almost identical in cost to build, it's just heavier FETs after all - there would have been redundancy if one of them died. Hot swap I can live without.

I very much doubt that they're arranged to supply only two cards per power supply, that would be the stupidest design ever. In fact Lukas has mentioned that with only one supply in place he couldn't run all four cards at full speed due to current limiting.
 
Last edited:

onsit

Member
Jan 5, 2018
98
26
18
33
My point was that with two 1600W power supplies - which would be almost identical in cost to build, it's just heavier FETs after all - there would have been redundancy if one of them died. Hot swap I can live without.

I very much doubt that they're arranged to supply only two cards per power supply, that would be the stupidest design ever. In fact Lukas has mentioned that with only one supply in place he couldn't run all four cards at full speed due to current limiting.
I also wouldn't be surprised if these are developer branded Phi chips similar to Intel ES series for new generations.

http://info.exxactcorp.com/G0M0BK10s31JN0Sq0000FW3

Exxact is selling the same setup, slightly more expensive. But they clearly state its a development kit / test bed.
 

coinstash

New Member
Mar 16, 2016
14
4
1
Brisbane
I don't know if you realize it but selling ES chips is illegal. If I received such from a reputable manufacturer I'd be very surprised. Unsure if Developer branding is equivalent but since Exxact are offering them I don't think so.

Price of the Exxact system is quite good considering it comes with 48GB of RAM and a 150GB SSD. They don't publish a full system spec or show the internals though, so I'm not very interested.