I am reviving this thread for a very good reason, I have upgraded the server! - I have finally made use of that second CPU socket.
Recently I have been hitting up against the performance limits of the E5-1650 I installed in this system in August 2014. I wrote an Inference Engine, a type of machine learning system and it depends heavily on processing internet data so it cannot really be accelerated by GPU's - It needs those brawny x86 cores.
So I considered my options. I could purchase an AMD 1950X which has ECC memory support and 16 cores. According to the benchmarks, quite a nice processor. But I'd need to purchase another 64GB of RAM, this time DDR4 ECC. The price of which even with the best deals would be £600. The CPU itself I could get for £699 and then £350 for a motherboard and another £50-£70 on a good cooler.
So altogether a 1950X upgrade was £1719 before shipping. Quite a sizeable chunk of change. So I looked at some alternatives and I found on ebay some E5-2667v2 processors going for £250 each including shipping costs. So £500 for two processors + £102 on nice Noctua Coolers and I was set for just £602! - That's almost 1/3rd the price of a 1950X upgrade and nets me similar performance in both Single Thread and Multithreaded workloads!
These E5-2667v2's offer 3.3-4GHz clock speeds, Ivy Bridge-E architecture, 22nm, 25MB of cache each and 8 cores with 16 threads. They are the highest clock speed chip you can get for this socket with the 4GHz turbo.
They have an all-core turbo of 3.6GHz, a 4 core turbo of 3.7GHz and so on from there to 1 core @ 4GHz turbo. I figured since I'll be using two processors I may get some extra turbo performance out and hit those 3.7-4GHz turbo rates more often which has been true.
Since I saved so much money on the processor upgrade I decided to spend out on two 10Gb ethernet adapters. I settled on the X540-T2 from Intel they are really quite nice adapters from 2012, 13.5 Watts and have two 10Gb ethernet ports on an x8 PCIe 2.0 interface. I got one for £80 and the other for £160 they are genuine Intel cards and not Chinese replicas.
So thought I'd show some photos and things.
The above photo was taken while troubleshooting some issues, this motherboard has been really stable but it is finicky. In the above photo I only had two RAM sticks installed (one per processor) to diagnose why it wasn't posting. Long story short, this motherboard needed the CPU's to be swapped over and the CMOS cleared multiple times before it would work correctly but now it does post every time without issue. I'd like to thank Shadow_X from the ServeTheHome IRC Channel for helping with troubleshooting on this motherboard, he was really helpful.
I did end up going with the Noctua NH-U9DX i4 coolers because I knew I'd be buying a second cooler anyway and the SuperMicro cooler I had been using was always a bit loud and while cooling okay, was not amazing. The Noctuas are absoloutely a dream, not only are they very easy to fit (I had to fit and remove them about 5 times during troubleshooting) but they are really quiet and move an enourmous amount of air.
Below I've included a screenshot of HWMonitor which shows the CPU temperatures over the past two days that the server has been running.
As you can see the first CPU hasn't even broken past 56c even with all the stress testing and benchmarking I've been doing. The second CPU is a little hotter as the hot air from the first one is blowing straight into it but that too hasn't gone past 65c under the same load scenarios. Quite impressive. Under normal loading they're sitting around 40c which is 20c lower than my E5-1650 sat under the same load scenarios. That CPU would sometimes hit 80c!
Now of course the task manager looks a lot more impressive with 32 threads:
I do have 8 x 8GB setup in Quad Channel on each processor. Making it an Octo channel setup total. This has provided a huge performance uplift in memory throughput.
I think it will actually hit 104GB/s but I had a lot of stuff running at the same time as I ran this particular memory read benchmark from AIDA64. Still, 94.2GB/s is incredibly fast.
So in conclusion, I've managed to give my old server another 3 years of life by recycling some used E5-2667v2's - These CPU's originally retailed for $2050 USD in 2013 but I snagged them for just $336 each including shipping. I've been absolutely blown away by their performance, my own code is built to be extremely parallel and I have been able to get a 181% increase in performance from my previous E5-1650 which has been a big boon to my business needs.
I am still waiting on my second 10Gb network card to come, I intend to team the ports for a 20Gb performance increase between this server and my desktop where I'll be running cluster experiments and other interesting things like that.
I hope you all found this post interesting and agree with me that it warrants necro-ing my thread!
Recently I have been hitting up against the performance limits of the E5-1650 I installed in this system in August 2014. I wrote an Inference Engine, a type of machine learning system and it depends heavily on processing internet data so it cannot really be accelerated by GPU's - It needs those brawny x86 cores.
So I considered my options. I could purchase an AMD 1950X which has ECC memory support and 16 cores. According to the benchmarks, quite a nice processor. But I'd need to purchase another 64GB of RAM, this time DDR4 ECC. The price of which even with the best deals would be £600. The CPU itself I could get for £699 and then £350 for a motherboard and another £50-£70 on a good cooler.
So altogether a 1950X upgrade was £1719 before shipping. Quite a sizeable chunk of change. So I looked at some alternatives and I found on ebay some E5-2667v2 processors going for £250 each including shipping costs. So £500 for two processors + £102 on nice Noctua Coolers and I was set for just £602! - That's almost 1/3rd the price of a 1950X upgrade and nets me similar performance in both Single Thread and Multithreaded workloads!
These E5-2667v2's offer 3.3-4GHz clock speeds, Ivy Bridge-E architecture, 22nm, 25MB of cache each and 8 cores with 16 threads. They are the highest clock speed chip you can get for this socket with the 4GHz turbo.
They have an all-core turbo of 3.6GHz, a 4 core turbo of 3.7GHz and so on from there to 1 core @ 4GHz turbo. I figured since I'll be using two processors I may get some extra turbo performance out and hit those 3.7-4GHz turbo rates more often which has been true.
Since I saved so much money on the processor upgrade I decided to spend out on two 10Gb ethernet adapters. I settled on the X540-T2 from Intel they are really quite nice adapters from 2012, 13.5 Watts and have two 10Gb ethernet ports on an x8 PCIe 2.0 interface. I got one for £80 and the other for £160 they are genuine Intel cards and not Chinese replicas.
So thought I'd show some photos and things.
The above photo was taken while troubleshooting some issues, this motherboard has been really stable but it is finicky. In the above photo I only had two RAM sticks installed (one per processor) to diagnose why it wasn't posting. Long story short, this motherboard needed the CPU's to be swapped over and the CMOS cleared multiple times before it would work correctly but now it does post every time without issue. I'd like to thank Shadow_X from the ServeTheHome IRC Channel for helping with troubleshooting on this motherboard, he was really helpful.
I did end up going with the Noctua NH-U9DX i4 coolers because I knew I'd be buying a second cooler anyway and the SuperMicro cooler I had been using was always a bit loud and while cooling okay, was not amazing. The Noctuas are absoloutely a dream, not only are they very easy to fit (I had to fit and remove them about 5 times during troubleshooting) but they are really quiet and move an enourmous amount of air.
Below I've included a screenshot of HWMonitor which shows the CPU temperatures over the past two days that the server has been running.
As you can see the first CPU hasn't even broken past 56c even with all the stress testing and benchmarking I've been doing. The second CPU is a little hotter as the hot air from the first one is blowing straight into it but that too hasn't gone past 65c under the same load scenarios. Quite impressive. Under normal loading they're sitting around 40c which is 20c lower than my E5-1650 sat under the same load scenarios. That CPU would sometimes hit 80c!
Now of course the task manager looks a lot more impressive with 32 threads:
I do have 8 x 8GB setup in Quad Channel on each processor. Making it an Octo channel setup total. This has provided a huge performance uplift in memory throughput.
I think it will actually hit 104GB/s but I had a lot of stuff running at the same time as I ran this particular memory read benchmark from AIDA64. Still, 94.2GB/s is incredibly fast.
So in conclusion, I've managed to give my old server another 3 years of life by recycling some used E5-2667v2's - These CPU's originally retailed for $2050 USD in 2013 but I snagged them for just $336 each including shipping. I've been absolutely blown away by their performance, my own code is built to be extremely parallel and I have been able to get a 181% increase in performance from my previous E5-1650 which has been a big boon to my business needs.
I am still waiting on my second 10Gb network card to come, I intend to team the ports for a 20Gb performance increase between this server and my desktop where I'll be running cluster experiments and other interesting things like that.
I hope you all found this post interesting and agree with me that it warrants necro-ing my thread!
