Machine learning system

[zer0sum]

I would love to get some opinions on a "budget" system I am looking to build so that I can get experience with machine learning in the field of malware analysis.
When it's not busy doing ML, it will also be used as a virtual machine host and will likely do some crypto mining

This is what I have right now for a single CPU system:

• 2 x GTX 1080Ti - $1449
• 1 Intel Xeon E5 2620V3 - $463
  
• 1 x SUPERMICRO X10SRL F - $281
  
• 1 x Kingston KVR24R17D8K4/64, PC4-2400 Registered ECC DIMM Kit (4 x 16 GB) - $570
  
• 4 Crucial MX300 SSD's 525 GB - $574
• 2 x HGST Deskstar NAS 4 TB HDD's - $124
• 1 x EVGA SuperNOVA 1000W PSU - $144
• 1 x Chenbro RM42300 rack mountable 4U - $120
  
• 1 x Supermicro heat sink 4U - $38

TOTAL - $3763

I plan to run the SSD's in raid 10 off the motherboard controller, and use them for most workloads etc.
The HDD's will be used for any large VM's or file storage.

IPMI is definitely important to me otherwise I might lean towards an X99 system.
The cost to move to a dual CPU setup isn't crazy when taking into account the full system cost, but I'm not sure if there is value there for this purpose?

Questions I guess are if I should be looking at a different CPU or faster memory?

 

[CookiesLikeWhoa]

What OS you running?

What are the graphics cards for?

A lot of this hardware doesn't seem to fit together unless you're also planning on gaming on it as well.

 

[zer0sum]

What OS you running?

What are the graphics cards for?

A lot of this hardware doesn't seem to fit together unless you're also planning on gaming on it as well.

The OS will most likely be Ubuntu
Graphics cards are for machine learning and crypto coin mining...no gaming at all

 

[Patriot]

What OS you running?

What are the graphics cards for?

A lot of this hardware doesn't seem to fit together unless you're also planning on gaming on it as well.

Most machine learning libraries run on cuda.
There is no packed math outside the top P100 chip so no reason to go titan or quadro over Ti.

Have you picked your library yet?
Honestly would start with 1 card till you figure out how to use all of it.

Make sure you get a g2 or g3 supernova as they are superflower based and the g1 is garbage.

 

[CookiesLikeWhoa]

I understand that aspect of it, but there is much more specialized hardware for mining than GPU's. They do work very well, but draw a lot of energy compared to specialized equipment, making the profitability margin smaller or none at all depending on the cost of electricity where you live. Hence my question about gaming, because this server seems like a "jack of all trades" type of build.

It seems like it would make more sense to separate the two and allow them to excel at their respective aspects.

 

[Patriot]

I understand that aspect of it, but there is much more specialized hardware for mining than GPU's. They do work very well, but draw a lot of energy compared to specialized equipment, making the profitability margin smaller or none at all depending on the cost of electricity where you live. Hence my question about gaming, because this server seems like a "jack of all trades" type of build.

It seems like it would make more sense to separate the two and allow them to excel at their respective aspects.

You are missing the deep learning aspect of it.

 

[CookiesLikeWhoa]

Possibly, but if just for learning, pick up a used GTX480 or like you said one card, that way when things are sorted out the money can be spent more effectively.

The only reason I say this is because I went down the same path the op is going down for mining. Started out with 3 power house GPU's and quickly (read expensively) found out it was not a good idea.

 

[zer0sum]

Possibly, but if just for learning, pick up a used GTX480 or like you said one card, that way when things are sorted out the money can be spent more effectively.

The only reason I say this is because I went down the same path the op is going down for mining. Started out with 3 power house GPU's and quickly (read expensively) found out it was not a good idea.

I don't want to turn this into a thread on GPU mining, as I need help on the Machine learning side of things mostly.

GPU mining is still profitable, even with expensive power, and I'm lucky that this is going into a DC with zero energy costs for me personally

A single GTX 1080ti uses about 280W fully loaded so it would be 560W for 2 cards.
The most profitable alt-coin to mine right now on Nvidia is ZEC and a 1080ti can get ~700+ sol/s = 1400+ sol/s for both cards
Which means you can mine ~ 0.08649 ZEC/day or $4.81 per card for $9 a day from this system

If you add the rest of the system you are drawing ~750W and even paying 30 cents/kw you would still be making $4/day

 

[CookiesLikeWhoa]

Agreed don't mean to derail the thread, I just don't want any one to really make the same mistakes I made doing mining.

To your original questions:
For the CPU I would try to nab an ES or OEM processor off Ebay for the same money you're spending on the 2620.
The 2675 V3 is a 16 core processor @1.8Ghz. While not the fastest out there, it does make up for it in the core count. Cheaper than the 2620 as well.
Intel Xeon E5-2675 v3 OEM CPU 1.8GHz 16-Core SR1XM Close to E5-2698v3 E5-2683 v4 | eBay

For the memory you can find better deals as well. I personally avoid Kingston and always go with Samsung, but that's my personal preference.
SAMSUNG M393A2G40DB0-CPB 16GB (1X16GB) 2RX4 PC4-2133P MEMORY | eBay
You can grab 4 of those for less than the kingston set. They're on Supermicro's compatibility list. Reg ECC DIMMS.

The SSD's I can't really argue for that price you can do much better. Just remember those are consumer grade SSD's, they will wear out faster than your server grade SSD's and don't have power loss protection.

 

[Patriot]

I can help a bit on the machine learning/deep learning front. Everything is iterative and based on what you are trying to learn.
I would suggest for inspiration ... Sirajology Youtuber that teaches 5 minute Neural nets.
Anyone Can Learn To Code an LSTM-RNN in Python (Part 1: RNN) - i am trask

Number of resources to get you started.

I have moved away from the m2070/2090s and k10s... The K10s and K40 are my playground and the K80s do the work.

 

[Patrick]

I think @Patriot is spot on on the single GPU recommendation.

Also, get only one CPU not two even with two GPUs. You do not want to deal with NUMA.

I hear there is a site that has a few guide on this: Optimizing a Starter CUDA Machine Learning / AI / Deep Learning Build

That type of build (sub E5 for Xeon D) will last you many months. At that time, we will have new GPUs.

When you are just starting, you will not need much by way of GPU power. 1060 6GB is the sweet spot still.

 

[RobertFontaine]

NUMA can be fun... The question is do you want to learn Machine Learning or do you want to learn heterogenous multi-procesor programming for supercomputer environments?

Things get very cool/ugly very quick. CUDA is the most mature but not open development standard and is leveraged by the most Machine Learning libraries... BUT Some have been ported to Xeon Phi and Intel CPU and AMD as well.... Much of the time memory shuffling kills any upside of pushing processing between hardware but if you can group your tasks and memory access efficiently (wizard magic here) then you can make these things quaintly called supercomputer humm.

Intel is headed down the FPGA rabbit hole for machine learning and Google has gone past FPGA and has developed their own ASICs purpose built for dnn/covnets.

I wish I was 20 again.

 

[zer0sum]

Thank you for all of the great replies so far!!

Is there any merit in looking at a Ryzen system instead of Intel right now?
I know I would lose IPMI

 

[Patrick]

Is there any merit in looking at a Ryzen system instead of Intel right now?
I know I would lose IPMI

Good question. If you have one GPU, do not care about higher-speed networking (40GbE/ FDR and up), IPMI, are ok with only 64GB RAM, and do not mind occasional software crashes, then it is very good.

Also, remember Intel has been pushing its MKLs out.

 

[TLN]

Looking at the build, I'd probably go with
=Different CPU. ES or QS from ebay will be way cheaper.
=Single videocard. Something mid-end. Whatevern I can get for $200. I can always spend 1.5 grand on 1080Ti later.
=Less SSD. I'd personally start with one SSD, you can add it later.
=Good PSU. Not sure how good is that EVGA though.
=All core BIOS hack for Haswell. Because why not? If you're able to provide good cooling (you'll need it for videocard anyway).

If you're going with hypervisor - I'm not sure about IPMI. You only need it once - to install Hypervisor. If something goes wrong - you only have 1 host, not 1000s. That means you can get away with X99 motherboard.

With mid-level videocard you'll be around $1000-1200 for 14 cores @ 3.0Ghz (2683v3 unlocked).

 

[zer0sum]

Good question. If you have one GPU, do not care about higher-speed networking (40GbE/ FDR and up), IPMI, are ok with only 64GB RAM, and do not mind occasional software crashes, then it is very good.

Also, remember Intel has been pushing its MKLs out.

Ohhhh K then...I think I'll stick to Intel. Haha

Looking at the build, I'd probably go with
=Different CPU. ES or QS from ebay will be way cheaper.
=Single videocard. Something mid-end. Whatevern I can get for $200. I can always spend 1.5 grand on 1080Ti later.
=Less SSD. I'd personally start with one SSD, you can add it later.
=Good PSU. Not sure how good is that EVGA though.
=All core BIOS hack for Haswell. Because why not? If you're able to provide good cooling (you'll need it for videocard anyway).

If you're going with hypervisor - I'm not sure about IPMI. You only need it once - to install Hypervisor. If something goes wrong - you only have 1 host, not 1000s. That means you can get away with X99 motherboard.

With mid-level videocard you'll be around $1000-1200 for 14 cores @ 3.0Ghz (2683v3 unlocked).

I'd love to get some more information on the all core bios hack for Haswell

Hmm...an Ebay CPU could be a good option, but I'm buying this through my work so I'm not sure how much my expense department will love my making purchases of ES/QS CPU's on Ebay

I prefer to make less purchases due to said expenses department, so I wanted to stick with 2 x 1080ti, and 4 x SSD's and then I don't have to bug them later. Seems trivial but I've learnt not to rock the boat too much over the years

I'm definitely going to research and go with a better PSU as you're the 2nd person that's mentioned it.

Is the 2683v3 the current value leader when thinking of a single CPU system?
I do want to make sure there is still fairly strong single core frequency, although more cores and loads of cache is always nice!

I realized that case is a short industrial style one, so I'll probably go with a simple Rosewell 4U setup.

Thank you for your thought out response.

 

[TLN]

If you're buying for the company via department - that's a different story.
CPU's on ebay are ES or QS and are not meant for sale and not supported by Intel. But it works for me and for many other people. There's a big topic on that on that forum. 2683 with 14 cores will cost you around $300-350. I'm not sure if your department will approve something like that
All core turbo will allow you to run all cores at Turbo Frequency. You'll need very good cooler for that and PSU. I'm not sure if your department will approve something like that as well

 

[BackupProphet]

For training, I'm more limited by the performance of a single thread than the number of cores. We buy Intel I7-7700K today for new engineers and 1080TI. A single GPU is enough.