The real answer PnoT is Xpenology (if your ok with setting up your own hardware).
As many cpu cores as you want (I'm running an AMD A8, so 8 cores total).
as much RAM as a standard mobo can hold.
DSM 5.2 = Samba 4.1.16 included.
I'm running a 10Gig ConnectX-2 EN pcie card. So no max 120mb/sec limit.
It's been a LONG time since I've looked at that site and the hardware was really specific at that point but it sounds like that's not the case anymore. I'll give it a look for sure.
Will the IB ConnextX-2 work as well?