You are going to think I am insane but...

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Falloutboy

Member
Oct 23, 2011
221
23
18
I am planning on building my own 4 port 40 Gb Infiniband, 2 Port 10 Gb SFP+, 1 Port 10 GbE Switch after watching an article on how to do so on the net, brilliantly cheap way of getting really high speed transfers between my soon to be Storage server and my two clients.

I brought
8 MHQH19B-XTR's for $120 which is 15$ each
and
8 [URL='https://www.ebay.com/itm/Mellanox-MC2206310-005-5M-Infiniband-Fiber-Optic-Cable-16-40-ft-F/202582001422?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2060353.m2749.l2649']MC2206310-005 for $360 which is
[/URL]45$ each - I didn't expect the cables to be cheap even 2nd hand.
 

mb300sd

Active Member
Aug 1, 2016
204
80
28
34
You're gonna need lots of CPU for it to work anywhere near line speed. 40GbE will bottleneck at the CPU regularly even for a single card.
 

Falloutboy

Member
Oct 23, 2011
221
23
18
Yup - which is why I will be looking round for a 2nd hand dual or quad processor board of the server variety - no one said I had thought this through quite literally a case of doing it to see if I can make it happen.
 

kapone

Well-Known Member
May 23, 2015
1,096
643
113
Two 6610-24s, stacked together (assuming 4x 40gb ports are an absolute requirement) will give you way more ports (10gb and 1gb), throughput, bandwidth, features, reliability... at a fraction of the cost.
 
  • Like
Reactions: BoredSysadmin

Rand__

Well-Known Member
Mar 6, 2014
6,638
1,769
113
Or a cheap sx6012/18/36 converted + qsfp->sfp+ -> RJ45 adapter ;)
For sfp+ either get more qsfp->sfp+ adapters or use a 4:1 split cable
 

Falloutboy

Member
Oct 23, 2011
221
23
18
Or a cheap sx6012/18/36 converted + qsfp->sfp+ -> RJ45 adapter ;)
For sfp+ either get more qsfp->sfp+ adapters or use a 4:1 split cable
if an sx6012 is a switch then where is the fun in that, the idea is to make one and learn something in the process even if that something is ... dont do this
 

Evan

Well-Known Member
Jan 6, 2016
3,344
598
113
DPDK compatible hardware and use :)
But yes your crazy ! Crazy can be fun
 

Falloutboy

Member
Oct 23, 2011
221
23
18
DPDK compatible hardware and use :)
But yes your crazy ! Crazy can be fun
That DPDK looks interesting I'll have to look into that, the whole setup will be going into one of these for the moment.

XIGMATEK

I'm tossing up as to if I should my 16 drive raid in there as well in 4 to 3 converter bays.
 

Rand__

Well-Known Member
Mar 6, 2014
6,638
1,769
113
if an sx6012 is a switch then where is the fun in that, the idea is to make one and learn something in the process even if that something is ... dont do this
Ah you're doing that for fun - o/c then knock yourself out;)

I thought it was for saving money;)


investing any money in infiniband in 2019 is insanity
Nooo don't say that ... I still hope that it will come back (to ESXi or something that can be used as storage backend for it)