Stupid 10GB question

[TType85]

This is a stupid question but I just have to ask it.

If you get a hold of a few dual 10G (or even 40G cards) is there any software/os that will allow you to create a switch with them?

Say I have 3 Brocade 1020's (6 ports total) in a machine. and I have 6 machines with Brocade 1010's. Is there any way to make that machine with the 1020's a switch?

 

[TuxDude]

Should be fairly easy to use pfSense (or many other linux/bsd-based OSs) to create a bridge across all of the NIC ports, effectively turning it into a switch. Performance won't be as good as a real switch, but it would have all of the functionality of a layer 3 router, perhaps even more.

 

[TType85]

Should be fairly easy to use pfSense (or many other linux/bsd-based OSs) to create a bridge across all of the NIC ports, effectively turning it into a switch. Performance won't be as good as a real switch, but it would have all of the functionality of a layer 3 router, perhaps even more.

How bad of a performance drop would it be? None of the clients will be capable of anything above ~4-500MB/s sequential. I don't think i'd ever saturate the full 10GBx6 ports at one time.

 

[Entz]

I haven't scaled out that far but I do have a small bridge on my file server (only 2 ports on the switch, so this gives me a third). Doing a 2 port bridge in Ubuntu 14.0.4 LTS, Mellanox ConnectX-2 EN adapters and a E3-1230v1 I average around 7 Gbits/s (Host->Bridge->Switch->Host). Can peak as high as 9 -- depending on what else is hitting the box. If your CPU is strong enough, 4 Gbit across the 6 ports likely wont be an issue, but I have never tried it.

 

[TuxDude]

How well it would perform is an interesting question, and is perhaps an idea for Patrick for a new article for the site...

Patrick: How about an article describing how to build a software 10GbE switch for cheap using pfSense + multiple e-bay NICs, including some benchmarks of what kind of performance to expect. I am just guessing that you likely already have the hardware on hand to toss something together.

 

[Kristian]

How well it would perform is an interesting question, and is perhaps an idea for Patrick for a new article for the site...

Patrick: How about an article describing how to build a software 10GbE switch for cheap using pfSense + multiple e-bay NICs, including some benchmarks of what kind of performance to expect. I am just guessing that you likely already have the hardware on hand to toss something together.

+1

 

[SteveBagley]

Patrick: How about an article describing how to build a software 10GbE switch for cheap using pfSense + multiple e-bay NICs, including some benchmarks of what kind of performance to expect. I am just guessing that you likely already have the hardware on hand to toss something together.

That'd be an interesting article -- I'd also be worth comparing performance of pfSense with that of OpenVSwitch running on top of Linux as well (for pure switching purposes as well as routing).

Steve

 

[RTM]

Without having a specific piece of software in mind, I would think the best performance would be achievable using something like Intels DPDK or maybe something similar like pf_ring or netmap.

Oh, and a word of caution, the Brocade 1020 as mentioned by the OP, is not supported on FreeBSD (at least afaik), you would have to use something like ESXi to passthrough as an emulated nic.

EDIT: Seems like Intel has created a patch for openvswitch that makes use of the DPDK: Packet Processing | 01.org
The release note mentions that they support the Fortville controllers, you might be able to make a 8 port 10Gb switch using two QSFP 40 Gb ports on a nic .

 

[Chuckleb]

I'd be interested in figuring out how much CPU needed per port so that I can reduce noise and heat... Since it wouldn't have to be in a 1U chassis, I could use bigger fans.

 

[mervincm]

VYOS worked for me.
didnt have much luck using pfsense or Windows 2012r2

I used it as a VM within ESXi
CPU was hit hard

VyOS

here are my notes

login vyos:vyos
configure
edit int eth eth0
set bridge-group bridge br0
exit
edit int eth eth1
set bridge-group bridge br0
exit
set system host-name vyos1-10G
set system domain-name host.domain.ca
set system name-server 192.168.110.1
set system gateway-address 192.168.110.1
set int bridge 'br0'
set int bridge br0 address '192.168.110.19/24'
set service ssh port 22
commit
save

 

[TType85]

VYOS worked for me.
didnt have much luck using pfsense or Windows 2012r2

I used it as a VM within ESXi
CPU was hit hard

VyOS

here are my notes

login vyos:vyos
configure
edit int eth eth0
set bridge-group bridge br0
exit
edit int eth eth1
set bridge-group bridge br0
exit
set system host-name vyos1-10G
set system domain-name host.domain.ca
set system name-server 192.168.110.1
set system gateway-address 192.168.110.1
set int bridge 'br0'
set int bridge br0 address '192.168.110.19/24'
set service ssh port 22
commit
save

I'm going to have to give this a try.