Cheap 10GBE FCoE networking + FC interconnect switch?

[BigJim]

I'd like to experiment with 10GB ethernet file and FCoE over DAC, and having a couple of fiber-channel tape drives on the same network.

It seems I need Converged Network Architecture (CNA) host adapters on my workstations and file server, and getting hold of these is straightforward (QLOGIC QLE8152 adapters are around $25 on eBay), and then using DAC cables between them and the switch. So far so good.

But I'm getting a bit lost about what switch(es) I need to hook up my tape drives (which have 2, 4 and 8GB fiber SFP ports) to the network.

10GB CNA switches with SFP+ ports are fairly cheap, but it's not clear to me if I can just plug in fiber transceivers into these and run fiber to my tape drives, or if I need something different.

Some CNA switches have dedicated fiber ports (Dell Brocade M8428-k Converged 24 x 10GbE Ports Switch - Ref | eBay) so I'm assuming not.

But I see the term "Data Center Bridging" (DCB) in connection with all this, and I see DCB FCoE switches with only 1/10GB SFP+ ports (HP 5920AF-24XG 24-Port 10GbE SFP+ FCoE / DCB Switch JG296A | eBay).

Any help appreciated, even if it's a link to some document or vid that helps explain this!

 

[BigJim]

OK, I've done some more research. Seems I need a fabric interconnect switch that also has 10GBE and FCOE.

Second hand switches on eBay aren't too bad

VAT INC EMC MP-8000B - 32 Active Ports - 24x 10G + 8x 8G San Switch | eBay

<p dir="ltr">EMC MP-8000B - 32 Active Ports - 24x 10G + 8x 8G San Switch. Condition is Seller refurbished. Dispatched with Royal Mail 1st Class. <br> EMC MP-8000B <br></p> <p dir="ltr">Fabric - Enhanced Group Management - FCoE Base License </p> <p dir="ltr">Ports on Demand License - additional 8...

www.ebay.co.uk

Cisco UCS 6120XP N10-S6100 20-Port Fabric Interconnect Switch | eBay

Find many great new & used options and get the best deals for Cisco UCS 6120XP N10-S6100 20-Port Fabric Interconnect Switch at the best online prices at eBay! Free delivery for many products!

www.ebay.co.uk

but they're quite power-hungry and noisy.

I don't need anywhere near that number of ports; 4 x 10GBE and 4 x FC would be fine.

I considered building one myself, but this thread:

https://forums.servethehome.com/index.php?threads/for-those-who-wants-to-build-their-own-10gb-switch.11939/

suggests the performance would be poor.

Any ideas?

 

[EffrafaxOfWug]

I could very well be wrong, but I was under the impression that FCoE is very much found only on datacentre-level switches; I don't think I've ever seen it on SOHO/homelab-grade networking equipment.

 

[BigJim]

You could well be right

But if anyone know different, please give us your input.

 

[oddball]

We run a lot of Cisco UCS gear, all supports FCoE. If you want straight fiber channel look at the Cisco MDS line. Older switches are practically free on eBay.

Cisco's whole mantra is converged network. Their NIC's are software defined CNA's, that connect to a fabric interconnect (software defined switch) etc. With the FI's you can change the characteristics of the ports on the fly.

If you don't want to go the Cisco route Juniper QFX switches support FCoE out of the box as well. It's a lot cheaper to pick up used Cisco gear vs a used QFX5100 or something, but if you can find a deal on Juniper go for it.

Arista doesn't support FCoE, I believe some Mellanox switches might, but it's hit or miss. You're going to have to go Cisco or Juniper.

 

[BigJim]

Thanks for the reply.

I want to talk to fiber-attached tape drives over the 10GBE network using FCoE, so - in addition to having FCoE capability - unless I'm mistaken the switch also needs to "unbundle" the fiber frames out of the Ethernet frames and then transmit/receive them on their own. Only a few switches seem to offer this, usually via their own dedicated 2/4/8G fiber ports (like the Brocade and Cisco 6100 switches I linked to above).

I see Cisco offers a switch (6332-16UP) that has "universal" SFP+ ports that each support both FC and Ethernet/FCoE, but this is unusual (and still very expensive). Brocade also do a converged HBA (BR0110401-38) that has SFP+ ports that will transmit 10GBE and 4/8/16G FC, so clearly if switch manufacturers wanted to incorporate this functionality in more of their offerings they could.

But, as far as I can determine, only a few switches seem to have both ethernet protocol SFP+ and FC protocol SFP ports.

 

[oddball]

My company has a pair of 6332-16UP's, and yes, you can do what you're asking. They're great machines. We are managing a UCS chassis plus some C-series servers with them. They connect upstream with a pair of Arista switches, all in a DC (so noise doesn't matter).

The 6332 or 6120 isn't really a switch per se. There's also the 6248's and the port monster the 6296.

These are pseudo switch/controllers for the Cisco UCS system. They're the devices that servers and blade chassis plug into that coordinate services profiles and networking before passing traffic north to the upstream switches.

The 6248 has 16GB of RAM and a Xeon, the 6332 has a beefy Xeon and 32GB of RAM with a 64GB flash. It's a server with a switch fabric. There is a GUI (USC Manager) in addition to a command line that you use to manage everything. You can reassign port profiles by clicking.

There is a switch mode, although I've never used it. From some docs I pulled up it looks like switch mode is what you'd want.

Note, these things are LOUD and the 6248's use a lot of power. You could conceivable replace the fans with silent ones. The power budget is based on ports used, so if you're using a few ports it'd be fine.

Where are you? I have 3x 6248's plus some FEX in my garage gathering dust. I'm in W PA, if you're close I'd be willing to give it away if you want to pick it up or meet up. If you are somewhat nearby I can ship, I'd need money for a box/shipping. Not trying to turn this into a sale thread, but it's awesome hardware and it's just taking up space. I also have a bunch of Cisco 4G/8G FC transceivers that I have no use for.

 

[petree77]

FCoE doesn't require any special switches, it's sole requirement is lossless ethernet. This means larger buffers and Flow Control are required on any switch you connect FCoE devices to. If all of your devices are straight FCoE then nothing special is required except for lossless ethernet. If you need to connect straight FC devices as well, that will require some sort of network device that can speak both and "convert" the traffic.

 

[BigJim]

My company has a pair of 6332-16UP's, and yes, you can do what you're asking. They're great machines. We are managing a UCS chassis plus some C-series servers with them. They connect upstream with a pair of Arista switches, all in a DC (so noise doesn't matter).

The 6332 or 6120 isn't really a switch per se. There's also the 6248's and the port monster the 6296.

These are pseudo switch/controllers for the Cisco UCS system. They're the devices that servers and blade chassis plug into that coordinate services profiles and networking before passing traffic north to the upstream switches.

The 6248 has 16GB of RAM and a Xeon, the 6332 has a beefy Xeon and 32GB of RAM with a 64GB flash. It's a server with a switch fabric. There is a GUI (USC Manager) in addition to a command line that you use to manage everything. You can reassign port profiles by clicking.

There is a switch mode, although I've never used it. From some docs I pulled up it looks like switch mode is what you'd want.

Note, these things are LOUD and the 6248's use a lot of power. You could conceivable replace the fans with silent ones. The power budget is based on ports used, so if you're using a few ports it'd be fine.

Where are you? I have 3x 6248's plus some FEX in my garage gathering dust. I'm in W PA, if you're close I'd be willing to give it away if you want to pick it up or meet up. If you are somewhat nearby I can ship, I'd need money for a box/shipping. Not trying to turn this into a sale thread, but it's awesome hardware and it's just taking up space. I also have a bunch of Cisco 4G/8G FC transceivers that I have no use for.

Thanks, that's a very kind offer. But I am in the UK, so I imagine the shipping would cost more than I can purchase them here locally. Do your 6248s have the N10-E0060 FC modules in them?

Have you ever pulled a 6248 or 6296 apart? They look ENORMOUS for what they do.