Connect Gigabit Switch and Router to Arista 40GB switch.

[K D]

How do you folks having 40G switch in your homelab manage this? I am moving from a 10G switch to a 40G switch and am not sure on how to manage this connection.

Current Layout : Using a router on a stick configuration with a Unifi USG. Only 2 VLANS applicable here. One for storage and one for everything else and there was no inter-vlan communication. So the 1G link to the USG was not a bottleneck.




New Requirements :

1. Need to connect 6 hosts with mellanox CX3 40G QSPF+ cards to Arista 7050
2. New Xeon D 1518 based pfsense box with 1G copper and 10G SFP+ connections.
3. ES24-LITE with 1G Copper and 1G SFP connections.

I want to use the ES24-LITE just for IPMI and possibly management traffic for hosts separated into their own VLAN. All other traffic via the 40G connections.

How do I make the physical connections to integrate the pfsense box and the 1G switch with teh 40G network?

 

[Patrick]

If you have SFP+ on the 1GbE switch, you can use a breakout cable.

 

[K D]

If you have SFP+ on the 1GbE switch, you can use a breakout cable.

2 questions :
It is an 1G SFP port. Will it work with an SFP+ module in a breakout cable?
How long are the "breakout" sections in a breakout cable usually? I have the pfsense ports facing the front and the switch facing the rear. So I'm wondering if I can use the same breakout cable to connect both.

 

[K D]

Will these work?

* NEW * Volex 2M Passive 40Gb QSFP+ to 4x SFP+ Breakout Cable - VAHS-30-0428-2M | eBay

 

[nkw]

Will these work?

* NEW * Volex 2M Passive 40Gb QSFP+ to 4x SFP+ Breakout Cable - VAHS-30-0428-2M | eBay

See: Arista DCS-7050QX-32

I was not able to get that Volex cable to work with the 7050QX. I was able to get the 10Gtek brand from Amazon to work.

 

[K D]

See: Arista DCS-7050QX-32

I was not able to get that Volex cable to work with the 7050QX. I was able to get the 10Gtek brand from Amazon to work.

Thanks. I should choose the arista compatible version for the 10GTek one from Amazon right? The link was to the Cisco version. Also Do you know how long the "breakout" is?

 

[nkw]

Thanks. I should choose the arista compatible version for the 10GTek one from Amazon right? The link was to the Cisco version. Also Do you know how long the "breakout" is?

I happened to buy the Cisco version (was an Amazon warehouse deal) and it works fine. I've ordered two more of the Arista coded versions, so I'll find out in a day or two if those work as well. With the one I received the breakout is pretty much right at the QSFP+ side so you have the max possible length of cable available to each SFP+ (mine was 2m).

It was mentioned in the other thread the fs.com ones work, only reason I haven't ordered those was the last couple of orders I did from fs.com the shipping took a little while more than Amazon prime.

 

[K D]

I happened to buy the Cisco version (was an Amazon warehouse deal) and it works fine. I've ordered two more of the Arista coded versions, so I'll find out in a day or two if those work as well. With the one I received the breakout is pretty much right at the QSFP+ side so you have the max possible length of cable available to each SFP+ (mine was 2m).

It was mentioned in the other thread the fs.com ones work, only reason I haven't ordered those was the last couple of orders I did from fs.com the shipping took a little while more than Amazon prime.

Awesome. I've ordered the 10GTek Arista version as well.

 

[Joshh]

2 questions :
It is an 1G SFP port. Will it work with an SFP+ module in a breakout cable?
How long are the "breakout" sections in a breakout cable usually? I have the pfsense ports facing the front and the switch facing the rear. So I'm wondering if I can use the same breakout cable to connect both.

All the breakout cables I have used have are split pretty much at the QSFP port.

 

[K D]

I should get the breakout cable delivered on Wednesday. Will check it out after that. Of it doesn't work I'll probably get one of the Mikrotik switches with an SFP+ port.

 

[i386]

@K D
Which arista switch do you have?
Models with the "s" suffix in the name have sfp/sfp+ports that can be used with a sfp to 1gbase-t transceiver to connect to a router/modem from your isp.

 

[K D]

@K D
Which arista switch do you have?
Models with the "s" suffix in the name have sfp/sfp+ports that can be used with a sfp to 1gbase-t transceiver to connect to a router/modem from your isp.

DCS-7050QX-32

No SFP+

 

[_alex]

I don't know if this can work on the Arista, but it does with Mellanox Switches.

QSFP Port on Switch -> QSA -> 1Gb RJ45 SFP -> Rj45 Switch

If the Switch would negotiate 1G on a QSFP Port it could be worth a try.

 

[Stril]

Hi!

I am not sure, if Arista supports QSA. Are you sure about this?

 

[_alex]

As i said, i have no idea if Arista supports them.
On mlx it works to go 1 Gb RJ45 from QSFP this way.

 

[Joshh]

Hi!

I am not sure, if Arista supports QSA. Are you sure about this?

I have one plugged into mine and it works just fine.

 

[K D]

Excuse my ignorance. But what is a QSA?

 

[i386]

MAM1Q00A-QSA, it's a qsfp+ to sfp+ adapter:


Edit: More info in this post Using a 40GbE (QSFP+) NIC with a 10GbE Switch (SFP+)

 

[K D]

Thanks @i386

@nkw. I recieved the Arista version of the breakout cable just now and connected it.

The breakout starts almost at the transceiver so each sfp+ cable is almost 6ft.

I see the connection leds blinking in the pfsense box but not in the 1g switch. The Arista switch has one of the 4 leds on for that particular port ( see pics). The management terminal is hooked up to the patch panel and connected to my workstation on the second floor. I am not able to check the terminal immediately. ( On s side note I really need to have a system or laptop downstairs so that I dont have to trudge up and down 2 floors each time)

Sent from my iPhone using Tapatalk

 

[K D]

So. With the current setup now that I know I cannot connect the Arista directly to my switch:

All IPMI in a separate subnet with the 1G switch connected to my pfsense.

Only esxi management and freenas /napp-it will need access to ipmi.

Can I have the pfsense box perform routing between the 2 subnets ? Don't care about bandwidth here as it is just IPMI connection.






Sent from my iPhone using Tapatalk