Some pics of the fan shrouds printed in PETG and used on both my switches. They are attached to the existing tray and completely removable, they can also be flipped. I'm pretty happy with the results.
Quanta LB6M (10GbE) -- Discussion
- Thread starter T_Minus
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Just want to let everyone know that after you flash the brocade os you can use the super cheap Cisco FET-10G transceiver. They can be had for under $3 each.
I use emulex Oce10100,11100,and 14100 cards and they also support the Cisco FET-10G optics.
I use emulex Oce10100,11100,and 14100 cards and they also support the Cisco FET-10G optics.
Hi, I see that you turned on tagging on 10-11. Maybe it is silly question but did you enable tagging on servers nics? If so ok, if not turn off tagging on 10-11 to untagged ports. It should work. I have router on stick plug to one port @LB6M and doing packet filtering/routing between vlans. This is the only tagged port (in both vlans). The rest is untagged but ports participate in vlans.
Hey Kirk, just wondering, how are those fans working out for you? I have some Noctua's in mine, but they barely move enough air and I'm afraid to leave it on during the day in the corner of my closet lol. Current low-load temp is about 65C. I know the shutdown on the TurboIron has it set at 90C, but that seems really high to me. Thanks in advance!
FWIW my LB6M (running Brocade firmware) has been running at 55C (night) to 70C (daytime) for the past 6 months, including throughout the whole (British) summer.
I went with a single Noctua 12mm fan option, mounted in the top panel, right above the main CPU. Dead silent at all times. Pictured here a few months ago, when I'd first done the mod:
The switch does feel rather hot to the touch, eg when I pull an SFP+ it's pretty warm. I wouldn't do this in production.. but it's going fine at home
That said, my load could be called "extra low", as I only have 4 x 10G ports populated + 4 x 1G, and I'm barely pushing anything through it.Regardless, I took it at its word when it said 80C was warning and 90C was shutdown, and figured that as long as I kept it below 75C it'd be fine. And it has been.
Last edited:
This was already answered somewhere in the thread but I'll tell you again.
The Quanta was sold as a generic "brandless" (not entirely true, but whatever) 10G switch to very big and well known companies like Amazon and Google for their datacenters.
They would buy tons of them and asked for modifications in the firmware (or did it themselves) to remove part they didn't intent to use anyway, like the GUI.
So yes, the GUI indeed exist, but as an option. And this option was not very popular. That's why you can see it in the Quanta manuals, but isn't installed in the models we can have for a cheap price on ebay or other places.
Hello all,
I have an LB6M thats working great. However I have a new requirement, I need to block traffic within VLAN -
for example if my VLAN 100 has IP range = 10.10.10.0/24, none of the IP within that should be able to communicate with each other (looking at multi-tenant environment).
Is there a way to implement this? I applied the rules on firewall but seems that the traffic never hits the firewall since it could be on the same Virtual environment (different VMs).
I have an LB6M thats working great. However I have a new requirement, I need to block traffic within VLAN -
for example if my VLAN 100 has IP range = 10.10.10.0/24, none of the IP within that should be able to communicate with each other (looking at multi-tenant environment).
Is there a way to implement this? I applied the rules on firewall but seems that the traffic never hits the firewall since it could be on the same Virtual environment (different VMs).
I am by no means experienced in this sort of advanced config, or indeed advanced switch config in general. But your question interested me so I went through the Brocade documentation.
I believe what you want should be achievable using rule-based ACLs. This is definitely supported on a Brocade-flashed LB6M, and all of the following info is taken from Brocade documentation and tested on my own Brocade-flashed LB6M.
I can't say for absolute certain that the same is available on the default FastPath LB6M firmware, however I checked my Quanta LB4M running FastPath routing FW (version 5.13.12.14) and found the same ACL commands were supported there (access-list and ip access-group), albeit with slightly different syntax.
Reading the Brocade documentation file TurboIron24X_08001_ConfigGuide.pdf, starting page 897:
- Types of IP ACLs
You can configure the following types of IP ACLs:- Standard - Permits or denies packets based on source IP address. Valid standard ACL IDs are 1 - 99 or a character string.
- Extended - Permits or denies packets based on source and destination IP address and also based on IP protocol information. Valid extended ACL IDs are a number from 100 - 199 or a character string
- Standard - Permits or denies packets based on source IP address. Valid standard ACL IDs are 1 - 99 or a character string.
The latter, Extended ACLs, supporting denying packets based on both source and destination IP, sounds like what you want? Specifically you'd block packets from 10.10.10.0/24 to 10.10.10.0/24 - though presumably also with a rule specifically allowing access to any router/gateway on that subnet.
The full Brocade docs are provided on the web page that details the Brocade flash, which is documented here in this thread. The best doc to read is the TurboIron ConfigGuide I just mentioned, as the other files are general docs for all FastIron switches, rather than specifically listing everything supported (or not) on the TurboIron - and therefore also supported on a Brocade-flashed LB6M.
I tested this myself on my Brocade-flashed LB6M, and it seems to work:
Code:
vlan 100 name vlan100 by port
untagged ethe 2 ethe 24
spanning-tree
!
access-list 100 bridged-routed
access-list 100 permit ip 192.168.200.0 0.0.0.255 host 192.168.200.10
access-list 100 deny ip 192.168.200.0 0.0.0.255 192.168.200.0 0.0.0.255
access-list 100 permit ip any any
!
interface ethernet 24
port-name 10Gdesktop2
ip access-group 100 in- Port 24, configured with 192.168.200.20, could ping 192.168.200.10 (which is connected to port 2)
- But it could not access any other IP on 192.168.200.0/24
- It could access any other subnets it was configured with, eg port 24 configured with 192.168.210.20 could ping 192.168.210.12 on port 2.
I was only able to apply the ip access-group 100 to port 24 once port 24 was in a VLAN. I wasn't previously using VLANs on this switch so I created one to test it out. That might be because I'm missing some other config that would enable per-port access-group without a VLAN. Although in your case you want it on a VLAN anyway. (Note that this did not apply on the LB4M, where ip access-group was available on any port, regardless of it already being in a VLAN.)
One final point: according to the Brocade docs, on our switch ACLs can only be applied "inbound" on a port, not outbound. Inbound means "from the NIC connected to the given port", ie you can ACL restrict traffic that comes from a NIC to a specified port, but not traffic from elsewhere that will exit out of that port (and end up at the NIC.) When I checked my LB4M, I found the same limitation there - it will only apply an access-group for "in".
I don't believe that affects what you want to do, given you want to block all traffic between 10.10.10.0/24, so it doesn't matter whether you block it at source or destination. Just bear in mind you need to write the ACL such that it blocks traffic from each NIC (in to its switch port), not traffic destined for a given NIC (which will exit out from its switch port.)
So from what I've read and can understand, I believe rule-based ACLs can achieve what you want? If you're running Brocade FW - as is highly recommend - then the above should work for you directly. If you're still on FastPath LB6M FW, I expect the same basic commands will exist (access-list and ip access-group) but might use slightly different syntax. FastPath might also have minor differences in implementation compared to Brocade, h0wever it seems my LB4M supports the same basic concepts as the Brocade-flashed LB6M, so I'd be quite surprised if a FastPath LB6M was much different.
That said, if you are on the FastPath FW, maybe this would make a good opportunity to flash to Brocade FW
It seems to be superior in most every way, and is also far better documented - a really big benefi in my view.
Last edited:
PS. Of course this ACL stuff requires that the traffic actually hits the switch. You said:
I don't know what virtualisation software you're using, but I'd expect most decent software to also provide virtualised networking. VMWare ESX does for example, where you can create a virtualised switch to which your VMs connect and apply restrictions there like you would on a real switch. That can include advanced ACLs (here's an example, a blog post describing traffic filtering rules in a vSphere Virtual Distributed Switch.)
So if all your multi-tenants are on the same virtualisation platform, it might be you can do what you want entirely with network virtualisation and don't need config on the physical LB6M (except perhaps as a second line of defence.)
You'd need to be sure that the traffic between VMs is actually going out through the physical NIC(s) to the switch, and is not being routed internally inside the VM software. Otherwise no config on the switch could help.
I don't know what virtualisation software you're using, but I'd expect most decent software to also provide virtualised networking. VMWare ESX does for example, where you can create a virtualised switch to which your VMs connect and apply restrictions there like you would on a real switch. That can include advanced ACLs (here's an example, a blog post describing traffic filtering rules in a vSphere Virtual Distributed Switch.)
So if all your multi-tenants are on the same virtualisation platform, it might be you can do what you want entirely with network virtualisation and don't need config on the physical LB6M (except perhaps as a second line of defence.)
Hello everyone,
I still consider myself relatively new to networking, despite what I've been dabbling in thus far between my home setup and our small business's configuration. Primarily one of my friends helped get me into the used enterprise gear market for various needs, and to be quite honest I really wouldn't want to give that up!
So my experience with networking has mostly consisted of running my own networking cable throughout the home and office, and planning out where I need to go with it. Of course setting up the patch panels and connecting everything to the network switch. My first managed network switch is my Cisco units, then the Blade switches.
Since I'm finally getting ready to explore 10gbe networking, I've explored my options and had stumbled upon this thread which interested me into getting the Quanta LB6M, which sounded like the perfect solution.
However, I feel this is where I'm getting myself sent out into the battlefield, so to speak. My whole system is also my learning tool, but there's a lot I'm not quite understanding and/or I may not be configuring correctly.
I have read through quite a few pages, but not all the pages of this thread since there's so much to take in, on top of everyone's needs/uses are going to vary.
So I'm going to start with listing what I have currently both at home and our business.
Home:
Servers:
Office:
Servers:
So at home, I'm utilizing dual WAN on the PfSense Router since we have two ISPs on hand, a faster connection from a local ISP and a DSL connection for backup.
Currently on LAN from there goes to the first interface port on my Cisco Catalyst 4948. Then everything else connects to this same switch. I've even managed to configure my FreeNAS box to utilize LACP with this recently. Though I did that through the GUI that Cisco provides for management.
Otherwise I have little to no experience with CLI for the network switches, though probably only the basics like maybe just assigning VLANs.
Right now I'm not using VLANs to separate anything within the network. Primarily right now I just have the FreeNAS be configured for us to have our own shares, plus a separate one for the media. Which that is accessed by Emby running on Ubuntu under a VM on my Dell R710. I plan to run more on ESXi later.
Though the primary goal with the Quanta LB6M is to upgrade actual servers to utilizing 10GbE. I want to get the ESXi and FreeNAS box on 10GbE. Then possibly have the LB6M provide my computer and one or both of mom's computers 10GbE as well.
Effectively the same/similar idea is to be done at the office as well, where the LB6M will provide the server and some office workstations 10GbE. At the office I originally ran shielded Cat6A cabling for preparing for 10GbE, but after seeing the costs of SFP+ modules that'd provide 10GBASE-T or 10GBASE-T switches I figured the LB6M would provide the best value. I don't mind running OM3 fiber cable.
So that brings me to what I'm working with right now. So far only one OM3 cable arrived for me to try the LB6M out with. I stuck an Intel X520-DA2 into my R710 and connected the OM3 cable between the two. Both ends using the Cisco FET-10G 10-2566-02 transceivers.
Then I plugged one of the 1GbE ports on the Quanta into my Cisco switch and the serial console cable to a laptop. Thus far I've managed to see that the Quanta shows to show a link on both, and identifying the link as up on both interface 0/1 and 0/27. "Show network" even gives me a DHCP given IP address within the console. Though I'm not sure what else to really do here.
ESXi on the R710 recognizes that I've added the Intel X520-DA2, and and with the one transceiver plugged in it also shows up as an interface I can add. Though it shows the link to be down. Despite the LED on the card showing some form of activity along with the indication on the Quanta. Thus no IP received from the DHCP server either.
The other OM3 cables should arrive today so I should be able to expand my test platform to being able to tap off the Blade G8000.
A lot of how-tos I've been reading around keep showing various VLAN configurations on this switch, which I hadn't planned to be using on this switch. Which might be a no-no, I'm not sure as I'm trying to learn how configure my whole network setup. So far with the 1GbE side, it's relatively easy since it's been plug-n-play experience or a few minor tweaks within the Cisco GUI to take care of what I needed it to. Of course, if the Quanta was that easy, it wouldn't be a good learning tool for me, now would it?
So what do I really need to do with the Quanta LB6M as far as configuration? I've never configured SFP+ interfaces before, so I don't know if there are any special requirements needed to get it to work with another component in a system.
The idea is to use the Blade G8000's SFP+ interfaces with the Quanta so the 1GbE switch has high speed traffic to the 10GbE switch that will handle the higher speed traffic for the servers and certain machines.
I've read the article that STH has on using the LB6M, but I'm not really understanding what OSPF is, or other options, or certain configuration with what's going on there, since it looks like it's going on with different subnets.
Anyway, thank you guys in advance for your time, I hope I've provided enough information to at least get the idea out on what I'm wanting to do here.
I still consider myself relatively new to networking, despite what I've been dabbling in thus far between my home setup and our small business's configuration. Primarily one of my friends helped get me into the used enterprise gear market for various needs, and to be quite honest I really wouldn't want to give that up!
So my experience with networking has mostly consisted of running my own networking cable throughout the home and office, and planning out where I need to go with it. Of course setting up the patch panels and connecting everything to the network switch. My first managed network switch is my Cisco units, then the Blade switches.
Since I'm finally getting ready to explore 10gbe networking, I've explored my options and had stumbled upon this thread which interested me into getting the Quanta LB6M, which sounded like the perfect solution.
However, I feel this is where I'm getting myself sent out into the battlefield, so to speak. My whole system is also my learning tool, but there's a lot I'm not quite understanding and/or I may not be configuring correctly.
I have read through quite a few pages, but not all the pages of this thread since there's so much to take in, on top of everyone's needs/uses are going to vary.
So I'm going to start with listing what I have currently both at home and our business.
Home:
Servers:
- 1u Supermicro Mini ITX Xeon-D Server - PfSense Router (This hardware was originally purchased to be used as a FreeNAS server, but plans changed for this as soon as I got my hands on a nice 2u chassis.)
- 2u Dell PowerEdge R710 - ESXi
- 2u Supermicro 12 x 3.5" bay chassis - FreeNAS (Primary)
- 2u Supermicro 12 x 3.5" bay chassis - Spare / Future Expansion
- 1u Dell PowerEdge R410 - FreeNAS (Secondary Backup - For more important data that's stored on the primary FreeNAS)
- Cisco Catalyst 4948 (Currently the primary switch)
- Cisco SG300-10PP (PoE switch for the IP cameras and two Ubiquiti APs)
- Blade G8000 with two SFP+ modules (Not in use right now.)
- Quanta LB6M (Not in use right now.)
Office:
Servers:
- 2u Dell PowerEdge R720xd - ESXi
- 1u Dell PowerEdge R410 - PfSense Router
- Blade G8000 with two SFP+ modules (Primary)
- Quanta LB6M (Not in use right now.)
So at home, I'm utilizing dual WAN on the PfSense Router since we have two ISPs on hand, a faster connection from a local ISP and a DSL connection for backup.
Currently on LAN from there goes to the first interface port on my Cisco Catalyst 4948. Then everything else connects to this same switch. I've even managed to configure my FreeNAS box to utilize LACP with this recently. Though I did that through the GUI that Cisco provides for management.
Otherwise I have little to no experience with CLI for the network switches, though probably only the basics like maybe just assigning VLANs.
Right now I'm not using VLANs to separate anything within the network. Primarily right now I just have the FreeNAS be configured for us to have our own shares, plus a separate one for the media. Which that is accessed by Emby running on Ubuntu under a VM on my Dell R710. I plan to run more on ESXi later.
Though the primary goal with the Quanta LB6M is to upgrade actual servers to utilizing 10GbE. I want to get the ESXi and FreeNAS box on 10GbE. Then possibly have the LB6M provide my computer and one or both of mom's computers 10GbE as well.
Effectively the same/similar idea is to be done at the office as well, where the LB6M will provide the server and some office workstations 10GbE. At the office I originally ran shielded Cat6A cabling for preparing for 10GbE, but after seeing the costs of SFP+ modules that'd provide 10GBASE-T or 10GBASE-T switches I figured the LB6M would provide the best value. I don't mind running OM3 fiber cable.
So that brings me to what I'm working with right now. So far only one OM3 cable arrived for me to try the LB6M out with. I stuck an Intel X520-DA2 into my R710 and connected the OM3 cable between the two. Both ends using the Cisco FET-10G 10-2566-02 transceivers.
Then I plugged one of the 1GbE ports on the Quanta into my Cisco switch and the serial console cable to a laptop. Thus far I've managed to see that the Quanta shows to show a link on both, and identifying the link as up on both interface 0/1 and 0/27. "Show network" even gives me a DHCP given IP address within the console. Though I'm not sure what else to really do here.
ESXi on the R710 recognizes that I've added the Intel X520-DA2, and and with the one transceiver plugged in it also shows up as an interface I can add. Though it shows the link to be down. Despite the LED on the card showing some form of activity along with the indication on the Quanta. Thus no IP received from the DHCP server either.
The other OM3 cables should arrive today so I should be able to expand my test platform to being able to tap off the Blade G8000.
A lot of how-tos I've been reading around keep showing various VLAN configurations on this switch, which I hadn't planned to be using on this switch. Which might be a no-no, I'm not sure as I'm trying to learn how configure my whole network setup. So far with the 1GbE side, it's relatively easy since it's been plug-n-play experience or a few minor tweaks within the Cisco GUI to take care of what I needed it to. Of course, if the Quanta was that easy, it wouldn't be a good learning tool for me, now would it?
So what do I really need to do with the Quanta LB6M as far as configuration? I've never configured SFP+ interfaces before, so I don't know if there are any special requirements needed to get it to work with another component in a system.
The idea is to use the Blade G8000's SFP+ interfaces with the Quanta so the 1GbE switch has high speed traffic to the 10GbE switch that will handle the higher speed traffic for the servers and certain machines.
I've read the article that STH has on using the LB6M, but I'm not really understanding what OSPF is, or other options, or certain configuration with what's going on there, since it looks like it's going on with different subnets.
Anyway, thank you guys in advance for your time, I hope I've provided enough information to at least get the idea out on what I'm wanting to do here.
Alright, so to update the situation at hand... this confirms my exact statement. More specifically with the SFP+ interfaces.
It turns out the Quanta LB6M's configuration wasn't the issue, in fact it's because of the transceiver compatibility on the X520-DA2 adapters. The Cisco ones they came with... well just doesn't work. I ended up testing the card in my Windows 7 machine and event viewer said it was an unsupported SFP+ module. So that explains why I'm having issues with my test.
The good news is, my IBM/Blade and Cisco transceivers all seem to be compatible with the Quanta LB6M just fine. I ended up firing up my Blade G8000 and hooking the Quanta up to it via SFP+, then a laptop up to one of the 1GbE ports. Everything simply just worked.
So right now I'm going to plan on getting the correct SFP+ modules for the X520-DA2s, until then I can't really move on with anything.
Yo Sleyk, thanks for the welcome!
They were certainly on the Intel X520-DA2 card, I just didn't catch on until I check it out on my Windows 7 rig.
Actually the Cisco transceivers work perfectly fine with my LB6M as-is, since I'm getting fully functional networking between it and my Blade G8000 with them. The real issue was just the X520-DA2 cards being picky.
I've read about the Mellanox cards being used, but I wasn't sure what models to look into. I certainly would have no problems considering them, especially in other machines that really wouldn't need dual SFP+, especially considering the price points.
With this being my first giant leap into network advancements, I've got quite a bit to learn from. Step one was figuring out transceiver compatibility with the Intel cards.
Which I did get some compatible transceivers ordered. One of which I'm trying from FS. I hope it arrives by time the weekend comes around. I know I'm getting some DACs in for sure. So if anything I can get my Dell R710 and Supermicro 12-bay chassis fully switched over to the LB6M. Especially when I get some final pieces to get everything finally switched over to the rack I got in today.
(It's not very fun working with machines simply stacked on top of each other... The more I keep adding, the more reason for a proper rack to be set in place.
)Hey guys thanks for all the info on this forum. I have read over and seem to have it working, but would like to have LACP working. My hopes are to have interface 0/1 and 0/2 to my unifi 48 switch. I see that LACP mode is already active on all ports, so that being said if I just plug in the ports on 0/1 and 0/2 will they automatically be ready? Aggregation is setup on the unifi switch already. ( I did read over page 12, but had a hard time understanding it, just looking for guidance). Other than that I have 3 servers and one PC to plug in that I don't need aggregation on at this time. Will LACP mode being enabled on those ports create an issue?
Edit: Think I may have got it, testing it out before I make the full switch to this switch. lol
show port all shows
1/1 Enable Up Disable N/A N/A
shouldn't LACP Mode here be Enable instead of N/A?
Edit: Think I may have got it, testing it out before I make the full switch to this switch. lol
show port all shows
1/1 Enable Up Disable N/A N/A
shouldn't LACP Mode here be Enable instead of N/A?
Last edited:
So I have a backup/test server connected, seems that when using jumbo frame on my main pc, I couldn't connect to the server. Disabling it fixed it. (Jumbo frames is on switch and all servers). Not sure why that happened. I'll play around with it for a bit see how it goes.