Quanta LB6M (10GbE) -- Discussion

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

av00va

Member
Dec 10, 2015
53
5
8
34
LB6M was the loudest thing in my rack. If you dig through this thread there are several people who have replaced the fans with ~20dBA units, so you can quiet it down if you're willing to fork out another $30-$50 and break out a screwdriver.

I replaced my Quantas with Ubiquiti US-16-XGs. Barely audible and ~110W less power consumption than the LB6M in my use.
I can see people replacing the fans. Spending $900 more [/i ] on a switch with the same or less features...not worf. The power constipation is definitely worf

Sent from my SM-N910V using Tapatalk

 

tbg

New Member
Dec 12, 2016
8
5
3
48
Thanks for the info, I ordered a LB6M, I guess ill see how it goes. If it is to loud I'll swap the fans like others have.
 

webeindustry

New Member
Dec 14, 2016
1
0
1
41
Hello. I am still novice with networking, but am looking to implement an advanced setup for production eventually. I will have a 5 node cluster with 2 lb6m's for intra-cluster private networking. I am looking to maximize bandwidth but prioritize failover/high-availability while still keeping to load-balancing.

It appears a combination of protocols such as VRRP + LAG (or LACP?) would be best here?

The layout would be independent 10gbe NICs on each node 1 interface per switch.

Does this make sense and is this doable with LB6M? I basically would like the same for my public facing switches, two LB4M, but the drop is just 1gbps so perhaps that could simplify.

Big picture is pure N + 1 system for SaaS. All gear having A +B for dual circuits finishes this off.
 

acmcool

Banned
Jun 23, 2015
610
76
28
40
Woodbury,MN
Anyone willing to share working layer 3 config on this switch with removing the ip's..I am just not able to get it working.
Appreciate the help..
 

av00va

Member
Dec 10, 2015
53
5
8
34
Anyone willing to share working layer 3 config on this switch with removing the ip's..I am just not able to get it working.
Appreciate the help..
I have been unable to ping the switch. I cannot set an ip for ssh, or intervlan routing. II've tried the guides but it always states 0.0.0.0 for the routing table and not the appropriate subnet for the configured ip. Show network is also blank with 0.0.0.0 as the network
 

jmoll

Member
Jan 1, 2017
32
5
8
42
Hey, Thnx for answering loads of questions already with this thread. I am not too much in to the networking and my knowledge is limited especially with the 10gbe. However I do have the need for speed in my small home/office setup. Does anyone have tips how to optimize/fix the 10gbe sfp+ performance with lb6m. I have few computers connected having Mellanox ConnectX-2 Ethernet Adapter. sfp+ DACables are 3 meter ciscos. I have seen people getting much better iperf results ? I am going to have a ssd raid 0 nas in the network and as the speeds are at the moment I am not really getting what I was expecting from 10gbe network.

iperf3:

Code:
C:\iperf>iperf3.exe -c 192.168.xxx.xxx
Connecting to host 192.168.xxx.xxx, port 5201
[  4] local 192.168.xxx.xxx port 51687 connected to 192.168.xxx.xxx port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   543 MBytes  4.54 Gbits/sec
[  4]   1.00-2.00   sec   632 MBytes  5.31 Gbits/sec
[  4]   2.00-3.00   sec   637 MBytes  5.34 Gbits/sec
[  4]   3.00-4.00   sec   581 MBytes  4.87 Gbits/sec
[  4]   4.00-5.00   sec   565 MBytes  4.75 Gbits/sec
[  4]   5.00-6.00   sec   521 MBytes  4.37 Gbits/sec
[  4]   6.00-7.00   sec   593 MBytes  4.98 Gbits/sec
[  4]   7.00-8.00   sec   626 MBytes  5.25 Gbits/sec
[  4]   8.00-9.00   sec   534 MBytes  4.48 Gbits/sec
[  4]   9.00-10.00  sec   578 MBytes  4.85 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  5.67 GBytes  4.87 Gbits/sec                  sender
[  4]   0.00-10.00  sec  5.67 GBytes  4.87 Gbits/sec                  receiver

iperf Done.
thnx
 

jmoll

Member
Jan 1, 2017
32
5
8
42
Also I am not completely sure if the "low performance" is just because of the packet size... since the performance seems to go up with larger transfers. I don't yet have my NAS running. I wanted to first make sure things work before I throw in the cash for the ssds... I can cap the single ssd I/O speed with the network. Just the iperf results got me worried
 

segfault

New Member
Apr 5, 2016
29
13
3
43
hasno.info
Also,



Yup, just as you said my friend. No 1Gbit SFP transcievers known to work as yet. Seems like we are out of luck here. My advice to all here is bests to just get a lb4m, or a 8 port Gigabit SFP switch, Linksys has a few, then daisy chain one port and be done with the searching right?



You know, I remember reading alittle while back where it was debatable if the Fast path software supported Layer 3. Someone said of course it does, but I think from reading the manual and checking out the command line, it seems to not be supported. I'm honestly not sure. I could be wrong, but I don't think so from what I've read.
I have L3 going. It looks like it only support IGP, I'm using OSPF. I have it playing nice with a Cisco 3750X. OSPF routing with VRRP for the vlan gateways.

I also have MST setup between the 3750x and the lb6m.


Excerpts from the running config:
Code:
!Current Configuration:
!
!System Description "Quanta LB6M, 1.2.0.14, Linux 2.6.21.7"
!System Software Version "1.2.0.14"
!System Up Time          "13 days 7 hrs 3 mins 56 secs"
!Additional Packages     FASTPATH QOS
!Current SNTP Synchronized Time: Jan  4 01:02:52 201 UTC
!
network protocol none
network parms 192.168.20.1 255.255.255.0 192.168.20.1

vlan database
vlan 2,10,20,30
vlan name 2 "SERVER-BACKUP"
vlan name 10 "WIFI"
vlan name 20 "VOIP"
vlan name 30 "UNTRUSTED"
set igmp 1
set igmp 10
set igmp 20
set mld 1

vlan routing 1 1
vlan routing 2 2
vlan routing 10 10
vlan routing 20 20
vlan routing 30 30
exit
ip ssh server enable
ip ssh protocol 2
sshcon maxsessions 3
no ip telnet server enable

sntp client mode unicast
sntp client port 123
! sntp server status is Success
sntp server "192.168.15.3"

clock summer-time recurring 2 sun Mar 02:00 2 sun Nov 02:00 zone EDT
clock timezone -4 minutes 0 zone EST

ip domain name "example.com"

ip name server 192.168.15.2
logging host "192.168.15.33" ipv4 5514 info
logging syslog
logging syslog port 5514
logging cli-command
ip routing
ip route 0.0.0.0 0.0.0.0 192.168.24.250 10
ip helper enable
aaa authentication enable "enableList" enable
line console
exit
line telnet
exit
line ssh
exit

spanning-tree configuration name "C4-7B-7D-B2-A6-E9"
spanning-tree bpduguard
spanning-tree bpdufilter default
spanning-tree mst priority 0 4096
spanning-tree mst instance 1
spanning-tree mst instance 10
spanning-tree mst instance 20
spanning-tree mst instance 30

port-channel system priority 128
port-channel name 1/1 whee-a-lagg
port-channel linktrap "1/1"
interface 0/26
addport 1/1
exit
interface 0/27
addport 1/1
exit
interface 0/28
addport 1/1
exit

snmp-server sysname "my-lb6m"
!
set igmp
set mld
ip dhcp snooping vlan 1-4093
no ip dhcp snooping verify mac-address
ip arp inspection vlan 2-30

interface 1/1
description 'a simple lagg to another switch'
no port-channel static
ip dhcp snooping trust
ip arp inspection trust
mtu 9000
vlan participation include 2,10,20,30
exit

interface 2/1
! this is a vlan interface.
routing
ip address 192.168.10.4 255.255.255.0
ip vrrp 1
ip vrrp 1 mode
ip vrrp 1 ip 192.168.10.1
ip vrrp 1 accept-mode
ip vrrp 1 authentication simple "example"
ip ospf area 0.0.0.0
ip ospf authentication Simple example
ip ospf mtu-ignore
exit

interface 2/2
description 'SERVERS-backup-vlan'
routing
ip address 192.168.24.3 255.255.255.0
ip vrrp 2
ip vrrp 2 mode
ip vrrp 2 ip 192.168.24.1
ip vrrp 2 accept-mode
ip vrrp 2 authentication simple "example"
ip ospf area 0.0.0.0
ip ospf authentication Simple example
ip ospf mtu-ignore
exit

router ospf
router-id 1.2.3.4
network 192.168.15.0 255.255.255.0 area 0.0.0.0
network 192.168.24.0 255.255.255.0 area 0.0.0.0
network 192.168.25.0 255.255.255.0 area 0.0.0.0
network 192.168.26.0 255.255.255.0 area 0.0.0.0
redistribute connected subnets
redistribute static
exit
ip vrrp
exit
Here's an example from the switch's route table:
Code:
(lb6m) #show ip route

Route Codes: R - RIP Derived, O - OSPF Derived, C - Connected, S - Static
       B - BGP Derived, IA - OSPF Inter Area
       E1 - OSPF External Type 1, E2 - OSPF External Type 2
       N1 - OSPF NSSA External Type 1, N2 - OSPF NSSA External Type 2

S      0.0.0.0/0 [10/0] via 192.168.24.250,   13d:07h:18m,  2/1
O E1   10.100.100.1/32 [110/30] via 192.168.24.10,   11d:16h:33m,  2/1
                            via 192.168.24.11,   11d:16h:33m,  2/1
                            via 192.168.24.12,   11d:16h:33m,  2/1
C      192.168.25.0/24 [0/1] directly connected,   2/1
O      192.168.150.0/24 [110/20] via 192.168.25.20,   03d:20h:01m,  2/1
C      192.168.24.0/24 [0/1] directly connected,   2/2
I've changed bits here and there to obfuscate my own setup, but the gist should hopefully come across.
I have tested failover with this setup and it works.
 

segfault

New Member
Apr 5, 2016
29
13
3
43
hasno.info
Ah, I see. Segfault, walk me through how you use it if you don't mind? What do you use Layer 3 for? Is it to route dhcp? I never finished my full networking course!
It all started with wanting to get a feel for ip anycast setups. I started with DNS anycast, which involves running some routing protocol on a host to advertise ip's if a service is up. If the service goes down or the box disappears then the route disappears and clients really don't notice.

I then moved on to other services like ntp and eventually setup a backup vlan using the 1gb links on the machines. At this point I have a vip advertised by each host which defaults to using the 10gb interface but will fallback to the 1gb link in case of a failure or a switch reboot.

I have freenas setup to run like this and I'm pretty happy with it so far.

I also have a failover route to the internet which is advertised by my ASA with an ip sla. So all of the lan hits one gateway to the net but if my primary gateway fails I get a transparent switch over to the slower backup internet link.


More recently I've setup a VyOS box on my ESX machine to advertise routes for the VMs I setup. It allows me to play with more complex network setups which is pretty fun. Eventually I plan to setup pfSense or SophosUTM as a VM to handle my guest wifi network's traffic.

TLDR: I really wanted to set things up so that I could reboot machines and switches without major impact to the consumers of my little network.
 
Last edited:
  • Like
Reactions: Sleyk and aero

Terry Kennedy

Well-Known Member
Jun 25, 2015
1,140
594
113
New York City
www.glaver.org
You know, I remember reading alittle while back where it was debatable if the Fast path software supported Layer 3. Someone said of course it does, but I think from reading the manual and checking out the command line, it seems to not be supported. I'm honestly not sure. I could be wrong, but I don't think so from what I've read.
It depends what features they put into the build. Here's FASTPATH on a different brand of switch:
Code:
switch#show tech   

 ***************** Show Version ******************

Switch: 1

System Description............................. Powerconnect 8024, 5.1.10.1,
                                                VxWorks 6.6
Machine Description............................ Dell Ethernet Switch
Machine Type................................... Powerconnect 8024
Machine Model.................................. PC8024
Serial Number.................................. CN0Y295K2829829xxxxxxxx
FRU Number.....................................
Part Number.................................... BCM56820
Maintenance Level.............................. A
Manufacturer................................... 0xbc00
Burned In MAC Address.......................... D067.xxxx.xxxx
Software Version............................... 5.1.10.1
Operating System............................... VxWorks 6.6
Network Processing Device...................... BCM56820_B0
Additional Packages............................ FASTPATH QOS
                                                FASTPATH Multicast
                                                FASTPATH Stacking
                                                FASTPATH Routing
                                                FASTPATH Data center
 

jmoll

Member
Jan 1, 2017
32
5
8
42
Hi just dropping in to say it is all working now, Thnx !

I did get the the performance up by adjusting switch MTU to 9014 and NIC's on the workstations to 9000. (and by rebooting everything.)

I was already starting to suspect that the throughput of the pci-e slot wasn't enough or on those there is something wrong... with drivers etc. but apparently it is now all good! So I will continue on with my NAS build.

Code:
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  1.12 GBytes  9.61 Gbits/sec
[  4]   1.00-2.00   sec  1.13 GBytes  9.74 Gbits/sec
[  4]   2.00-3.00   sec  1.09 GBytes  9.33 Gbits/sec
[  4]   3.00-4.00   sec  1.11 GBytes  9.50 Gbits/sec
[  4]   4.00-5.00   sec  1.12 GBytes  9.64 Gbits/sec
[  4]   5.00-6.00   sec  1.12 GBytes  9.62 Gbits/sec
[  4]   6.00-7.00   sec  1.10 GBytes  9.47 Gbits/sec
[  4]   7.00-8.00   sec  1.11 GBytes  9.54 Gbits/sec
[  4]   8.00-9.00   sec  1.11 GBytes  9.51 Gbits/sec
[  4]   9.00-10.00  sec  1.10 GBytes  9.44 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  11.1 GBytes  9.54 Gbits/sec                  sender
[  4]   0.00-10.00  sec  11.1 GBytes  9.54 Gbits/sec                  receiver

iperf Done.
 

jmoll

Member
Jan 1, 2017
32
5
8
42
Also I want to add one thing if someone is having similar "problems". In the beginning I didn't know how to saturate the 10gbe connection with iperf. So if you think you are having a bad performance try to adjust the TCP window size with the -w flag.
 

Metrotyranno

New Member
Jan 4, 2017
1
0
1
28
Hi,

We are trying to make 1Gbit optics work but the physical mode of the interface is set to 10G, has anyone managed to get it down to 1000M / 1G ?
 

c6100

Member
Oct 22, 2013
163
1
18
USA
Hi,

We are trying to make 1Gbit optics work but the physical mode of the interface is set to 10G, has anyone managed to get it down to 1000M / 1G ?
I never got it working and in the manual it specifically says you can't set the speed on the SFP interfaces. I ended up using another switch because of this.
 

jmoll

Member
Jan 1, 2017
32
5
8
42
Here is my silent mod! I destroyed my lb6m today :D

Front to back airflow and this stays cool and silent. Holes became bit too large but I was too lazy to go and buy correct size hole saw. Maybe I will fill the caps with something when I put the fan grills in. I need to sit with this thing in the same room and now we are starting to get along... It goes in to the spacious rack cabinet which will help also. Reminds me off old VHS tape.

modded lb6m.jpg
 

aero

Active Member
Apr 27, 2016
346
86
28
54
Here is my silent mod! I destroyed my lb6m today :D

Front to back airflow and this stays cool and silent. Holes became bit too large but I was too lazy to go and buy correct size hole saw. Maybe I will fill the caps with something when I put the fan grills in. I need to sit with this thing in the same room and now we are starting to get along... It goes in to the spacious rack cabinet which will help also. Reminds me off old VHS tape.

View attachment 4107
Nice mod! What fans did you go with?
 

jmoll

Member
Jan 1, 2017
32
5
8
42
Nice mod! What fans did you go with?
I cannot recommend the fans I bought. These are the cheapest stuff I found. for sure these are cooling enough but more premium products would probably last longer and make it even more silent. I only modded one psu.

I did put 2x these in the PSU and also 1 at the back of the switch all blowing out. It is propably almost obsolete.
(in the PSU the wires have to be reconfigured.)
Silent Fan 4010 - 4,000 rpm | Silent Fan Series | Fans | Cooltek

These are at the top blowing in, PWM fans so the temperature measurement works with the "show environment"

GlideStream 120 PWM SC: CPU Kühler, Lüfter, Lüftersteuerung, PC Netzteile von Scythe

it runs at 35c with the load I can throw at it
 

jmoll

Member
Jan 1, 2017
32
5
8
42
I cannot recommend the fans I bought. These are the cheapest stuff I found. for sure these are cooling enough but more premium products would probably last longer and make it even more silent. I only modded one psu.

I did put 2x these in the PSU and also 1 at the back of the switch all blowing out. It is propably almost obsolete.
(in the PSU the wires have to be reconfigured.)
Silent Fan 4010 - 4,000 rpm | Silent Fan Series | Fans | Cooltek

These are at the top blowing in, PWM fans so the temperature measurement works with the "show environment"

GlideStream 120 PWM SC: CPU Kühler, Lüfter, Lüftersteuerung, PC Netzteile von Scythe

it runs at 35c with the load I can throw at it
I am quite sure that the small fan at the back only does cause harm since it cannot handle the airflow big boys are pushing in. I think I might still get rid of that and mod the fan backplate so that it allows more air to flow through. however it does work fine at the moment
 

jtroberts

New Member
Jan 5, 2017
8
2
3
51
Below you can find a snippet of a lb6m running Indigo on 2.6.25-bcm-ntsw

I had tried all the ports and they seem to work but as you can seem there are som inconsistencies as port 1 is port 49 and port 9 is port 2, port 10 is port 3 ....

for those playing at home: (you need to have u-boot, if you don't update to it)
Code:
1.) stop uboot
2.)
setenv netmask switch_netmask
setenv ipaddr switch_ip
tftpboot 0x8000000 tftp_server:00_firmware
bootm 0x8000000
I uploaded the file here:
File-Upload.net - 00_firmware.zip
as it's too big to upload to the forum
@Sorin Have you made any progress with alternative firmware for the LB6M? I did find a PDF from QSSC that shows the unit has the BCM 56820 chip. That chip on Broadcom's site states that it does support 1G, 2.5G and 10G. This makes me assume that the switch can support a GLC-T transceiver but it isn't enabled. This is my ultimate goal and I think the goal of many others. PS: can't test yet unit arrives next week but I can certainly help out.

EDIT: Also has anyone tried Cumulus OS? I've seen rumors about it running on the LB6M and LB8. thoughts? Is this something that we can load via uboot?

Link to QSSC PDF Link to BCM 56820
 
Last edited: