Quanta LB6M (10GbE) -- Discussion
- Thread starter T_Minus
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Hi, I just received one this past week it seems to work good. I am running it in a VMWARE VSAN env and intergraded it into the network at the same time. My vmotions are screamingly fast but for datastorage transfers; they are very slower than a snail. Could the network traffic be creating the slow down. I had tested it with jumbo frames enable, and just came across with more VSAN issues. Does anyone have any thoughts in how to improve the bandwidth for the backside of the vm env datastorage transfers?
If you are doing vmotions of vm's on vsan storage to another compute node that is also running vsan then it's likely that the "vmotion" is only actually transferring memory contents from 1 node to the next (if the vm is up). The 2nd node already has the vm on it's vsan storage.
When you move a vm from one data store to another you are moving the whole shebang.
When you move a vm from one data store to another you are moving the whole shebang.
I wish that was it... but no one thing I found was the ups was being over loaded and making the switch reboot very 3 hours... well since that been resolved, I decide to trunk my vm hosts to a vSwitch. and vlan throttle the networks. I gave vSan 5gs to max 8gd with a burst for the whole switch. One things I learned for there this provisioning setting for cold storage moves (not vmotion) and I turned that on. Well with all that speed from a 10g switch.. and basically I am still not seeing any improvements at all on the back plane for the systems. I was thinking moving away from FC and using iSCSI for the backup NAS but I am not sure at this time I could recommend. Does VMWARE off any tools to diagnose the bottlenecks?
So, I've been digging through threads and I was able to figure out and set up ssh via DHCP on the LB6M (which I got off ebay for $260 all costs) on the management port.
What I'm currently trying to figure out is how I can set an ssh public key to login via ssh leveraging RSA key exchange. I've seen a few mentions here and there as well as on the cli that show that you can do something like this:
copy scp://user@host/path/to/key/file.pem nvram:sshkey-rsa2
However, what I'm curious about is if this is actually the ssh HOST key or the ssh User key. Is it possible to set up passwordless login to this switch?
Also, I haven't had the opportunity to check out a lot of features, but will share some of my automation scripts once I get smaller things (like the ssh conundrum) ironed out.
Also, all my efforts in turning up a way to update the firmware has basically gone the way of nothing. Has anyone tried to load an OpenSource 3rd party software and been successful?
What I'm currently trying to figure out is how I can set an ssh public key to login via ssh leveraging RSA key exchange. I've seen a few mentions here and there as well as on the cli that show that you can do something like this:
copy scp://user@host/path/to/key/file.pem nvram:sshkey-rsa2
However, what I'm curious about is if this is actually the ssh HOST key or the ssh User key. Is it possible to set up passwordless login to this switch?
Also, I haven't had the opportunity to check out a lot of features, but will share some of my automation scripts once I get smaller things (like the ssh conundrum) ironed out.
Also, all my efforts in turning up a way to update the firmware has basically gone the way of nothing. Has anyone tried to load an OpenSource 3rd party software and been successful?
I was considering the same switch for vSAN as well. Were you able to overcome the latency issues with it?
I have VSAN up and running with this switch, both VSAN 6.2 and VSAN 6.5. My setup appears to have the same feature set yours does:
Additional Packages............................ FASTPATH QOS
FASTPATH Routing
I enabled IGMP querier on my VSAN VLAN, and confirmed multicast functioned as expected using tcpdump-uw (vSphere 6.5 has a nifty VSAN tester in the GUI that will test multicast too)
That's awesome to hear it indeed works! If you don't mine, what type of performance are you seeing from vSAN in your proactive tests in vCenter?
Sent from my Nexus 6P using Tapatalk
[ESXHOST:~] tcpdump-uw -i vmk3 -n -s0 -t -c 20 udp port 12345 <<- Command to verify the vmKernel port
tcpdump-uw: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmk3, link-type EN10MB (Ethernet), capture size 262144 bytes
IP 10.1.0.34.17089 > 224.1.2.3.12345: UDP, length 200 <<-- responses from each f the hosts
IP 10.1.0.32.55200 > 224.1.2.3.12345: UDP, length 200
[ESXHOST:~] nc -uz 224.1.2.3 12345 <<--Command to generate some multicast traffic
Connection to 224.1.2.3 12345 port [udp/*] succeeded!
My issue currently is that I'm using an unsupported SAS controller, and only a single 10Gb NIC in each host, So when I apply a load, with other VLAN's on the same port i notice a spike in write latency, read latency is almost always in the 1ms or less range (I'm running an all flash setup), but i've seen write latency hit 100ms. The only place i really feel it is in a couple of VMs running docker containers that consume storage via NFS across that same port. I have 7 active VLAN's going in and out of the single port.
I have ordered additional multiport 10GbE adapters, but have not installed yet to see if the issue is resolved. Local disk latency isn't a problem, thats why I've focused on the single NIC port as the culprit. I've been able to measure up to 1GB/s (~8Gb/s) read and write into and out of the VSAN datastore, so sequential access (storage vMotion) can easily saturate the link.
BTW, this is just a lab setup.
Just a little more detail on the VSAN performance I'm seeing, I just modified my storage policy and am now resyncing ~.5TB. on each node, i'm getting between 300MB/s and 550MB/s through the VSAN VMK on each node(this is pretty much the limits of the disks i'm using, 1 cache drive and 1 capacity SSD per node), and between 15,000 and 30,000 IOPS on each node (4 nodes). So this switch, I can safely say is NOT a bottleneck for a VSAN setup.
