Hello,
Long time lurker here, I finally got an ICX6610 thanks to
@fohdeesha !
I got it from g-electronics on ebay, like some people here, and I've issues with the 40Gbps breakout ports. It does seem to be the exact same issue as
@N1h1l1ty on page 50 of this topic, as I have those messages on console at boot:
Code:
Parsing Config Data ...
------------------------------------------------------------------
M:9 L:0 - chow_qsfp_read, qsfp 3, error in seting up mux
------------------------------------------------------------------
M:9 L:0 - link_40G_4x10G_get_media: qsfp 3, port 1/2/7 error in reading qsfp
chow_40G_4x10G_get_media: error in reading qsfp 1/2/7
It was working for one of my servers (ESXi) but I could not make a second port work, even from the other QSFP port. Tested with two cables (some Cisco generic on eBay and a Brocade one from FiberStore), same result. All firmwares (bootloader, firmware and PoE firmware) are up-to-date.
Now with the exact same NIC (HP NC530SFP+) on an other server (FreeNAS), it doesn't work at all. It appears up on the interfaces brief, but goes down as soon as I tag it with a VLAN for some reason. It also goes up and down randomly without any vlan tagged (but traffic does not flows anyway, even when switch says it's up).
So I guess my QSFP module is broken ?
Honestly I don't think I'll ever need more than 8 SFP+ ports (all licensed, again thanks to
@fohdeesha), but it's kind of a bummer to see it broken. I don't really want to send the switch back as everything is now cabled and I live in France, so shipping to USA would be really tedious.
Is the module replaceable easily ? Is there a way to purchase it alone for cheap ?
Or am I missing something obvious ?
Thanks a lot
Hello,
Sooooo... I received my new switch, from a different seller, reconfigured it and... Same behavior.
The error after Parsing Config Data is gone, but I tried:
- Breakout to HP 530SFP+ on FreeNAS: Link goes up/down like crazy on the logs, no traffic flows
- Breakout to HP 530SFP+ on ESXI 6.7: Link is up until I tag it with a VLAN, then it goes down and won't come up until I reboot the switch
- Breakout to Chelsio T320 on FreeNAS:
Seems to work fine with VLANs (nope, see below)
I'm confused. When I plug the HP NIC on front SFP+ ports, it works perfectly so it's definitely working fine, but for some reason seems to crash with the breakout ports.
I might end up using DAC on front ports to get those working, but with two QSFP breakout cables that's a shame, I have four servers and could have done 2x10Gbps on each...
EDIT: Tried a second port into the Chelsio card, same behaviour than with HP NIC. Looks like I can't connect more than one SFP+ port from the breakout...
If anyone has an idea, I'll take it.
EDIT2: Ok so I plugged every four of the SFP+ ports, and the error came back:
Code:
Parsing Config Data ...
------------------------------------------------------------------
M:9 L:0 - chow_qsfp_read, qsfp 3, error in seting up mux
------------------------------------------------------------------
M:9 L:0 - link_40G_4x10G_get_media: qsfp 3, port 1/2/2 error in reading qsfp
chow_40G_4x10G_get_media: error in reading qsfp 1/2/2
It's a different switch from a different seller, so either I'm a really unlucky guy, or there's something really wrong with my setup.
EDIT3: Just realized one of the port was still plugged on HP 530SFP+ when I did first edit. Unplugged it and magic, everything is working fine. So it seems the breakout ports really hates the HP NICs.
The error is still there at boot when e 1/2/2 is plugged in (even only with the Chelsio card) but it works. I disabled it and used an other port, the error is gone.
EDIT4: Ok so two SFP+ from breakout in a lag, I had to reboot the switch but it finally came up with VLAN. The others two SFP+ are unplugged as the switch doesn't seem to like my HP NICs.
I have other (weird) issues like one of my ESXi saw one NIC (gigabit) as down but the switch saw it up. Up/down on ESXi did nothing, but disable/enable on the switch made it come back up.
An other ESXi node (on a NUC, so Intel NIC this time) has gone crazy after rebooting the switch (up/down really fast). Rebooting NUC fixed the issue, but it seems to be recurring when I reboot the switch.
It also rebooted a few times on its own over the past few weeks, so I'll wait a little bit to see if those issues persists. If they do, I'll try to swap my current unit with the new one.
If the issues still persists, I don't know, maybe I'll buy some 7250 instead as I have some trust issues with the 6610 now...
EDIT5: It rebooted again for no reason and boom, rear SFP+ are broken again. I can't get them to go up again. Will try unit 2. I also still have the NIC going up and down in loop for the NUC. Rebooting the NUC fixed it.