G8124 issues...

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Michiel

New Member
Jul 5, 2016
2
0
1
56
Hi guys,

Hoping to find someone with some insights into 10Gbe networking.

I recently picked up a "cheap" G8124 in the hope I could replace my Lenovo/Intel X520-DA2 strait DAC connection between my DL160-G6 and QNAP TVS-1271U-RP with a switch and add the other Xen servers and backbone switches to the party.
The configuration I want to go for is a single switch with multiple servers and nas servers attached running Xenserver.

However, whatever I try, I can not get the 8124 to recognize my DAC cables. I have several sfp+ DAC cables, all either Lenovo 2m 00AY765 or HP 1m 487652-B21. Think the cables are OK as they report to the servers but the switch keeps telling me that the tranceivers are NOT APPROVED. The logs shows me:

Jul 5 11:09:00 phssw02 NOTICE system: Port 1 Installed Unknown
Jul 5 11:09:01 phssw02 NOTICE system: Port 1 Wrong Transceiver Type Unknown

I have tried all sorts of configurations but no luck so far. I have the feeling I need to turn on or turn off something..... or need other tranceivers/cables all together or I may have gotten a broken switch.... Anyone with an idea on how the configure the 8124 or at least confirm it's fubar.


Some info on the Switch and cards:

show version:
IBM Networking Operating System RackSwitch G8124

Switch has been up for 0 days, 1 hour, 31 minutes and 5 seconds.
Last boot: 13:20:40 Tue Jul 5, 2016 (reset by WEB-UI)

MAC address: 34:40:b5:6b:a5:00 IP (If 1) address: 192.168.49.50
MGMT-A Port MAC Address: 34:40:b5:6b:a5:fe
MGMT-A Port IP Address (if 127): 172.16.0.34
MGMT-B Port MAC Address: 34:40:b5:6b:a5:ef
MGMT-B Port IP Address (if 128): 192.168.51.50
Hardware Revision: 0
Board Revision: 2
Switch Serial No: Y350UN26S01B
Hardware Part No: BAC-00045-00 Spare Part No: BAC-00045-00
Manufacturing date: 12/27

MTM Value: 1455-24E
ESN: 23W0435

Software Version 7.11.7 (FLASH image1), factory default configuration.
Boot kernel version 7.11.7

------------------------------------------------------------------
NOTE: System dump exists in FLASH.
The dump was saved at 12:05:05 Tue Jul 5, 2016.
Use 'copy flash-dump' to
extract the dumps for analysis and 'clear flash-dump' to
clear the FLASH region.
------------------------------------------------------------------


Temperature Sensor 1: 41.5 C
Temperature Sensor 2: 37.5 C
Temperature Sensor 3: 42.75 C
Temperature Sensor 4: 46.50 C
Temperature Sensor 5: 44.75 C

Warning at 85C and Failure at 100C

Speed of Fan 1: 8307 RPM (75 PWM)
Speed of Fan 2: 8503 RPM (75 PWM)
Speed of Fan 3: 8896 RPM (75 PWM)
Speed of Fan 4: 8503 RPM (75 PWM)
Speed of Fan 5: 8640 RPM (75 PWM)
Speed of Fan 6: 8095 RPM (75 PWM)

System Fan Airflow: Front to Rear

State of Power Supply 1: On
State of Power Supply 2: On


RS G8124#show transceiver

Port Device TX TXuW RXuW TXFlt Volts DegsC Laser Approval
-------- ------- --- ----- ----- ----- ----- ----- ----- ------ ----------
Port 1 < WRONG Device Type (SFP vs SFP+) >
Detail -> Vendor: Error
Part: Error
Date: Error
S/N: Error
Port 2 < WRONG Device Type (SFP vs SFP+) >
Detail -> Vendor: Error
Part: Error
Date: Error
S/N: Error

end more like that...


X520-DA2 info

lspci -vvv -s 07:00
07:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin A routed to IRQ 24
Region 0: Memory at fbc00000 (64-bit, non-prefetchable) [size=1M]
Region 2: I/O ports at e880
Region 4: Memory at fbdf8000 (64-bit, non-prefetchable) [size=16K]
Expansion ROM at fbb80000 [disabled] [size=512K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=64 Masked-
Vector table: BAR=4 offset=00000000
PBA: BAR=4 offset=00002000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 256 bytes, MaxReadReq 4096 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s, Exit Latency L0s unlimited, L1 <8us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Vital Product Data
Product Name: Intel X520 Dual Port 10GbE SFP+ Adapter
Read-only fields:
[SN] Serial number: 11S00JY855Y050KK58N274
[PN] Part number: 49Y7960
[MN] Manufacture ID: 37 41 31 32
[V1] Vendor specific: Intel X520 Dual Port 10GbE SFP+ Adapter
[V2] Vendor specific: 00JY855
[VL] Vendor specific: 49Y7962
[VR] Vendor specific: 22
[VS] Vendor specific: 32773
[VT] Vendor specific: 9
[EC] Engineering changes: N34030
[VU] Vendor specific: 2C55B7D5DE7C435A885E592FFA246522
[RV] Reserved: checksum good, 0 byte(s) reserved
End
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [140 v1] Device Serial Number 90-e2-ba-ff-ff-ae-d4-d8
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 1
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+
IOVSta: Migration-
Initial VFs: 64, Total VFs: 64, Number of VFs: 0, Function Dependency Link: 00
VF offset: 128, stride: 2, Device ID: 10ed
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 00000000c0000000 (64-bit, prefetchable)
Region 3: Memory at 00000000c0100000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Kernel driver in use: ixgbe
Kernel modules: ixgbe



Server info (DAC between GL160 en QNAP installed):

ethtool eth2
Settings for eth2:
Supported ports: [ FIBRE ]
Supported link modes: 10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: No
Advertised link modes: 10000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: No
Speed: 10000Mb/s
Duplex: Full
Port: Direct Attach Copper
PHYAD: 0
Transceiver: external
Auto-negotiation: off
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: no

and nas shows:

Interface :
Adapter 5 (10GbE)
DHCP : No
IP Address : 172.16.11.2
Submask : 255.255.255.0
Gateway : 172.16.11.1
MAC Address : 90:E2:BA:AF:EC:19
Speed : 10 Gbps
MTU : 9000
Virtual LAN (VLAN) : No

eth3 is the same both with HP and lenovo cables when connected between the server and the nas box.


If i need to provide more info, please let me know....

Any help appreciated
 

s0lid

Active Member
Feb 25, 2013
259
35
28
Tampere, Finland
Well there's only one solution: Buy transceivers or dac cables approved by the switch.

1m IBM Passive DAC SFP+ Cable 90Y9427
3m IBM Passive DAC SFP+ Cable 90Y9430
5m IBM Passive DAC SFP+ Cable 90Y9433
8.5m IBM Passive DAC SFP+ Cable 90Y9436
IBM BNT SFP+ LR Transceiver 90Y9412
IBM BNT SFP+ ER Transceiver 90Y9415
IBM BNT SFP ZX Transceiver 90Y9418
IBM BNT SFP LX Transceiver 90Y9424
 

s0lid

Active Member
Feb 25, 2013
259
35
28
Tampere, Finland
Is that due to differences between lenovo OS and IBM N/OS?
Yes and the switch you've got is G8124 which is manufactured by BNT/IBM. The levonopress page you linked is for G8124E which is made by lenovo and probably has significant software and hardware differences with G8124.
 

pyite

New Member
May 15, 2013
9
1
3
FYI, according to the installation guide the difference between the G8124 and G8124E is the CPU:
"The RackSwitch G8124-E is a performance-enhanced switch with a faster management processor for improved aggregation layer functionality"

I just inherited one of these. It looks like a good switch with plenty of layer 3 options. The big drawback is that it seems to reject all of the third party SFPs I tried apart from an Arista 3 meter DAC. If anyone knows a way to fix this problem, please let me know -- I had a big box of free SFPs but I couldn't use them... I had to buy some IBM (Blade) branded SFPs.

It isn't too loud once it settles down, but I wouldn't want it in the room with me.