EXPIRED Mellanox ConnectX-6 2x100GbE NIC for $229

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

NablaSquaredG

Layer 1 Magician
Aug 17, 2020
1,353
821
113
1686067400531.png

755 sold?!?! This morning there were over 700 left.

STH effect TM

That's the reason why I don't post deals here. It's too public. I'm almost certain that a majority was picked up by either a reseller or some big company...
 
  • Like
Reactions: mach3.2 and Jaket

Jaket

Active Member
Jan 4, 2017
242
131
43
Seattle, New York
purevoltage.com
View attachment 29455

755 sold?!?! This morning there were over 700 left.

STH effect TM

That's the reason why I don't post deals here. It's too public. I'm almost certain that a majority was picked up by either a reseller or some big company...
That's pretty insane. I bet it was a reseller who picked these up knowing they could get double the price. Which is quite sad that this happens. I know I could have done this. But bought 4 that we would need at the moment. Should have bought a lot more last night myself....
 
  • Like
Reactions: NablaSquaredG

Jaket

Active Member
Jan 4, 2017
242
131
43
Seattle, New York
purevoltage.com
I've seen this a few times from resellers. Sometimes they will sit on all of those for a year or two and then have to sell at or below cost depending. However, this was a hot deal for the time being that's for sure.

Wish I picked up a few more but at the same time we only rent out servers with 100G a few times a month at most right now still most customers want 40Gbps at most other than our really high end 128 core amd beasts.
 
  • Like
Reactions: Rttg

GladLock96

Member
Nov 8, 2016
81
20
8
I bought 5
I bought one of these Mellanox ConnectX-6 CX623106A NICs last month from this seller at slightly higher price; since then, they've reposted the listing with the price lowered to $229 (plus $10 shipping). This seemed like too good a deal to pass up, so I bought another and thought you all might also appreciate a heads up as well.


Edit: as pointed out by GladLock96 below, these are actually Cisco UCS-flavored Mellanox NICs. In my experience, they are not transceiver-locked and work fine with non-Cisco switches, but GladLock96 pointed out and I confirmed with my own sample that they cannot be directly flashed by Nvidia's firmware tool to an Nvidia-supplied firmware image. Caveat emptor, they seem to work fine IME, but firmware updates will be an issue unless you have the expensive MTUSB-1 tool mentioned further downthread.
My wallet hates you. I bought 5, and now I'm about to buy MikroTik CRS504-4XQ-IN Cloud Router Switch too.
 

Jaket

Active Member
Jan 4, 2017
242
131
43
Seattle, New York
purevoltage.com
Argh, I offered $220 for 5 and they accepted. I should have gone much lower like you.
Yeah it's always a tricky one we sometimes will do it and other times we have lost out because someone else bought everything before our offer was accepted.
We've spent a good 200k I bet over the years from those sellers as well so that could have also had some effect.
 

bob_dvb

Active Member
Sep 7, 2018
214
116
43
Not quite London
www.orbit.me.uk

klui

Well-Known Member
Feb 3, 2019
844
463
63
Could you guys report if CX6s accept non-Cisco 40G transceivers like the Intel, KAIAM, Arista for CX4s/CX5s?
 

bitbckt

will google compiler errors for scotch
Feb 22, 2022
213
134
43
I received my order today. I should be able to get it flashed to OEM firmware in the next few days. Here's some info on one board as I received it:

Code:
# flint -d /dev/mst/mt4125_pciconf0 query
Image type:            FS4
FW Version:            22.30.1004
FW Release Date:       29.3.2021
Product Version:       22.30.1004
Rom Info:              type=UEFI version=14.23.17 cpu=AMD64,AARCH64
                       type=PXE version=3.6.301 cpu=AMD64
Description:           UID                GuidsNumber
Base GUID:             e8ebd30300d09292        4
Base MAC:              e8ebd3d09292            4
Image VSD:             N/A
Device VSD:            N/A
PSID:                  CIS0000000008
Security Attributes:   secure-fw
Code:
# lspci -vvvv -s 01:00.1
lspci -s 01:00.1 -vvvv
01:00.1 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
    Subsystem: Cisco Systems Inc MT2892 Family [ConnectX-6 Dx]
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin B routed to IRQ 241
    IOMMU group: 49
    Region 0: Memory at 280b4000000 (64-bit, prefetchable) [size=32M]
    Expansion ROM at f3300000 [disabled] [size=1M]
    Capabilities: [60] Express (v2) Endpoint, MSI 00
        DevCap:    MaxPayload 512 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 75W
        DevCtl:    CorrErr- NonFatalErr- FatalErr- UnsupReq-
            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
            MaxPayload 512 bytes, MaxReadReq 512 bytes
        DevSta:    CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
        LnkCap:    Port #0, Speed 16GT/s, Width x16, ASPM not supported
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl:    ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta:    Speed 16GT/s, Width x16
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABC, TimeoutDis+ NROPrPrP- LTR-
             10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- TPHComp- ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: ReqEn+
        LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
             EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [48] Vital Product Data
        Product Name: Cisco-MLNX MCX623106AS-CDAT 2x100GbE QSFP56 PCIe NIC                                                                                                                                  
        Read-only fields:
            [PN] Part number: 30-100260-01             
            [EC] Engineering changes: A1
            [V2] Vendor specific: 30-100260-01             
            [SN] Serial number: MLX26290108    
            [V3] Vendor specific: cabcf5bf7009ed118000e8ebd3d09292
            [VA] Vendor specific: MLX:MN=MLNX:CSKU=V2:UUID=V3:PCI=V0:MODL=CX623106A      
            [V0] Vendor specific: PCIeGen4 x16 
            [RV] Reserved: checksum good, 1 byte(s) reserved
        End
    Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
        Vector table: BAR=0 offset=00002000
        PBA: BAR=0 offset=00003000
    Capabilities: [c0] Vendor Specific Information: Len=18 <?>
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0-,D1-,D2-,D3hot-,D3cold+)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [100 v1] Advanced Error Reporting
        UESta:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt:    DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        CEMsk:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap:    First Error Pointer: 04, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
        ARICap:    MFVC- ACS-, Next Function: 0
        ARICtl:    MFVC- ACS-, Function Group: 0
    Capabilities: [180 v1] Single Root I/O Virtualization (SR-IOV)
        IOVCap:    Migration- 10BitTagReq- Interrupt Message Number: 000
        IOVCtl:    Enable- Migration- Interrupt- MSE- ARIHierarchy- 10BitTagReq-
        IOVSta:    Migration-
        Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 00
        VF offset: 9, stride: 1, Device ID: 101e
        Supported Page Size: 000007ff, System Page Size: 00000001
        Region 0: Memory at 00000280b6800000 (64-bit, prefetchable)
        VF Migration: offset: 00000000, BIR: 0
    Capabilities: [230 v1] Access Control Services
        ACSCap:    SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
        ACSCtl:    SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
    Capabilities: [420 v1] Data Link Feature <?>
    Kernel driver in use: mlx5_core
    Kernel modules: mlx5_core
 

GladLock96

Member
Nov 8, 2016
81
20
8
I received my order today. I should be able to get it flashed to OEM firmware in the next few days. Here's some info on one board as I received it:

Code:
# flint -d /dev/mst/mt4125_pciconf0 query
Image type:            FS4
FW Version:            22.30.1004
FW Release Date:       29.3.2021
Product Version:       22.30.1004
Rom Info:              type=UEFI version=14.23.17 cpu=AMD64,AARCH64
                       type=PXE version=3.6.301 cpu=AMD64
Description:           UID                GuidsNumber
Base GUID:             e8ebd30300d09292        4
Base MAC:              e8ebd3d09292            4
Image VSD:             N/A
Device VSD:            N/A
PSID:                  CIS0000000008
Security Attributes:   secure-fw
Code:
# lspci -vvvv -s 01:00.1
lspci -s 01:00.1 -vvvv
01:00.1 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
    Subsystem: Cisco Systems Inc MT2892 Family [ConnectX-6 Dx]
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin B routed to IRQ 241
    IOMMU group: 49
    Region 0: Memory at 280b4000000 (64-bit, prefetchable) [size=32M]
    Expansion ROM at f3300000 [disabled] [size=1M]
    Capabilities: [60] Express (v2) Endpoint, MSI 00
        DevCap:    MaxPayload 512 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 75W
        DevCtl:    CorrErr- NonFatalErr- FatalErr- UnsupReq-
            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
            MaxPayload 512 bytes, MaxReadReq 512 bytes
        DevSta:    CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
        LnkCap:    Port #0, Speed 16GT/s, Width x16, ASPM not supported
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl:    ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta:    Speed 16GT/s, Width x16
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABC, TimeoutDis+ NROPrPrP- LTR-
             10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- TPHComp- ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: ReqEn+
        LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
             EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [48] Vital Product Data
        Product Name: Cisco-MLNX MCX623106AS-CDAT 2x100GbE QSFP56 PCIe NIC                                                                                                                                 
        Read-only fields:
            [PN] Part number: 30-100260-01            
            [EC] Engineering changes: A1
            [V2] Vendor specific: 30-100260-01            
            [SN] Serial number: MLX26290108   
            [V3] Vendor specific: cabcf5bf7009ed118000e8ebd3d09292
            [VA] Vendor specific: MLX:MN=MLNX:CSKU=V2:UUID=V3:PCI=V0:MODL=CX623106A     
            [V0] Vendor specific: PCIeGen4 x16
            [RV] Reserved: checksum good, 1 byte(s) reserved
        End
    Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
        Vector table: BAR=0 offset=00002000
        PBA: BAR=0 offset=00003000
    Capabilities: [c0] Vendor Specific Information: Len=18 <?>
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0-,D1-,D2-,D3hot-,D3cold+)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [100 v1] Advanced Error Reporting
        UESta:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt:    DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        CEMsk:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap:    First Error Pointer: 04, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
        ARICap:    MFVC- ACS-, Next Function: 0
        ARICtl:    MFVC- ACS-, Function Group: 0
    Capabilities: [180 v1] Single Root I/O Virtualization (SR-IOV)
        IOVCap:    Migration- 10BitTagReq- Interrupt Message Number: 000
        IOVCtl:    Enable- Migration- Interrupt- MSE- ARIHierarchy- 10BitTagReq-
        IOVSta:    Migration-
        Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 00
        VF offset: 9, stride: 1, Device ID: 101e
        Supported Page Size: 000007ff, System Page Size: 00000001
        Region 0: Memory at 00000280b6800000 (64-bit, prefetchable)
        VF Migration: offset: 00000000, BIR: 0
    Capabilities: [230 v1] Access Control Services
        ACSCap:    SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
        ACSCtl:    SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
    Capabilities: [420 v1] Data Link Feature <?>
    Kernel driver in use: mlx5_core
    Kernel modules: mlx5_core
If you get it flashed properly back to OEM firmware - please let me know your process, as I would love to do the same.
 

bitbckt

will google compiler errors for scotch
Feb 22, 2022
213
134
43
I could probably hook it up to my Saleae and snoop the I2C traffic. Maybe there’s something to be learned in there.
 
  • Like
Reactions: mach3.2