ConnectX-4 LX $25.99-Seller raised to $39.99

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
C

Civiloid

Active Member
Jan 15, 2024
230
175
43
Switzerland
That seems to be rather common price point for lots from US nowadays. I've some time ago bought about 5 of HP-branded CX 4 Lx for about same price and even with forwarding from US it was cheaper than China prices.
 
  • Like
Reactions: nexox
D

DavidVermillion

Member
Sep 22, 2015
68
33
18
Be a little careful. I've had issues with these being sensitive to motherboards, and downgrading link speed to PCIe 1.0.

I've had two different cards and two different motherboards exhibit this issue. 25gb Intel NICS and even Broadcom NICS didn't have the same issue.
 
  • Wow
  • Like
Reactions: Rcdriver and T_Minus
J

jbv1982

Member
Nov 5, 2017
32
16
8
49
I was able to pick up a couple of Cx4-Lx with one Mellanox SFp28 for 25 each last week, so this seems to be a normal price.
 
  • Like
Reactions: T_Minus
M

Michael Hall

Member
Oct 9, 2015
44
13
8
DavidVermillion said:
Be a little careful. I've had issues with these being sensitive to motherboards, and downgrading link speed to PCIe 1.0.

I've had two different cards and two different motherboards exhibit this issue. 25gb Intel NICS and even Broadcom NICS didn't have the same issue.
Click to expand...
Hmm... They'll be going in the X10SLL-F in my NAS and the P330 Tiny-based Proxmox+Ceph cluster I'm building, so hopefully everybody'll get along...
 
nabsltd

nabsltd

Well-Known Member
Jan 26, 2022
813
614
93
luckylinux said:
Of, if you don't care about ASPM and Power Consumption / Temperatures at all, plug it into the fastest x16/x8 Slot you want, and most likely you'll be stuck at PC2/PC3 (Package C-State 2 or Package C-State 3) and high Idle Power Consumption.
Click to expand...
My 10Gbps (or faster) server NICs are never idle, and I suspect that this is true for many people. With even 3-4 running VMs where their storage is on the other end of the link, it's always doing something.
 
C

Civiloid

Active Member
Jan 15, 2024
230
175
43
Switzerland
DavidVermillion said:
Be a little careful. I've had issues with these being sensitive to motherboards, and downgrading link speed to PCIe 1.0.
Click to expand...
I haven't seen such problems on latest firmware. On earlier ones - it is hit-or-miss tbh, they even don't post on some consumer motherboards with really old firmwares.
 
B

bugacha

Active Member
Sep 21, 2024
509
159
43
C

Cruzader

Well-Known Member
Jan 1, 2021
961
951
93
I had 20$/ea for 12 accepted on this listing, feels like a fairly standard price for them.

There is a 25$/ea listing with offers enabled that has been up for a long time also, but have not tried offering on that one.
I always do 1-2 lowball offers before offering what i expect them to accept to feel out if its declined or where the counters are at.

Now and then you get a nice suprised when the borderline insulting lowballs like 20-25$ on 100$ listings are accepted.
 
Prophes0r

Prophes0r

Active Member
Sep 23, 2023
167
212
43
East Coast, USA
Michael Hall said:
It was listed for $25.95 at the time I made my offer, and the initial post. Seems the seller raised the price since then.
Click to expand...
Why not update the post title then?
You took the time to reply with a perfectly valid reason, but only you(or a mod?) can update the title.
It takes moments to fix.
 
fohdeesha

fohdeesha

Kaini Industries
Nov 20, 2016
2,987
3,609
113
35
fohdeesha.com
luckylinux said:
Sure, not 100% Idle, but maybe 30%-50% Idle and there from PC2/PC3 down to PC6 (or PC7 or higher) makes quite a bit of Difference I'd say.

Take a look at turbostat e.g.

Code: 
turbostat --show Avg_MHz,Busy%,Bzy_MHz,TSC_MHz,POLL,POLL%,C1%,C1E%,C3%,C6%,C7s%,CPU%c1,CPU%c3,CPU%c6,CPU%c7,Pkg%pc2,Pkg%pc3,Pkg%pc6,Pkg%pc7,Pkg%pc8,Pkg%pc9,Pk%pc10,PkgWatt,CorWatt,IPC,IRQ,SMI --interval 0.5
Click to expand...
luckylinux said:
No Issues with Supermicro X10SLL-F or Supermicro X10SLM-F or even Supermicro X11SLL-F, provided that you plug the NIC into the PCH/DMI/Chipset Connected PCIe Slot, for ASPM to work. But you'll be limited to 16gbps Total Bandwidth (X10SLL-F in PCH Slot), so either you enable Switchdev / Offload and/or possibly ROCE v2 if you want to switch/route between the 2 Ports and possibly saving on PCIe Traffic/Bandwidth (unsure how much Bandwidth would be saved by these Offloads though). PCH/DMI/Chipset Shares Bandwidth with SATA/Onboard NIC so you'll lose quite a bit of Bandwidth if you put it there. On the Supermicro X11SSL-F it's a bit better with up to 32gbps theoretical Bandwidth so if you don't use too much in SATA then it's OK. It depends what you care about: Speed or power Consumption ?

Of, if you don't care about ASPM and Power Consumption / Temperatures at all, plug it into the fastest x16/x8 Slot you want, and most likely you'll be stuck at PC2/PC3 (Package C-State 2 or Package C-State 3) and high Idle Power Consumption :) .
Click to expand...
what's high power consumption? 9 watts instead of 6? I think I'll survive lmao
 
fohdeesha

fohdeesha

Kaini Industries
Nov 20, 2016
2,987
3,609
113
35
fohdeesha.com
luckylinux said:
You'll see :p.

EDIT 1: it's NOT (only) about the Power Consumption of your NIC. It's about the CPU NOT being able to sleep, so maybe 10-20W more on a Xeon E3 Platform ...

EDIT 2: So maybe you'll have 30-40W at the Wall instead of 15-20W (at Idle) :p . Then the only one LMAO will be the Electricity Supplier :p
Click to expand...
An extra 20 watts would be about 90 cents a month here, not that I'd notice it given my rack is currently at something like 930w. I can't imagine a cpu in a 10gbe machine would have the opportunity to go to sleep very often if ever, if I'm building a server or firewall it tends to.....do things
 
  • Like
Reactions: nabsltd and nexox
W

WhiteNoise

Active Member
Jan 20, 2024
130
57
28
I believe that not supporting ASPM and not letting the CPU enter into deeper C-states is a feature and not a bug.

In server workload scenarios, predictable, consistent and low latency response is a feature. It's possible that driver themselves set up the card to not go sleep (ASPM) or let the CPU go to sleep (package c states).
 
D

duckduck

Member
Aug 22, 2016
37
41
18
luckylinux said:
Intel X710 (and allegedly Intel XXV710 and Intel E810) do support ASPM.

And, as I said, the Mellanox ConnectX-4 LX seems to support ASPM, when plugged into a PCH/DMI/Chipset Slot. But NOT if plugged into a CPU-connected PCIe Slot.
Click to expand...
Does the Intel NICs work in the CPU connected PCIe lanes as well Or just via the chipset?
I first saw the ASPM issue with the CPU PCIe lanes on W680 boards. But seems to go much further back at least on Intel motherboards. Not sure about AMD.
 
D

duckduck

Member
Aug 22, 2016
37
41
18
luckylinux said:
9)
My Experience on Intel Systems (Supermicro X10SLL-F/X10SLM(+)-F/X11SSL-F/X11SSM-F) with the Intel X710 worked in CPU Connected PCIe Slots as well as PCH/Chipset connected PCIe Slots. Maybe the difference is between PC6 (CPU Connected) and PC7/PC8 (PCH/Chipset Connected). Anyhow, the Difference between those States is very small compared to the Difference between PC2/PC3 and PC6.

Mellanox ConnectX-4 LX on those Systems in CPU Connected PCIe Slot would be stuck at PC2/PC3. In PCH/Chipset connected PCIe Slot it would be PC6.

I don't think I ever tested the Mellanox ConnectX-4 LX on an AMD System. I could now, but on AMD it's also more difficult to Troubleshoot since powertop and turbostat report MUCH LESS Information. Typically on AMD you only have 3 Package States IIRC. And the Software Tools are not always working great ...

So I cannot emphasize this enough: Testing with powertop / turbostat has to be complemented with measurements using a Watt-Meter ! Just like ASPM reporting of PCIe Devices can lie to the System (and BIOS can lie to the System as well), so can powertop and turbostat. I also had a ConnectX-2 reporting ASPM Supported on a Supermicro X9DRi-LN4F+ IIRC (not sure about the Package C-State the System was in, but that was clearly NOT possible).

EDIT 1: the Issue on the W680 System (if that's Intel 11th-13th Generation) if CPU PCIe Connected is probably due to PCIe Multi-VC (Multiple Virtual Channels). I mentioned that on my Repository briefly: GitHub - luckylinux/aspm-troubleshooting: aspm-troubleshooting

But nope, I don't own such System and while disassembling BIOS, the Multi-VC or Virtual Channel Options didn't turn up either.
Click to expand...
Thanks. This will save me a lot of time, having to try things out myself.

I'm still in the planning of what gear to get for a little home proxmox cluster setup. Aiming to keep the power usage of three nodes, close to that of my current server. With the same electricity prices as you (DK). Even if ASPM only brings it down a little overall. If the different NICs cost about the same. Might as well get the kind where it works.
 
You must log in or register to reply here.
Top Bottom