Recent content by jpmomo

  1. J

    Mellanox/Nvidia Connectx-7 fw update

    please try with a qsfp56 dac and let us know the results.
  2. J

    Mellanox/Nvidia Connectx-7 fw update

    the good news is that the link capacity and more importantly link status look correct: 32GTs and x16. also, the qsfp112 cx7 is a lot easier to use than the osfp version. You should be using a qsfp56 dac as each port will use the full capacity (200G). What is your eventual use case for your...
  3. J

    Mellanox/Nvidia Connectx-7 fw update

    what are you using to test small frame sizes? What are your system specs? What does the output of lspci -vv look like for the pci id of the cx7 nic?
  4. J

    Mellanox/Nvidia Connectx-7 fw update

    try the following command: mlxconfig -d /dev/mst/mt4103_pci_cr0 set LINK_TYPE_P1=2 LINK_TYPE_P2=2 where the device id is your device id of the actual nic. even though the description of the card says it is 1 IB and 1 EN, this fw should allow 2 EN ports.
  5. J

    (FS-US-AZ) JP's custom high end servers!

    Nothing too special:) The servers need to be optimized for generating high traffic rates via VMs or containers. I just prefer that they don't look like the typical beige box!
  6. J

    (FS-US-AZ) JP's custom high end servers!

    Moving again and need to clear out some of my custom server builds. I have several items that I will be listing this week but wanted to get started. 1. The first build is based on an Asrockrack X570D4I-2T utilizing an AMD Ryzen 9 3900X. It was built using a Lianli TU150 case that has the...
  7. J

    CX516A-GCAT -> CX556A-ECAT?

    the adapter properties only shows it working at 40Gbps. it show the bus is 16GT but only x4
  8. J

    400G QSFP-DD switch recommendations?

    Does anyone have any recommendations on a "relatively" cheap 400G QSFP-DD switch? One requirement is to be able to configure each port to fanout to 2x200G. I noticed that on some of the tofino based edgecore switches, they only fanout to 4x100G.
  9. J

    Nvidia CX-7 400G NIC --- QSFP-DD to OSFP DAC?

    I am not exactly following what you need to accomplish. Originally, I was trying to connect a qsfp-dd (400G) device to a single port osfp cx-7 nic. that required several expensive and hard to get components: mms1v00-wm (400G DR4 single mode transceiver) mfp7e30-nxxx (single mode crossover...
  10. J

    Where can I find NMLXCLI (Mellanox Management CLI tools)

    I am trying to find the NMLXCLI vib file. I need to use this cli tool to set spoof checking off and trust on in vmware. This needs to be done in order for VLANs, SRIOV and ESX to live happily ever after! If I use PCI-PT instead of SRIOV, it works. Unfortunately, I need to use SRIOV and need to...
  11. J

    Go on, admit it, who here is already running at 200Gbe??

    the are a few variants of the cx-7 nics. there are the nics that use an osfp connector (physically larger than the qsfpxxx counterpart) and some that are dual port qsfp112. the dual port cx-7 nics that are qsfp112 work with a cheap qsfp56 dac if you are just connecting the 2 ports b2b (back to...
  12. J

    Go on, admit it, who here is already running at 200Gbe??

    see the other cx-7 threads. next stop cisco 32x800GE in 1U! (just need to find the damn breakout cables! at least until they come out with the cx-8!)
  13. J

    Nvidia CX-7 400G NIC --- QSFP-DD to OSFP DAC?

    STH just posted an article on these nic cards. I am trying to get some clarification on the specific cable to connect a QSFP-DD switch to one of the OSFP cx-7 nics. There are 2 issues that we encountered: 1. the nic side requires an OSFP "Flat top" connector not the "Fin top" that is normally...
  14. J

    AMD EPYC 9684X ES 96C168T 3D L3

    I still can't quite tell if either of these boards might work. I do see Microcode of A0 for both but not sure of the rest of the version info. The AGESA on the tyan also looks to be newer which may not be better if we are trying to get an ES chip to post.
  15. J

    (all sold) [FS] [US-Seattle] Very High-end Servers (added a couple more!) and 2 x amd genoa 9654 LOTs of Pictures now!

    All sold! Thanks for everyone that purchased! Next round......I will list some more interesting items ;)