Search results

  1. J

    2x 56GbE Optimization, slow ceph recovery, and MLNX-OS S.O.S.

    During rados benchmarking Example iostat: Example htop:
  2. J

    2x 56GbE Optimization, slow ceph recovery, and MLNX-OS S.O.S.

    Necro'ing my own thread to keep the conversation in one place, with a ~1 year later update. Been chipping away at this for a while now. Swapped back to IB, then ETH, tried a ton of config changes, etc. At this point, I am not actually concerned about my recovery speed, nor my RBD VM disk...
  3. J

    US MELLANOX SX6036 - $200

    I also had to buy an adapter, but it's been useful to have on hand for other peices of hardware too. I recommend just buying a cheap "USB console cable" (<$20). The manual for these is pretty easy to find online, and there is a factory reset button too. But I beleive DHCP is off by default, and...
  4. J

    US MELLANOX SX6036 - $200

    +1 for @cw823, who helped me out with the same.
  5. J

    2x 56GbE Optimization, slow ceph recovery, and MLNX-OS S.O.S.

    Hey, Thats a really great catch, thank you for highlighting it. Ill be swapping half the ports back to ipoib for further testing, will post some benchmarks in the next day or so while debugging further! This because of PCIe saturation, to avoid the bonding during testing, and so I have side by...
  6. J

    2x 56GbE Optimization, slow ceph recovery, and MLNX-OS S.O.S.

    I had this as a comment in another thread, but moved it here to it's own thread. I have a three node Ceph cluster that I am diagnosing the presumed slow throughput of. Each node has a on a connectx-3 Pro 2 port qsfp+. While I was running them via ib_ipoib, in an attempt to get past the low...
  7. J

    Connectx-3 Pro (FDR) port_xmit_discards counter overflow / PortXmitDiscard

    Edit: Dis-regard, I am going to open a more appropriately titled thread.
  8. J

    US MELLANOX SX6036 - $200

    Wow, it strikes me as super fortunate that I found this thread! I am coincidentally in the thick of the firmware upgrade process, having just failed several attempts before finding HPE's upgrade chart. @Rand__ Could you expand on the Ethernet conversion you mentioned (for me and any other...
  9. J

    Connectx-3 Pro (FDR) port_xmit_discards counter overflow / PortXmitDiscard

    Hello! I am having some troubles understanding a counter overflow error per the title. I've tracked down Mellanox' support detailing this counter: port_xmit_discards Total number of outbound packets discarded by the port because the port is down or congested. PortXmitDiscards Error...
  10. J

    Seagate Utils - Seems I've enabled something and now my HDD do not spin up at boot!

    Whoops, forgot to reply here. Thank you for your response. the seagate tools eventually did manage to get these alive again. Needed to use an original enclosure, on windows, and a few days of mucking about. I ended up going through their support, getting a runaround for over a month, got them...
  11. J

    Seagate Utils - Seems I've enabled something and now my HDD do not spin up at boot!

    Hoping someone can help me retrace my steps to identify wtf I've done to my drives. I was trying to format Seagate SMR drives in an effort to bring them back to their original read and write speeds while also trying to see if there was a way to patch the massive amount of SMART errors on the...
  12. J

    Install Mellanox Driver Proxmox VE 5.1

    Woops, my apologies, I spent several hours in "debug hell" before making my way here, so you're right, I am off thread a bit. I am using Proxmox 6.0, and more of my issue may be related to the fact that I am trying to get a couple connectx2 working, as they're fairly old hardware... Could also...
  13. J

    Install Mellanox Driver Proxmox VE 5.1

    Hello, Sorry to dredge up this old thread, but I am having some troubles with the following error: One or more required packages for installing MLNX_OFED_LINUX are missing. Attempting to install the following missing packages: libgfortran3 libssl1.0.0 Failed command: apt-get install -y...