Search results

  1. A

    FS: Mini HPC Cluster, 108 cores

    For sale is my mini-HPC cluster setup. I built it for running CFD and astrophysics simulations, but don't have any room for it in my current place. I know roughly what I could part it out for, but it has some value as a ready-to-go cluster, so I'd like to try to sell it as a whole first...
  2. A

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    AHA! It worked! I have FDR speeds now! You can burn mellanox .bin firmware directly to EMC SX6005 switches almost exactly like you do with rebranded HCAs. You need the LID of the switch from "ibswitches", and you need to install MFT. Then the command is: flint -d lid-X -i fw.bin...
  3. A

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    Ok, made a little progress on this. I installed MFT. Then I did "ibswitches" to find the LID of the switch. I then did "flint -d lid-<switch lid> query full". This returned the firmware version (9.3.2000, so not latest), as wel as the PSID, which was "EMC1260110026". Now, if you look at the...
  4. A

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    Good call. Mine has two psus, so there's a better chance it's an FDR switch than FDR10, but EMC could have some special configuration, so who knows. I'm going to install MFT and try some of those tools to figure out what's going on.
  5. A

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    No idea how to do that unfortunately. The only command I know for looking at switch info is "ibswitches". Know any others?
  6. A

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    Yes. There are two types of SX6005, one with a "T" in the part name and one with an "F". The former is FDR10 and the latter is FDR. However, the EMC switches just have "SX6005" and then a purely numerical part number on the pull out tab. Basically, I'm trying to figure out if the switch is...
  7. A

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    I know the mellanox ones do. The "T" means FDR10, and the "F" means FDR, but the EMC part numbers are just numbers. Do you know how I could look up my EMC part number to determine whether it's an FDR10 or FDR?
  8. A

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    I bought one of these EMC SX6005 switches. My IB cards and cables will show 56 Gbit/s link speed when connected back to back, but the link speed drops to FDR10 when connected through the switch. The switch is unmanaged, so I don't think I can check its port speeds. Since it's EMC, the part...
  9. A

    FS: 80+ Intel Xeon Phi 7-ES, 71S1P, 7110P Co-processor PCIe cards -used

    I wouldn't say "completely abandon"... The MPSS for these is fully working and still available. They just aren't going to update it anymore and have ceased production.
  10. A

    FS: 80+ Intel Xeon Phi 7-ES, 71S1P, 7110P Co-processor PCIe cards -used

    For sale are 80+ Intel Xeon Phi Co-processors. The available model numbers are 7-series ES (engineering sample), 71S1P, and 7110P. I also have many of the extension brackets and pcie brackets. Conditions range from tested/good to for-parts/broken. Prices range from $15 to $179. See THE TABLE for...
  11. A

    Flash OEM (IBM) Mellanox ConnectX-3 EN with stock firmware?

    Awesome, I added it to my mirror, too.
  12. A

    Flash OEM (IBM) Mellanox ConnectX-3 EN with stock firmware?

    Awesome! Thanks! The pro firmware link is broken, but the normal one worked. I mirrored it, along with all of the cx354/3a bin's from the Mellanox website, on my google drive.
  13. A

    Flash OEM (IBM) Mellanox ConnectX-3 EN with stock firmware?

    Is the mlx file and ini necessary, or can just a bin work? It looks like izx's instructions require the mlx and ini, while kiteboarder's go straight to the bin. It looks like Mellanox has removed all of their custom firmware downloads from all sites now :( , so I can't find the mlx and ini...
  14. A

    QDR Mellanox Performance Test Results with Varying BAR-space Size

    Answer to question 1 here. So the answer is yes. In fact 25.6 Gbit/s is the fastest expected bandwidth for QDR. The key is this: PCI_LANES(8)*PCI_SPEED(5)*PCI_ENCODING(0.8) = 32 Gb/s (that's where that number comes from). 32*PCI_HEADERS(128/152)*PCI_FLOW_CONT(0.95) = 25.6 Gb/s. My system is...
  15. A

    Infiniband PCIe card preventing boot in one server but not another

    I created a separate thread with the performance tests of the two firmwares. There was no significant difference.
  16. A

    QDR Mellanox Performance Test Results with Varying BAR-space Size

    System: Desktop (headnode): I7-5960X, GA-X99-SLI motherboard Server (slave nodes): 6027TR-HTR (four X9DRT-HF). Only using one node for this testing. 2x E5-2667 V2 OS: CentOS 7.5 with Infiniband Support and various other packages. No custom OFED installs. HCAs: One Sun/Oracle X4242A QDR...
  17. A

    Infiniband PCIe card preventing boot in one server but not another

    Which motherboard does it not work on? Did you try increasing the BAR space size?
  18. A

    Infiniband PCIe card preventing boot in one server but not another

    Here is my guide for flashing firmware to these cards. It pulls from various sources and covers flashing firmware with and without psid changes. Also summarizes everything I've done about this specific issue.
  19. A

    Infiniband PCIe card preventing boot in one server but not another

    So here the ODM is Mellanox, and the OEMs are Sun/Oracle, IBM, HP, Dell, etc? Nice catch with HP increasing the BAR size. I wonder if 32MB would work without turning on above 4G decoding. I bet most motherboards have a default upper limit on BAR size. So the versions on the broadcom/emulex...