Recent content by mpogr

  1. M

    CX3 iSER

    @efschu2 , the bench results at the end, you have 3 ATTO graphs, what's the difference between them? Also, why did you insist on sync=always? This totally destroys write performance for me. Below are my graphs with sync=standard and sync=always for comparison.
  2. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    Not at all. You can add multipathing the same way as for iSCSI. Have a look at VMware documentation for iSER and you'll see all these details there.
  3. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    iSER initiator setup for VSphere 6.7 is very straightforward and doesn't deal with any particulars of RoCE, as it's all high level. Look up VMware documentation for this. What are you planning to use as your iSER target? I use SCST on CentOS (over ZFS) and, again, I don't need to deal with any...
  4. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    Yes, I meant RoCE, and, indeed, CX-3 supports v1, but only CX-3 Pro and up support v2, which is quite a bit of a difference latency-wise. I have done some measurements with my setup using SRP a while ago, now need to redo the same baseline to see how things are different with iSER, but, from...
  5. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    Guys, is there any reasonable (price-wise) substitute to CX-3 cards available now in the wild? My CX-3s are fine, but they don't perform with RDMA-over-IP as well as newer cards would, impacting iSER performance comparing to what SRP used to provide... Has to be at least 56Gbps in EN mode though...
  6. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    I'm not aware of any overviews, but, AFAIK, if you want to stick with the recent versions of ESXi (6.5 and above), your only option is Linux with ZoL and iSER (I use SCST implementation, but the built in should work as well). Edit: and yes, it is slower than pre-6.5 SRP had been...
  7. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    A remark on IPoIB vs EN mode: everything else being equal (FDR and 56GbpE), you get SIGNIFICANTLY less network throughput in IPoIB mode comparing to native EN. And when I say significantly, I mean 1/2 the throughput as the best case scenario. That said, as long as storage speed is the main...
  8. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    I think point in case here is one shouldn't reset the env variables in UBoot, otherwise bad things happen. Can someone with a working switch send this fellow their output of printenv? I would do it myself, but travelling at the moment... Thanks!
  9. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    The guide is available, no need to worry. Just PM me. As for the "right or wrong", just avoid buying the SX6XX5 models (which don't have a CPU) and you will be fine, as long as it's EMC. If it's Dell or HP, then it's even better, as you won't need to do the conversion.
  10. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    Sorry mate, I'll do when I reboot my switch. I don't do this very often...
  11. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    @Hindsight this is strange. The only case I had such a problem was when I tried putting CX-3 Pro firmware on my CX-3 (non-Pro) cards. Then they lost 56Gbps speed. But, otherwise, never had issues like this...
  12. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    As @Hindsight pointed out, you need to run opensm first. Make sure it's configured properly. The lights will come on then. If anything else fails, you can also run flint on the switch itself. PM me for details.
  13. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    The short answer is no, at least not easily. PM me for the longer answer...
  14. M

    Beware of EMC switches sold as Mellanox SX6XXX on eBay

    You can't enable both SM and Gateway at the same time...