Mellanox IS5030 - Managed QDR Infiniband Switch Write-up

poto

Active Member
May 18, 2013
199
61
28
Another factor may be the benchmark used - I don't believe iperf supports rdma extensions. IOmeter with network target may show better results on latency and throughput.
 
  • Like
Reactions: Dajinn

Chuntzu

Active Member
Jun 30, 2013
383
98
28
Did you ever get this sorted out, if you need any help let me know I have ib working in windows server 2012r2 and 2016 tp5

Sent from my SM-N920T using Tapatalk
 

ServerMonkey

New Member
Nov 13, 2017
2
0
1
52
I know this is an old thread. Anyone got anyway to get the latest licenses etc for the Mellanox IS 5030? We just want support for VLAN and multicast IGMP. Please share your experience get this switch to work. Thanks.
 

Rus Taliss

New Member
Sep 14, 2017
4
0
1
55
Did you ever get this sorted out, if you need any help let me know I have ib working in windows server 2012r2 and 2016 tp5

What about on CentOS7.3 .. a main issue is sorting why 7.2 has IB performance issues in datagram mode (apparently not on Connect mode) and to what extent 7.3 has solved this .. have 2 identical Servers , not sure what the best module ordering may be, so as not to affect / break dependency on a co-located Cilk install (for optimised PHI libs). I suppose a best approach is to install 7.2 / 7.3 on a side by side basis and talk to a third Iperf Server .. OK these are available with ConnectX2 or 3 (i hear X3 degrade IB perf , ( in X2 > X2 // X2 > X3 ) what agnostic OS to mount on the 3rd Server ? and what loading to apply? Certainly like some assist with this.. if u have the time.. even space, should u prefer to have hardware on the bench .. (tho remote install works, why pay 24/7 power bills )

A centos7.3 with the latest Lustre 2.10 is tempting if only to play with the header addressable stack that promises a per-layer vector of function pointers .. finally surfacing after 23 years..

Index of /public/lustre/lustre-2.10.0/el7.3.1611/server/RPMS/x86_64

In this approach, instances of key entity types (files, pages, locks, etc.) are represented as a header, containing attributes shared by all layers. Each header contains a linked list of per-layer `slices',
each of which contains a pointer to a vector of function pointers. Generic operations on layered objects are implemented by going through the list of slices and invoking the corresponding function from the operation vector at every layer.
 

Rus Taliss

New Member
Sep 14, 2017
4
0
1
55
I know this is an old thread. Anyone got anyway to get the latest licenses etc for the Mellanox IS 5030? We just want support for VLAN and multicast IGMP. Please share your experience get this switch to work. Thanks.
SwitchX-interfaces/SwitchX-2 at master · Mellanox/SwitchX-interfaces · GitHub (not IS5030 Switch)

http://www.mellanox.com/related-docs/prod_management_software/MLNX-OS_VPI_v3_6_3004_UM.pdf

Annoying how quickly support is withdrawn (or hidden) im sure its all there in IB spec , from switches. Think to jump into SwitchX2 fabrics simply from the support side.. heres what i found in Mell, as to the last OFED to support X2 VPI : (correct if im wrong) , wont do RoCE but who needs Eth in cluster .. just starting out at : InfiniBand Trade Association: Home or

InfiniBand Solutions: IBTA - InfiniBand Trade Association

as far as IB goes , doesnt 5030 managed IB switch come with 40Gb ports enabled ? (mine are OS) there is mention of Mellanox API, a search brings a few card related mods (FPGA in this case)

cpldupdate tool allows the user to program on-board CPLDs on supporting mellanox products. The on-board CPLDs, as well as the core engine for programing them, are provided by Lattice Semiconductors. cpldupdate accepts VME files as input and programs the appropriate CPLD on the device. The CPLD ID is embedded in the VME file. The user need not be aware of the board composition "

The ispVM Embedded software, provided as source code in C, interprets the VME data to manipulate the JTAG signals of connected target devices. cpldupdate tool was provided by Lattice Semiconductors and was modified to fit Mellanox needs. " (only back to SwitchX / X2 / Spectrum) as with all >MFT 4.0 + revs. Only the 4.0 supports InfiniScale IV (incl IS50XX series ) at least according to the table. My switches are halfway around the world.. any volunteers for remote install, welcome.

http://www.mellanox.com/page/management_tools

http://www.mellanox.com/related-docs/MFT/MFT_user_manual_4_7_0.pdf p 109

http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_Release_Notes_3_4-2_1_8_0.pdf

whereas what u need is the latest 4 rel with : HowTo Configure SR-IOV for ConnectX-3 with KVM ... | Mellanox Interconnect Community
 
Last edited:

Johndrell

New Member
Jan 15, 2018
1
0
1
37
I know this thread is really old now but I thought others should know. I purchased this same switch, mine did not include the entitlement license on the underside of the pullout tab and Mellanox support will no longer lookup the basic license for you, they now require you to purchase a new one at just over $2000.00.
 

Derwood

Member
May 22, 2019
46
0
6
This is a really good thread, some wonderful replies I'm now stuck with deciding between the IBM Mellanox IV IS5030 or the voltaire 4036 GRID director only about $40~$45 difference in price and an interface too I assume.

I also think the drops in transfer rate over a period regardless of the data is maybe to do with buggy drivers?
 

Derwood

Member
May 22, 2019
46
0
6
I know this thread is really old now but I thought others should know. I purchased this same switch, mine did not include the entitlement license on the underside of the pullout tab and Mellanox support will no longer lookup the basic license for you, they now require you to purchase a new one at just over $2000.00.
The fella that created this review/post mentioned it's burnt in to the firmware, I'm sure he mentioned flicking the reset button for a period, resetting to factory defaults he then managed to extract the license number?

Was this not the same for yourself I assume?
 

EdwardTwiss

New Member
Nov 16, 2016
5
2
3
46
Does anyone know how to get these switches to serve DHCP addresses to IB through eth0?
Do I need to have a VPI license on the switch?

The fella that created this review/post mentioned it's burnt in to the firmware, I'm sure he mentioned flicking the reset button for a period, resetting to factory defaults he then managed to extract the license number?

Was this not the same for yourself I assume?