Upgrade my network to SFP28 ....(w QSPF): Any reason to use Mellanox..?

[TrumanHW]

Are there advantages of a homogeneous InfiniBand //

If so, what IS infiniband...?

Amy other manufactures which provide better performance in their homogenous proprietary network..?

Network will consist of a FreeBSD (TrueNAS) server, MacOS peers

Thanks.

And it goes without saying I'd prefer to be under $200 per card unless there's great reason to splurge.

 

[i386]

Are there advantages of a homogeneous InfiniBand //

If so, what IS infiniband...?

Yes...
Infiniband is a protocol specification for extremely fast "messages" (small amount of data that is transmitted with the lowest possible latency) and high bandwidth (infiniband = infinite bandwidth, paralles lanes can be added to increase total bandwidth. "normal" ib stuff uses 1 or 4 data lanes, science and hpc stuff uses 12 or more lanes).
The specification describes interfaces in terms of verbs (like "write to memory x", "read from memory y"), the interfaces are implemented by a software layer like winofed/ofed (by mellanox) or proprietary software (I know only of ibm having their own implementation).

It's great for certain workloads, but a pain in the ass as a ethernet replacement. Unless you really need it (you will know when this happens) stick with "normal" ethernet and roce/iwarp.

Amy other manufactures which provide better performance in their homogenous proprietary network..?

There are other technologies but they are very specific and proprietary (for example slingshot from cray, now hpe) that won't work with esxi or your vanilla linux/bsd system.
Check the top 500 systems and their interconnects to see more of such protocolls/hardware.


Edit:fixed some typos

 

[TrumanHW]

Wow, outstanding information!!! Thank you!