FreeNas/TrueNas RDMA Support (FR for voting)

tsteine

Active Member
May 15, 2019
119
52
28
@BoredSysadmin This is very interesting.

I've personally abandoned Truenas Scale in the homelab in favor of running openzfs on my linux distro of choice, there is no reason why I can't simply test this, and see whether iSER has tangible benefits for iSCSI, or nfs over rdma.
 
  • Like
Reactions: BoredSysadmin

i386

Well-Known Member
Mar 18, 2016
2,981
946
113
33
Germany
in a homelab its probably not really relevant with RDMA
When I copy files via robocopy from my workstation to my windows file server the server cpu stays at ~2.4GHz (it's a 1620v3 with 3.5GHz clock), so it probably saves some watts here and there :D
 

Rand__

Well-Known Member
Mar 6, 2014
6,081
1,458
113
If anyone has positive (or negative ) experience or expectations regarding RDMA then now might be a good time to chime in ...

He explicitly asked for it, even on non TN systems, so ...
 

tsteine

Active Member
May 15, 2019
119
52
28
@Rand__

Judging from the the two posts you've gotten out of them, it seems to me like they haven't actually bothered testing RDMA themselves.
 

Rand__

Well-Known Member
Mar 6, 2014
6,081
1,458
113
Well I am not sure they didnt test it, it sounds rather like they did test with a certain setup and it did not yield significant improvement (which is o/c exactly what they say).
Now whether that was a best or worst case scenario test (or if its was even relevant to the potential improvements RDMA could bring) i don't know .
 

efschu3

Member
Mar 11, 2019
88
30
18
Couldn't let it go;)

I could Not let it Go too. Especialy when they Switched to Debian, the "extra" Work to be done would only be a few minutes. But I dont have the testlab for doing this by myself right now.

But I told them and pointed them on how less Work IT would be.

Hope they finally understand the benefit of RDMA.
 
Last edited:
  • Like
Reactions: RageBone and Rand__

efschu3

Member
Mar 11, 2019
88
30
18
Well, it seems we must write them a step by step guide for dummies how to Install rdma-core package and enable iser Portal on target.

I dunno why they are so lazy to "activate" an already existing and already integrated Feature.

When I get back my testlab I will write them a "Patch" o_O:rolleyes::oops::eek::cool:
 

i386

Well-Known Member
Mar 18, 2016
2,981
946
113
33
Germany
I dunno why they are so lazy to "activate" an already existing and already integrated Feature.
Probably because of their customers? I think ix customers are the same people that would use qnap or synology stuff, but want (or need) it to be "free" (like freedom, not like free beer :D). And these people don't need/know about rdma :D

But maybe I'm wrong...
 

PigLover

Moderator
Jan 26, 2011
3,050
1,350
113
@i386 the lack of demand may be part of it (though I'd argue most of IX main customer base is much more savvy than you suggest).

That said, the biggest reason is likely related to testing, tools and delivery practices. Once IX lists RDMA as 'supported' they have a responsibility to make sure it works. While turning on the existing upstream code may be easy the effort to integrate testing and validation into their CICD tool chains is non trivial. And actually testing RDMA right requires quite a bit of investment in test target equipment and tools.

Lastly, RDMA is of limited value to the masses - but almost limitless value to the small set of customers that really need it (need, not want...). This is exactly the kind of feature a vendor may want to hold back for a paid feature with later upsell opportunities.
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
786
291
63
Probably because of their customers? I think ix customers are the same people that would use qnap or synology stuff, but want (or need) it to be "free" (like freedom, not like free beer :D). And these people don't need/know about rdma :D

But maybe I'm wrong...
I disagree, my company went with two iX X20-HA systems for secondary storage due to a combination of support for HA, cost, and support. Qnap and Synology STILL don't have 24/7 support and without it, I'd personally NOT consider ANY storage system in an enterprise.
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
786
291
63
Lastly, RDMA is of limited value to the masses - but almost limitless value to the small set of customers that really need it (need, not want...).
I am curious to learn more about these cases, I know that the NEED for RDMA is likely to be driven by the need to have ultra-low latency, but what are specific use cases?
 

i386

Well-Known Member
Mar 18, 2016
2,981
946
113
33
Germany
I am curious to learn more about these cases, I know that the NEED for RDMA is likely to be driven by the need to have ultra-low latency, but what are specific use cases?
No matter what uses cases I think of it always ends at persisting data -> my answer would be "fast" storage (low latency, high throughput) for databases, message brokers and stuff like that