Omni OS Replication

acmcool

Banned
Jun 23, 2015
611
76
28
37
Woodbury,MN
My replication tops at 375MB/s on a 10G connection with mtu 9000...
I have 2x10 drive raidz2 vdev's...The RAM is 32GB on both appliances..
Any ideas to improve performance?
 

whitey

Moderator
Jun 30, 2014
2,774
873
113
39
Here's some tuning variables I typically use on my OmniOS AIO's:

ndd -set /dev/vmxnet3s0 accept-jumbo 1
ndd -set /dev/ip ip_lso_outbound 0
ifconfig vmxnet3s0 mtu 9000
ipadm set-prop -p max_buf=4194304 tcp
ipadm set-prop -p recv_buf=1048576 tcp
ipadm set-prop -p send_buf=1048576 tcp

Also there is tuning on the NFS server/client variables if you're feelin' froggy :-D Well documented on the web but a lil' all over the place.
 
  • Like
Reactions: gb00s and RobertCO

whitey

Moderator
Jun 30, 2014
2,774
873
113
39
What exact performance gains did you see, curious to know a number and a lil' more on env.
 
  • Like
Reactions: RobertCO

RobertCO

New Member
Jan 2, 2022
6
1
3
What exact performance gains did you see, curious to know a number and a lil' more on env.
Necropost alert but it bugs me your question wasn't answered... I ran into this looking for help for SMB woes on r151038.

Thank you!

I have an Intel X540-T1 in my NAS running omnios-r151038 on a SuperMicro X9SCM-F mobo.

Pulling my hair out trying to get SMB to break 100MB/sec when copying files from windows 10 to the SMB share.

Apparently ip_lso_outbound is set to 1 by default on this release of omnios:

Code:
# ndd -get /dev/ip ip_lso_outbound
1
Created a file in /etc/rc3.d called S98ndd_lso and put the " ndd -set /dev/ip ip_lso_outbound 0 ".

Code:
# ls -l /etc/rc3.d/S98ndd_lso_outbound
-rwxr-xr-x 1 root root 35 Jan  9 09:52 /etc/rc3.d/S98ndd_lso_outbound
# cat $_
ndd -set /dev/ip ip_lso_outbound 0

I also changed the buffers with ipadm as suggested (those changes were persistent).

5x speedup!!!

now hovers around 550MB/s (and sometimes breaks 600MB/s) when dragging and dropping large files from my W10 box to the SMB share (zfs pool) on my NAS.
 
  • Like
Reactions: gb00s