ESXi, OmniOS VM and problem with portforwarding (TCP_NODELAY: Invalid argument)


Oct 24, 2012
Hi Guys, I have successfully installed ESXi and then OmniOS VM (with napp-it –*thanks Gea!)

Im trying to SSH with portforwarding to the OmniOS VM (with two aggregated passthrough NICs), but I keep getting "sshd[695]: [ID 800047 auth.error] error: setsockopt TCP_NODELAY: Invalid argument" in messages. Normal SSH works fine.

I try to connect with:
ssh -L 4200:localhost:4243 username@server.hostname
It connects, but when my CrashPlan clients try connects I get this in messages:
Jun 29 22:42:43 hostname sshd[695]: [ID 800047 auth.error] error: setsockopt TCP_NODELAY: Invalid argument
Jun 29 22:45:35 hostname last message repeated 1 time
Jun 29 22:52:50 hostname sshd[1081]: [ID 800047 auth.error] error: setsockopt TCP_NODELAY: Invalid argument
It seems like the CrashPlan service is listening on port 4242
#netstat -na | grep LISTEN | grep 42
      *.4242               *.*                0      0 65880      0 LISTEN
::ffff:                   *.*                             0      0 128000      0 LISTEN
      *.4242                            *.*                             0      0 65880      0 LISTEN

I have tried to set
AllowTcpPortforwarding yes 
GatewayPorts yes
in sshd_config, but none of the above works.

Current sshd_config:
# Copyright (c) 2001, 2010, Oracle and/or its affiliates. All rights reserved.
# Configuration file for sshd(1m) (see also sshd_config(4))

# Protocol versions supported
# The sshd shipped in this release of Solaris has support for major versions
# 1 and 2.  It is recommended due to security weaknesses in the v1 protocol
# that sites run only v2 if possible. Support for v1 is provided to help sites
# with existing ssh v1 clients/servers to transition. 
# Support for v1 may not be available in a future release of Solaris.
# To enable support for v1 an RSA1 key must be created with ssh-keygen(1).
# RSA and DSA keys for protocol v2 are created by /etc/init.d/sshd if they
# do not already exist, RSA1 keys for protocol v1 are not automatically created.

# Uncomment ONLY ONE of the following Protocol statements.

# Only v2 (recommended)
Protocol 2

# Both v1 and v2 (not recommended)
#Protocol 2,1

# Only v1 (not recommended)
#Protocol 1

# Listen port (the IANA registered port number for ssh is 22)
Port 22

# The default listen address is all interfaces, this may need to be changed
# if you wish to restrict the interfaces sshd listens on for a multi homed host.
# Multiple ListenAddress entries are allowed.

# IPv4 only
# IPv4 & IPv6
#ListenAddress ::

# If port forwarding is enabled (default), specify if the server can bind to
# This allows the local port forwarding to work when connections are received
# from any remote host.
#GatewayPorts no #default
GatewayPorts yes

# X11 tunneling options
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes

# The maximum number of concurrent unauthenticated connections to sshd.
# start:rate:full see sshd(1) for more information.
# The default is 10 unauthenticated clients.
#MaxStartups 10:30:60

# Banner to be printed before authentication starts.
Banner /etc/issue

# Should sshd print the /etc/motd file and check for mail.
# On Solaris it is assumed that the login shell will do these (eg /etc/profile).
PrintMotd no

# KeepAlive specifies whether keep alive messages are sent to the client.
# See sshd(1) for detailed description of what this means.
# Note that the client may also be sending keep alive messages to the server.
KeepAlive yes

# Syslog facility and level 
SyslogFacility auth
LogLevel info

# Authentication configuration

# Host private key files
# Must be on a local disk and readable only by the root user (root:sys 600).
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key

# Length of the server key
# Default 768, Minimum 512
ServerKeyBits 768

# sshd regenerates the key every KeyRegenerationInterval seconds.
# The key is never stored anywhere except the memory of sshd.
# The default is 1 hour (3600 seconds).
KeyRegenerationInterval 3600

# Ensure secure permissions on users .ssh directory.
StrictModes yes

# Length of time in seconds before a client that hasn't completed
# authentication is disconnected.
# Default is 600 seconds. 0 means no time limit.
LoginGraceTime 600

# Maximum number of retries for authentication
# Default is 6. Default (if unset) for MaxAuthTriesLog is MaxAuthTries / 2
MaxAuthTries  →  6
MaxAuthTriesLog  →  3

# Are logins to accounts with empty passwords allowed.
# If PermitEmptyPasswords is no, pass PAM_DISALLOW_NULL_AUTHTOK 
# to pam_authenticate(3PAM).
PermitEmptyPasswords no

# To disable tunneled clear text passwords, change PasswordAuthentication to no.
PasswordAuthentication yes

# Are root logins permitted using sshd.
# Note that sshd uses pam_authenticate(3PAM) so the root (or any other) user
# maybe denied access by a PAM module regardless of this setting.
# Valid options are yes, without-password, no.
PermitRootLogin no

# sftp subsystem
Subsystem  →  sftp  →  internal-sftp

# SSH protocol v1 specific options
# The following options only apply to the v1 protocol and provide
# some form of backwards compatibility with the very weak security
# of /usr/bin/rsh.  Their use is not recommended and the functionality
# will be removed when support for v1 protocol is removed.

# Should sshd use .rhosts and .shosts for password less authentication.
IgnoreRhosts yes
RhostsAuthentication no

# Rhosts RSA Authentication
# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts.
# If the user on the client side is not root then this won't work on
# Solaris since /usr/bin/ssh is not installed setuid.
RhostsRSAAuthentication no

# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication.
#IgnoreUserKnownHosts yes

# Is pure RSA authentication allowed.
# Default is yes
RSAAuthentication yes

#portforwarding #non-default
AllowTcpForwarding yes
Last edited:


Oct 24, 2012
I managed to fix the problem. After alot of googling, irc, etc. I suddenly remembered I experienced the same error on Nexenta. The error is due to IPV6.

You can fix it by disabling ipv6 in the sshd, and here is what you do:

Modifiy: /lib/svc/method/sshd
Find the line "/usr/lib/ssh/sshd" and change it to "/usr/lib/ssh/sshd -4"

Then edit "/etc/ssh/sshd_config" and comment
out "ListenAddress ::" and uncomment "ListenAddress"

Execute "svcadm restart ssh" and you are good to go.