Problems accessing OmniOS repository

[TechTrend]

Are there any recent certificate changes or restrictions accessing the main OmniOS repository using pkg from r151046?

I periodically update VMs running OmniOS r151046 and napp-it on ESXi 7.0.3 using the OmniOS main repository at pkg.omnios.org. Yet for the past couple of days, access to that repository using pkg doesn't seem available from OmniOS VMs. Invoking 'pkg update' produces no output and eventually times out. An openssl client test from OmniOS r151046 behaves similarly.

# openssl version
OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024)
# openssl s_client -connect pkg.omnios.org:443
CONNECTED(00000004)
^C

Those OmniOS r151046 VMs can access the OmniOS mirror (-m) repository. Unfortunately, that mirror can't be used as origin (-g) publisher for pkg.

# openssl s_client -connect us-west.mirror.omnios.org:443
CONNECTED(00000004)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = us-west.mirror.omnios.org
verify return:1
---
Certificate chain
0 s:CN = us-west.mirror.omnios.org
i:C = US, O = Let's Encrypt, CN = R3
a: PKEY: id-ecPublicKey, 256 (bit); sigalg: RSA-SHA256
v:NotBefore: Apr 18 03:00:30 2024 GMT; NotAfter: Jul 17 03:00:29 2024 GMT
1 s:C = US, O = Let's Encrypt, CN = R3
i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
a: PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
v:NotBefore: Sep 4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIEqjCCA5KgAwIBAgISBGkJagZ3juxArF29gLmmkwjMMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
...

Other Linux VMs in the same subnet can access the main OmniOS repository.

# openssl version
OpenSSL 1.1.1 11 Sep 2018
# openssl s_client -connect pkg.omnios.org:443
CONNECTED(00000005)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = pkg.omnios.org
verify return:1
---
Certificate chain
0 s:CN = pkg.omnios.org
i:C = US, O = Let's Encrypt, CN = R3
1 s:C = US, O = Let's Encrypt, CN = R3
i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFwzCCBKugAwIBAgISBAzx3N9vKryFs8EvgVDaUMLpMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
...

Thanks.

 

[TechTrend]

The OpenSSL version installed appears to be the cause. On that same subnet there is an older OmniOS VM that has been upgraded many times. It is at r151046 but still has /opt/local/bin:/opt/local/sbin in front of the $PATH, causing it to use an older OpenSSL version 1.0.2. With that version, the connection to pkg.omnios.org works properly and 'pkg update' works fine.

Last login: Tue May 14 22:02:37 2024 from x.x.x.x
OmniOS r151046 omnios-r151046-d2b54a0125 February 2024
# type openssl
openssl is /opt/local/bin/openssl
# openssl version
OpenSSL 1.0.2d 9 Jul 2015
# type openssl-3
openssl-3 is /usr/bin/openssl-3
# openssl-3 version
OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024)
# openssl s_client -connect pkg.omnios.org:443
CONNECTED(00000004)
depth=1 C = US, O = Let's Encrypt, CN = R3
verify error:num=20:unable to get local issuer certificate
---
Certificate chain
0 s:/CN=pkg.omnios.org
i:/C=US/O=Let's Encrypt/CN=R3
1 s:/C=US/O=Let's Encrypt/CN=R3
i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFwzCCBKugAwIBAgISBAzx3N9vKryFs8EvgVDaUMLpMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
...

Last updates of the other r151046 VMs were in December and worked fine. Maybe an OpenSSL change installed with those December updates caused this issue? Yet the OmniOS main repository should probably allow access using the OpenSSL 3.0.13 in the current OmniOS LTS (r151046).

The OmniOS mirror repository allows access using that OpenSSL 3.0.13, although it is missing some content to allow using it as origin publisher for pkg.

 

[gea]

Can you access OmniOS r151046 core via Browser?
Is your system date correct (but this should only give a certificate error)?

 

[TechTrend]

Thanks for your suggestions.

OmniOS VMs on that site are headless, e.g. no GUI browser. I can access OmniOS r151046 core from other VMs with browsers in that subnet (screenshot attached).

Yes, the system date is correct.

 

[gea]

OmniOS blocks ping but you can try a
traceroute pkg.omnios.org to check network connectivity.

This should at least give the path up to eth zuerich
..

5 swiix3-10ge-0-0-0-22-1.switch.ch (194.42.48.11) 10.517 ms 10.553 ms 10.557 ms
6 swiEZ2-B3.switch.ch (130.59.36.176) 13.702 ms 11.896 ms 12.032 ms
7 swiEZ3-B1.switch.ch (130.59.36.126) 10.979 ms 11.030 ms 10.729 ms
8 rou-gw-lee-tengig-to-switch.ethz.ch (192.33.92.1) 10.669 ms 11.607 ms 10.468 ms
9 rou-fw-rz-rz-gw.ethz.ch (192.33.92.169) 9.094 ms 9.452 ms 9.553 ms
.
.

 

[gea]

For anyone with this problem, follow

Topicbox

 

[TechTrend]

Thanks for pointing out that Topicbox thread. The fix suggested was to lower the MTU to the destination network for the OmniOS repository. It worked.

route -p add 129.132.2.0/24 <your default router> -mtu 1400

 

[gea]

Jumboframes have the problem that any unit in the communication path must support. For external connections outside your control a simple switch replacement at any point between you and the destination server may be enough to stop connectivity.

 

[lexustwice]

OmniOS blocks ping but you can try a
traceroute pkg.omnios.org to check network connectivity.

This should at least give the path up to eth zuerich
..

5 swiix3-10ge-0-0-0-22-1.switch.ch (194.42.48.11) 10.517 ms 10.553 ms 10.557 ms
6 swiEZ2-B3.switch.ch (130.59.36.176) 13.702 ms 11.896 ms 12.032 ms
7 swiEZ3-B1.switch.ch (130.59.36.126) 10.979 ms 11.030 ms 10.729 ms
8 rou-gw-lee-tengig-to-switch.ethz.ch (192.33.92.1) 10.669 ms 11.607 ms 10.468 ms
9 rou-fw-rz-rz-gw.ethz.ch (192.33.92.169) 9.094 ms 9.452 ms 9.553 ms
. geometry dash world
.

Thanks gea for detailed instructions.