FYI.. OmniOS OpenIndiana IPoIB config

[zane]

I had a little trouble finding this info just though I would post it up. Here is my config. Got info from here Please add if I'm missing something.

root@SAN-0B:~# dladm show-ib
LINK         HCAGUID         PORTGUID        PORT STATE  PKEYS
ibp0         2C903002B170E   2C903002B170F   1    up     FFFF
ibp1         2C903002B170E   2C903002B1710   2    up     FFFF

root@SAN-0B:~# dladm show-phys
LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
nge0         Ethernet             up         1000   full      nge0
nge2         Ethernet             down       0      unknown   nge2
ibp0         Infiniband           up         32000  unknown   ibp0
ibp1         Infiniband           up         32000  unknown   ibp1
nge3         Ethernet             down       0      unknown   nge3
nge1         Ethernet             down       0      unknown   nge1

root@SAN-0B:~# dladm show-ib
LINK         HCAGUID         PORTGUID        PORT STATE  PKEYS
ibp0         2C903002B170E   2C903002B170F   1    up     FFFF
ibp1         2C903002B170E   2C903002B1710   2    up     FFFF

root@SAN-0B:~# dladm create-part -l ibp0 -P 0xFFFF pFFFF.ibp0

root@SAN-0B:~# dladm create-part -l ibp1 -P 0xFFFF pFFFF.ibp1

root@SAN-0B:~# dladm show-part
LINK         PKEY  OVER         STATE    FLAGS
pFFFF.ibp0   FFFF  ibp0         unknown  ----
pFFFF.ibp1   FFFF  ibp1         unknown  ----

root@SAN-0B:~# ipadm create-addr -T static -a 10.200.200.200/24 pFFFF.ibp0/v4

root@SAN-0B:~# ipadm create-addr -T static -a 10.200.200.201/24 pFFFF.ibp1/v4

root@SAN-0B:~# ipadm show-if
IFNAME     STATE    CURRENT      PERSISTENT
lo0        ok       -m-v------46 ---
nge0       ok       bm--------46 -46
nge1       failed   bm--------46 -46
nge2       failed   bm--------46 -46
nge3       failed   bm--------46 -46
pFFFF.ibp0 ok       bm--------46 -46
pFFFF.ibp1 ok       bm--------46 -46

 

[cactus]

The Solaris docs are still a good place to get information also.

 

[sarakhan]

You should easily be able to source a 2U chassis (used SC826TQ with 12 3.5" drive slots and PSUs run ~$200 on eBay right now). Get a single-socket 2011 motherboard - probably the older Sandy/Ivy bridge version to save some money on memory (DDR4 is still too much of a price premium) and a low scale CPU and you should be able to stay inside your $1k budget.

 

[markpower28]

I had a little trouble finding this info just though I would post it up. Here is my config. Got info from here Please add if I'm missing something.

root@SAN-0B:~# dladm show-ib
LINK         HCAGUID         PORTGUID        PORT STATE  PKEYS
ibp0         2C903002B170E   2C903002B170F   1    up     FFFF
ibp1         2C903002B170E   2C903002B1710   2    up     FFFF

root@SAN-0B:~# dladm show-phys
LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
nge0         Ethernet             up         1000   full      nge0
nge2         Ethernet             down       0      unknown   nge2
ibp0         Infiniband           up         32000  unknown   ibp0
ibp1         Infiniband           up         32000  unknown   ibp1
nge3         Ethernet             down       0      unknown   nge3
nge1         Ethernet             down       0      unknown   nge1

root@SAN-0B:~# dladm show-ib
LINK         HCAGUID         PORTGUID        PORT STATE  PKEYS
ibp0         2C903002B170E   2C903002B170F   1    up     FFFF
ibp1         2C903002B170E   2C903002B1710   2    up     FFFF

root@SAN-0B:~# dladm create-part -l ibp0 -P 0xFFFF pFFFF.ibp0

root@SAN-0B:~# dladm create-part -l ibp1 -P 0xFFFF pFFFF.ibp1

root@SAN-0B:~# dladm show-part
LINK         PKEY  OVER         STATE    FLAGS
pFFFF.ibp0   FFFF  ibp0         unknown  ----
pFFFF.ibp1   FFFF  ibp1         unknown  ----

root@SAN-0B:~# ipadm create-addr -T static -a 10.200.200.200/24 pFFFF.ibp0/v4

root@SAN-0B:~# ipadm create-addr -T static -a 10.200.200.201/24 pFFFF.ibp1/v4

root@SAN-0B:~# ipadm show-if
IFNAME     STATE    CURRENT      PERSISTENT
lo0        ok       -m-v------46 ---
nge0       ok       bm--------46 -46
nge1       failed   bm--------46 -46
nge2       failed   bm--------46 -46
nge3       failed   bm--------46 -46
pFFFF.ibp0 ok       bm--------46 -46
pFFFF.ibp1 ok       bm--------46 -46

Mind to share partitions.conf?

 

[gea]

Info:

With the help of Frank,
i am on the way to include basic IB support in napp-it

First steps: napp-it 0.9f4 dev

 

[markpower28]

Info:

With the help of Frank,
i am on the way to include basic IB support in napp-it

First steps: napp-it 0.9f4 dev

That will be great!! Please keep us posted!!