Network IPv6, firewalling, routing, oh my...

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

ArmedAviator

Member
May 16, 2020
91
56
18
Kansas
I'm expanding my knowledge and wanting to learn more about IPv6 in general and enable it's use in the home network.

I'm awfully confused on how it will work with my present network setup.

Keep in mind this is on a home network with a residential ISP. I am able to get a /59 of IPv6 addresses from DHCP6.

IPv4 only setup:
  • pfSense NAT router + DHCP + DNS + NTP
  • 4 VLANs with WAN access
    • iot - 10.1.3.0/24 - very limited access to WAN (cameras, smart thermostat,
    • app -10.1.26.0/24 - Physical and VM server "applications" access
    • mgmt - 10.1.1.0/24 - Management VLAN for switch and BMC/DRAC access
    • cli - 10.1.10.0/24 - Standard end-device access (laptops, cell phones, ipads, desktops
  • ICX 6610-48P switch as each VLAN's primary gateway for L3 routing on the switch
    • access-lists on each ve interface as firewall rules (VLAN to VLAN, VLAN/host to WAN, etc.)
So with this basic setup, RFC1918 IPv4 routing is all handled in the switch so I get 10Gbit and 40Gbit line speed on several servers and end-clients. pfSense primarily just does IPv4 NAT routing to the internet, thus also doing de-facto inbound traffic firewalling.

I got this far in enabling IPv6....
  • Acquired /59 subnet from ISP.
  • Set each pfSense VLAN interfaces to "Track Interface", selected the WAN interface to track, and assigned each VLAN a different IPv6 Prefix ID (1-4) (TOTALLY NOT SURE THIS IS RIGHT)
  • Only devices on one cli VLAN are getting IPv6 internet addresses
  • Enabled IPv6 rules to allow all outbound traffic on the cli VLAN.

The questions I have (more to come!):
  • How do I know what IPv6 address to give the ICX6610 interfaces?
  • How do I ensure the ICX 6610 will be the L3 router for the LAN/VLAN traffic?
  • Should I/how do I configure DHCP6 server on pfSense (currently hosts are getting IPv6 addresses without DHCP6 server enabled) for the above issues?


I'm really having trouble understanding taking IPv4 NAT practices and converting them to good IPv6 practices and keeping things protected (firewalled) and maintaining wire-speed performance.
 
Last edited:

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
Yup. pfSense still can't do DHCP-PD properly. OpnSense can do most things. You have the right idea for assigning prefixes, but pfSense doesn't support it and they aren't interested in fixing it. If you bring it up with them, they insist that your ISP is stupid and you should get a new one. Not an option for most of us.

DHCP6 isn't required on your internal network to get addresses. You can do it, and some clients support it, some don't. Android, for example, uses SLAAC only and ignores DHCP6. The default is usually unmanaged. The router announces itself on the network (Router Advertisements) and the clients see that, get the subnet, and assign themselves addresses based on local MAC and other things. That generally is "sticky" and will stay the same other than the prefix getting changed. Then another one usually based on random numbers for privacy extensions.

Assigning addresses to static equipment is easy, just pick one. But as soon as your prefix changes, it will break unless you can set up sub-delegations. IPv6 will detect if an address it wants to use is taken and pick a new one. As for routing on the switch, it probably doesn't support working with delegated prefixes.

So, if you want to use the native PD prefix from your ISP, you need to switch to OpnSense and let it do the routing. Another option is to use HE.net and get a static /48. Then you solve all those issues. But that's not local native IPv6, so it might not be how you want to do it. You can also use ULA for internal traffic (think 10.x type addresses in v6 land, but not exactly the same). And just use the native v6 for internet access.

As for security, the routing platforms tend to block all inbound connections, so you're good unless you open something up. The first thing that takes some getting used to, forget NAT exists for v6. NAT is a broken idea and can't die soon enough, so good riddance. Performance is easy, no NAT means packets just get firewalled and transit faster.
 

ArmedAviator

Member
May 16, 2020
91
56
18
Kansas
Yup. pfSense still can't do DHCP-PD properly. OpnSense can do most things. You have the right idea for assigning prefixes, but pfSense doesn't support it and they aren't interested in fixing it. If you bring it up with them, they insist that your ISP is stupid and you should get a new one. Not an option for most of us.
I had many reasons to switch to OPNSense and am spinning up a new VM with it right now. Should be live and replacing pfSense this evening.

DHCP6 isn't required on your internal network to get addresses. You can do it, and some clients support it, some don't. Android, for example, uses SLAAC only and ignores DHCP6. The default is usually unmanaged. The router announces itself on the network (Router Advertisements) and the clients see that, get the subnet, and assign themselves addresses based on local MAC and other things. That generally is "sticky" and will stay the same other than the prefix getting changed. Then another one usually based on random numbers for privacy extensions.
Without DHCP6, what is the method for keeping DNS records for each devices' IPv6 address?

Assigning addresses to static equipment is easy, just pick one. But as soon as your prefix changes, it will break unless you can set up sub-delegations. IPv6 will detect if an address it wants to use is taken and pick a new one. As for routing on the switch, it probably doesn't support working with delegated prefixes.

So, if you want to use the native PD prefix from your ISP, you need to switch to OpnSense and let it do the routing. Another option is to use HE.net and get a static /48. Then you solve all those issues. But that's not local native IPv6, so it might not be how you want to do it. You can also use ULA for internal traffic (think 10.x type addresses in v6 land, but not exactly the same). And just use the native v6 for internet access.
I'm trying to go native IPv6 so HE.net isn't exactly the route I want to go. It sounds like OPNSense will help me out, but the issue I have is I do NOT want OPNSense doing the local routing, only what's heading for the internet. OPNSense will not be able to achieve 10Gbps like the switch can.
 

ArmedAviator

Member
May 16, 2020
91
56
18
Kansas
I have now running OPNSense. It already is working better than pfSense. Got a /56 block from Spectrum, too.

So now I have set a unique IPv6 Prefix ID to each VLAN interface. The devices behind them are now getting IPv6 addresses. I see them show up in the IPv6 DHCP lease table, but IPv6 is not configurable unless I enable the "Manual configuration" option under each VLAN interface's IPv6 options.

Before I go clicking that, I would like to know how others set up IPv6 DHCP/DNS assignments - or other similar options.
 

ArmedAviator

Member
May 16, 2020
91
56
18
Kansas
Update:

I chose to manually configure DHCPv6. I have OPNSense offering the range :::::1000-2000 on each subnet.

OPNSense was giving out IPv6 addresses fine until I started changing things. All IPv6-enabled hosts are getting IPv6 addresses, however they don't appear to be leased from OPNSense. This is likely from the settings on the ICX 6610 - and I can't find very much documentation on these ipv6 options, so I'm guessing alot here.

Here's some configuration I have in the ICX:

Code:
hostname ks-icx-01
ip dhcp-client disable
ip dns server-address 10.1.1.254
ip route 0.0.0.0/0 10.1.199.2
!
ipv6 dns server-address 2605:xxxx:xxxx:3701:1cf3:c1ff:fe81:ed6b
ipv6 unicast-routing
ipv6 route ::/0 ve 199  fe80::847c:fdff:feef:a07d
Code:
interface ve 2
 port-name VLAN-VOIP
acl-logging
ip access-group 102 in
ip address 10.1.2.1 255.255.255.0
ip mtu 1500
ipv6 address 2605:xxxx:xxxx:3702::1/64
ipv6 enable
ipv6 mtu 1500
ipv6 dhcp-relay destination fe80::8f1:a0ff:fe79:3717 outgoing-interface ve 2
ipv6 nd managed-config-flag
ipv6 nd router-preference high
!
interface ve 3
port-name VLAN-IOT
acl-logging
ip access-group 103 in
ip address 10.1.3.1 255.255.255.0
ip mtu 1500
ipv6 address 2605:xxxx:xxxx:3703::1/64
ipv6 enable
ipv6 mtu 1500
ipv6 dhcp-relay destination fe80::84ce:79ff:fe4d:ef36 outgoing-interface ve 3
ipv6 nd managed-config-flag
ipv6 nd router-preference high
!
interface ve 4
port-name VLAN-SAN
ipv6 enable
ipv6 mtu 9000
ipv6 nd router-preference high
!
interface ve 5
port-name VLAN-MGMT
ip address 10.1.1.1 255.255.255.0
ip mtu 1500
ipv6 address 2605:xxxx:xxxx:3705::1/64
ipv6 enable
ipv6 mtu 1500
ipv6 dhcp-relay destination fe80::1cf3:c1ff:fe81:ed6b outgoing-interface ve 5
ipv6 nd managed-config-flag
ipv6 nd router-preference high
!
interface ve 10
port-name VLAN-CLI
ip address 10.1.10.1 255.255.255.0
ip mtu 1500
ipv6 address 2605:xxxx:xxxx:3710::1/64
ipv6 enable
ipv6 mtu 1500
ipv6 dhcp-relay destination fe80::84ce:79ff:fe4d:ef36 outgoing-interface ve 10
ipv6 nd managed-config-flag
ipv6 nd router-preference high
!
interface ve 26
port-name VLAN-APP
ip address 10.1.26.1 255.255.255.0
ip mtu 1500
ipv6 address 2605:xxxx:xxxx:3726::1/64
ipv6 enable
ipv6 mtu 1500
ipv6 dhcp-relay destination fe80::a4fc:cff:fe79:1293 outgoing-interface ve 5
ipv6 nd router-preference high
!
interface ve 199
port-name VLAN-TRNK
ip address 10.1.199.1 255.255.255.252
ip mtu 1500
ipv6 enable
ipv6 mtu 1500
ipv6 nd router-preference high
And here's the routing table of OPNsense:
Code:
rich@opns02:~ % netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            xx.xx.xx.xx.1      UGS      vtnet0
10.1.0.0/16        10.1.199.1         UGS      vtnet2
10.1.1.0/24        link#2             U        vtnet1
10.1.1.254         link#2             UHS         lo0
10.1.2.0/24        link#4             U        vtnet3
10.1.2.254         link#4             UHS         lo0
10.1.3.0/24        link#5             U        vtnet4
10.1.3.254         link#5             UHS         lo0
10.1.10.0/24       link#6             U        vtnet5
10.1.10.254        link#6             UHS         lo0
10.1.26.0/24       link#7             U        vtnet6
10.1.26.254        link#7             UHS         lo0
10.1.199.0/30      link#3             U        vtnet2
10.1.199.2         link#3             UHS         lo0
10.25.112.0/24     10.25.112.1        UGS      ovpnc1
10.25.112.1        link#12            UH       ovpnc1
10.25.112.2        link#12            UHS         lo0
xx.xx.xx.x.0/22    link#1             U        vtnet0
xx.xx.xx.x.165     link#1             UHS         lo0
127.0.0.1          link#9             UH          lo0

Internet6:
Destination                       Gateway                       Flags     Netif Expire
default                           fe80::201:5cff:fe77:6c46%vtnet0 UG     vtnet0
::1                               link#9                        UH          lo0
2605:xxxx:xxxx:3700::/56          fe80::768e:f8ff:fee7:b4b0%vtnet2 UGS   vtnet2
2605:xxxx:xxxx:3701::/64          link#2                        U        vtnet1
2605:xxxx:xxxx:3701:1cf3:c1ff:fe81:ed6b link#2                  UHS         lo0
2605:xxxx:xxxx:3702::/64          link#4                        U        vtnet3
2605:xxxx:xxxx:3702:8f1:a0ff:fe79:3717 link#4                   UHS         lo0
2605:xxxx:xxxx:3703::/64          link#5                        U        vtnet4
2605:xxxx:xxxx:3703:60b3:45ff:feb5:134f link#5                  UHS         lo0
2605:xxxx:xxxx:3710::/64          link#6                        U        vtnet5
2605:xxxx:xxxx:3710:84ce:79ff:fe4d:ef36 link#6                  UHS         lo0
2605:xxxx:xxxx:3726::/64          link#7                        U        vtnet6
2605:xxxx:xxxx:3726:a4fc:cff:fe79:1293 link#7                   UHS         lo0
2605:xxxx:xxxx:3799::/64          link#3                        U        vtnet2
2605:xxxx:xxxx:3799:847c:fdff:feef:a07d link#3                  UHS         lo0
2605:xxxx:xxxx:10:18da:b722:a00a:a2ba link#1                    UHS         lo0
fe80::%vtnet0/64                  link#1                        U        vtnet0
fe80::5ce8:2aff:fe74:c06b%vtnet0  link#1                        UHS         lo0
fe80::%vtnet1/64                  link#2                        U        vtnet1
fe80::1cf3:c1ff:fe81:ed6b%vtnet1  link#2                        UHS         lo0
fe80::%vtnet2/64                  link#3                        U        vtnet2
fe80::847c:fdff:feef:a07d%vtnet2  link#3                        UHS         lo0
fe80::%vtnet3/64                  link#4                        U        vtnet3
fe80::8f1:a0ff:fe79:3717%vtnet3   link#4                        UHS         lo0
fe80::%vtnet4/64                  link#5                        U        vtnet4
fe80::60b3:45ff:feb5:134f%vtnet4  link#5                        UHS         lo0
fe80::%vtnet5/64                  link#6                        U        vtnet5
fe80::84ce:79ff:fe4d:ef36%vtnet5  link#6                        UHS         lo0
fe80::%vtnet6/64                  link#7                        U        vtnet6
fe80::a4fc:cff:fe79:1293%vtnet6   link#7                        UHS         lo0
fe80::%lo0/64                     link#9                        U           lo0
fe80::1%lo0                       link#9                        UHS         lo0
fe80::98ef:24df:a1ca:870d%ovpnc1  link#12                       UHS         lo0
I don't know assigning globally addressable IPs is needed/smart/right on the switch. I'm also not confident the settings are correct, of course.

So to recap, I'm still trying to:
  • Assign DNS hostnames for each device's IPv6 address
  • Ensure all local traffic is routed through the ICX6610 switch for maximum performance Seems to be working correctly at present. All devices have the link local IPv6 of the switch as the default route and am getting wire-speed iperf3 tests across VLANs.
 
Last edited:

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
Sounds like you have a good start. The DNS thing is one of the things on v6 doesn't get handled well. The DHCP service isn't required to get addresses, so the server doesn't have the information.

The common suggestion is to use multicast DNS or dynamic DNS. This requires each machine to send updates. So you can't just set it up on the server and be done. I've seen some discussions about it but no easy solution yet.

I'm not sure what best practice is on assigning addresses to the switch. The general idea on v6 is that everything has a global address though, so it's likely fine. Just make sure you can't hit it from outside the network. The default firewall rules should prevent that.
 

ArmedAviator

Member
May 16, 2020
91
56
18
Kansas
I modified my network a bit since starting this process - OPNSwitch is now linked to the switch via a single IPv4 and a single IPv6 link. I tried this in the past but went back to VLANs connected to pfSense due to DHCP limitations of pfSense/OPNSense. I finally set up ISC DHCP server and BIND named for DHCP on a VM. ISC DHCP supports DHCP6, but I haven't yet set it up. My major concern is I'm not sure how long my IPv6 /56 will remain tied to me. My IPv4 changed for the first time in 2 years after changing the MAC address on the WAN port (new VM).

For now, IPv6 is working great. IPs are acquired via SLAAC on the VLANs that I had set an IPv6 address to on the switch's VLAN router-interface (int ve in the case of Brocade).

I still need to research the world of DHCP6 before DNS bindings will work. I don't want to go too in-depth here until I see how long I keep my current /56 assignment from Spectrum residential.
 

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
That's the thing that sucks about DHCP-PD. Your prefix can change. In practice they seem to stick pretty well unless you send a release. I can see why the ISPs like it, you request, you get a prefix that works. No problem. But when you are using those addresses inside the network, having the prefix change on you is really annoying.

I can see why they wouldn't want to, but it would be really nice if they would set something up like HE has. The default stays as DHCP-PD, but if you get on their website and request a prefix, they give you one that is static for the life of your account.

Another option is to work with the fact the IPv6 is designed to have many addresses per interface from the start. So use the ISP DHCP addresses for internet stuff and use ULA addresses for internal traffic. You can trust them not to change on you, so static DNS entries are not an issue. Use a ULA generator and you can be pretty sure you'll never have overlap with anyone else. Not a huge issue, but it's a bit like everyone using 192.168.1.1/24. If you ever set up a VPN or similar and they use the same range, you have to adjust things. If you want to try it, generate a range, then add an alias to OpnSense for a /64 and it will advertise on the interface so client machines will get an address in that prefix as well.
 

ArmedAviator

Member
May 16, 2020
91
56
18
Kansas
I've had success for the past few weeks now with native IPv6 on my network using SLAAC but now I want to get DHCP6 working.

ISC DHCP6 is giving out IPv6 addresses to the VLANs I have IPv6 enabled on. The addresses are always /128, however. For some reason, there's no /64 route for the devices, although from my reading of the Brocade IPv6 documentation, the Router Advertisements should take care of it by default.

What am I missing?

Here's an example of the interface config on the ICX6610:

Code:
SSH@ks-icx-01#show run int ve 10
interface ve 10
port-name VLAN-CLI
ip address 10.1.10.1 255.255.255.0
ip helper-address 1 10.1.26.3
ip mtu 1500
ip ospf area 0
ipv6 address 2605:aaaa:bbbb:7a10::1/64
ipv6 enable
ipv6 mtu 1500
ipv6 ospf area 0
ipv6 dhcp-relay destination 2605:aaaa:bbbb:7a26::3
ipv6 nd managed-config-flag
And here's the address and route I'm getting on my laptop using NetworkManager:

Code:
wlp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.1.10.11  netmask 255.255.255.0  broadcast 10.1.10.255
inet6 fe80::b68a:3ee9:f369:48c0 prefixlen 64 scopeid 0x20<link>
inet6 2605:aaaa:bbbb:7a10::b493 prefixlen 128 scopeid 0x0<global>
ether fc:f8:ae:7b:c1:13 txqueuelen 1000 (Ethernet)
RX packets 393137 bytes 424261474 (404.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 266482 bytes 169900507 (162.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

rich@zenith ~ $ route -6n
Kernel IPv6 routing table
Destination Next Hop Flag Met Ref Use If
2605:aaaa:bbbb:7a10::b493/128 :: U 600 1 0 wlp3s0
fe80::/64 :: U 600 1 0 wlp3s0
::/0 :: !n -1 1 0 lo
::1/128 :: Un 0 7 0 lo
2605:aaaa:bbbb:7a10::b493/128 :: Un 0 2 0 wlp3s0
fe80::b68a:3ee9:f369:48c0/128 :: Un 0 3 0 wlp3s0
ff00::/8 :: U 256 6 0 wlp3s0
::/0 :: !n -1 1 0 lo
 
Last edited:

ArmedAviator

Member
May 16, 2020
91
56
18
Kansas
I gave up on DHCP6 for the most part. I am using it on only a few client devices, but it seems picky with what hosts it will input into DNS for some odd reason. I'm reasonably happy with it for now however if my IPv6 delegation changes in the future from my residential ISP, it's going to be an annoying time logging into each host to change the static IP address.
 

juju

New Member
Sep 29, 2021
29
1
3
I gave up on DHCP6 for the most part. I am using it on only a few client devices, but it seems picky with what hosts it will input into DNS for some odd reason. I'm reasonably happy with it for now however if my IPv6 delegation changes in the future from my residential ISP, it's going to be an annoying time logging into each host to change the static IP address.
@ArmedAviator What is the state of your ipv6 experimentation now? I am headed down that path but seems like a lot of moving parts . I have a pfsense box doing just firewall duties with layer 3 routing on a brocade box and dhcp handled by separate KEA dhcp servers.