Looking for router advice before moving to AT&T fiber

[marcoi]

If you want to distribute out the static IPs, add another LAN interface with an IP in your static block, and connect your nodes to that interface. Create a firewall rule allowing traffic to those IPs in from the WAN interface. Haven't used pfSense for a couple years, so I'm not sure on exactly how to do it.

Have you tried speedtest.net? They have more local servers. I just got 430 on fast.com and 860 on speedtest.net, ton of stuff currently using the network.

yeah i did both test sites. I jsut deployed a new VM with fresh install of PfSense and base configured for pfatt. still cant get past the 500 range. I replugged in the att router and it local test hit 950/950 speeds.

Not sure what else to tweak here. but im starting to think it time to go back to original setup and deal with 8k double nat tables.

 

[Gene]

I did not know you could bypass these AT&T routers completely. Just ordered NVG589 to desolder (i have a Pace) for $19. Ill be dumping firmware for certs and going with supplicant method on my pfsense box

 

[fohdeesha]

yeah wow, I had no idea that cert progress was made. I'd be able to terminate their incoming fiber directly on an ICX and handle the 802.1x auth with said certs right onboard if it weren't for ATT using vlan 0, have yet to find a brocade/juniper/arista switch that will let me tag vlan 0, usually reserved for RSPAN or packet generation

 

[marcoi]

i went back to my original configuration for now. i might just order the certs usb off ebay and try the 2 nic method and see if i have better results.
the att RW would be ok if the 8k limit was bumped up. Anyone ever try to hack the FW to up the limit?

 

[mb300sd]

yeah wow, I had no idea that cert progress was made. I'd be able to terminate their incoming fiber directly on an ICX and handle the 802.1x auth with said certs right onboard if it weren't for ATT using vlan 0, have yet to find a brocade/juniper/arista switch that will let me tag vlan 0, usually reserved for RSPAN or packet generation

Yeah, I didn't know there was so much progress either. Just switched over to the wpa_supplicant method today. No IP change even with the MAC change

Wrote a config script for VyOS to enable EAPOL. T1466 Add EAPOL login support Might work on the EdgeRouters too.
Doing it on the post-config script breaks my VPN tunnels and pre-config is done before the MAC gets changed and if-names assigned.

i went back to my original configuration for now. i might just order the certs usb off ebay and try the 2 nic method and see if i have better results.
the att RW would be ok if the 8k limit was bumped up. Anyone ever try to hack the FW to up the limit?

I don't think anyone has, it would be on the DSLReports site if possible.

 

[zedkyuu]

yeah wow, I had no idea that cert progress was made. I'd be able to terminate their incoming fiber directly on an ICX and handle the 802.1x auth with said certs right onboard if it weren't for ATT using vlan 0, have yet to find a brocade/juniper/arista switch that will let me tag vlan 0, usually reserved for RSPAN or packet generation

Wow, I didn't realize they made SFP modules for GPON. (As in, I didn't know, and just did a search, and found at least a few...)

That said, one thing I never knew was whether the ONT you get from AT&T does anything special, or if it's just a simple bridge. You have to redo 802.1x auth whenever the Ethernet link from the ONT to your router goes down, so maybe it does?

FWIW, I did the BGW210 hack myself (soldering, baby!) and have this running off an EdgeRouter. VLAN 0 isn't a problem with it. Apparently, VLAN 0 is some kind of way to attach QoS bits to packets without VLAN, so maybe finding some way to enable QoS will do it as well?

 

[mb300sd]

I tried the GPON SFP a year or 2 ago but couldn't get it working. Managed to get into the ONT and clone it's SLID and serial number, but couldn't get any traffic to pass. Might tinker more once I switch to AT&T at my second house, they just laid fiber and I'm ready for gigabit VPN! Don't want to mess with the line the servers are connected to.

If the pings are low enough, it's gonna feel like one big LAN (well, most of my LAN is 10/40g, so maybe not) but still, should be able to stream 4K over SMB and do a ~1TB nightly off-site backup. Wonder if a Hyper-V live migration will work.

 

[Kev]

Anyone willing to put up a tutorial on /29 fixed IP, IPv6 handed to multiple subnets, etc for pFsense?

 

[Gene]

the ebay guy is out of certs so i'm getting the flash tool for the nvg589 cheapo i have. I'll keep an eye out for cheap att routers i can dump the certs on for others if he doesn't load up again

 

[mb300sd]

Just got fiber installed at the lake house today. Pulled the certs out of my BGW210 as soon as the tech left. Gigabit VPN is as great as I thought it was gonna be. Traceroute shows 2 hops between my routers, and 7ms pings through the tunnel. Now I need to upgrade the router here, maxing out CPU at 500mbit IPSec in one direction.

 

[ejecthawk]

Does anyone have the dumped certificate bypass running on a Edgerouter Lite? I'm following the devicelocksmith guide, but have come across an issue. When trying to manually start wpa_supplicant or on boot it immediately crashes:

ubnt@RT1:$ sudo journalctl -u wpa_supplicant-wired@eth0.service
-- Logs begin at Thu 2016-11-03 13:16:46 EDT, end at Thu 2019-06-27 22:18:47 EDT. --
Jun 27 20:55:01 RT1 systemd[1]: Started WPA supplicant daemon (interface- and wired driver-specific version).
Jun 27 20:55:02 RT1 systemd[1]: wpa_supplicant-wired@eth0.service: Main process exited, code=killed, status=4/ILL
Jun 27 20:55:02 RT1 systemd[1]: wpa_supplicant-wired@eth0.service: Unit entered failed state.
Jun 27 20:55:02 RT1 systemd[1]: wpa_supplicant-wired@eth0.service: Failed with result 'signal'.

 

[mb300sd]

Try starting it on the command line.

sudo /sbin/wpa_supplicant -dd -Dwired -ieth0 -c wpa_supplicant.conf

and see what you get.

 

[marcoi]

I got my wpa working on pfsense now. I still need to figure out how to pass out my static ips. Do I need an interface for each or can i use LAN and configured a gateway, then configure end device on the lan to pull a static ip?

 

[mb300sd]

You need an interface on your router configured with one of the static IPs. Configure all your hosts with a different IP and use that at the gateway, just like your regular LAN interfaces using private IPs, but using your public IPs instead.

For example, my setup
Router:
eth0: dhcp public IP
eth1: 10.1.1.1/24
eth2: xx.xx.xx.200/29

Server1:
IP: xx.xx.xx.201
Netmask: 255.255.255.248
Gateway: xx.xx.xx.200

Server2:
IP: xx.xx.xx.202
Netmask: 255.255.255.248
Gateway: xx.xx.xx.200

Don't set up NAT, just firewall rules allowing traffic destined to xx.xx.xx.xx/29 in from eth0 out eth2.

 

[ejecthawk]

Try starting it on the command line.

sudo /sbin/wpa_supplicant -dd -Dwired -ieth0 -c wpa_supplicant.conf

and see what you get.

Appreciate the help!

Your command is returning illegal instruction:

ubnt@RT1:~$ sudo /sbin/wpa_supplicant -dd -Dwired -ieth0 -c wpa_supplicant.conf
Illegal instruction
ubnt@RT1:~$ sudo journalctl -u /sbin/wpa_supplicant -dd -Dwired -ieth0 -c/etc/wpa_supplicant.conf
journalctl: invalid option -- 'd'

Everything from a config perspective looks good to my eyes:

ubnt@RT1:~$ ls -lh  /sbin/ |grep wpa_supplicant
-rwxr-xr-x    1 root     root        2.6M Jan  8 18:48 wpa_supplicant
ubnt@RT1:~$ cat /config/wpa_supplicant.conf
# Generated by 802.1x Credential Extraction Tool
# Copyright (c) 2018-2019 devicelocksmith.com
# Version: 1.04 windows 386
#
# Change file names to absolute paths
eapol_version=1
ap_scan=0
fast_reauth=1
network={
        ca_cert="/config/auth/CA_****KEYNAME****.pem"
        client_cert="/config/auth/Client_****KEYNAME****.pem"
        eap=TLS
        eapol_flags=0
        identity="****MAC****" # Internet (ONT) interface MAC address must match this value
        key_mgmt=IEEE8021X
        phase1="allow_canned_success=1"
        private_key="/config/auth/PrivateKey_PKCS1_****KEYBANE****.pem"
}
ubnt@RT1:~$ ls -lh /etc/wpa_supplicant/ |grep wpa_supplicant-wired-eth0.conf
lrwxrwxrwx    1 root     root          27 Jun 28 20:27 wpa_supplicant-wired-eth0.conf -> /config/wpa_supplicant.conf
ubnt@RT1:~$ systemctl status wpa_supplicant-wired@eth0.service
* wpa_supplicant-wired@eth0.service - WPA supplicant daemon (interface- and wired driver-specific version)
   Loaded: loaded (/lib/systemd/system/wpa_supplicant-wired@.service; enabled; vendor preset: enabled)
   Active: failed (Result: signal) since Fri 2019-06-28 21:29:13 EDT; 11h ago
  Process: 5024 ExecStart=/sbin/wpa_supplicant -c/etc/wpa_supplicant/wpa_supplicant-wired-eth0.conf -Dwired -ieth0 (code=killed, signal=ILL)
 Main PID: 5024 (code=killed, signal=ILL)

Jun 28 21:29:13 RT1 systemd[1]: Started WPA supplicant daemon (interface- and wired driver-specific version).
Jun 28 21:29:13 RT1 systemd[1]: wpa_supplicant-wired@eth0.service: Main process exited, code=killed, status=4/ILL
Jun 28 21:29:13 RT1 systemd[1]: wpa_supplicant-wired@eth0.service: Unit entered failed state.
Jun 28 21:29:13 RT1 systemd[1]: wpa_supplicant-wired@eth0.service: Failed with result 'signal'.

 

[mb300sd]

Illegal instruction means the wpa_supplicant wasn't compiled correctly for your the processor in your router. Did you add it or was it already in the firmware?

 

[ejecthawk]

Illegal instruction means the wpa_supplicant wasn't compiled correctly for your the processor in your router. Did you add it or was it already in the firmware?

Gotcha, that makes sense I was starting to wonder if it was a package issue.

Following the guide I did mentioned repos didn't have the correct package and you had to manually load it. Unfortunately his link was to the old community site for Ubitquiti, they changed their site, and the link is dead. I thought I had found a good copy of it for download somewhere else but I guess not, I'll search around and see if I can get the package elsewhere or it may just be a waiting game until they fix the link.

EDIT: I was able to find the updated package in the back-port repos. And wouldn't you know it it took maybe 15 minutes to configure once I had a working package.

 

[Corey Clingo]

Just got fiber installed at the lake house today. Pulled the certs out of my BGW210 as soon as the tech left. Gigabit VPN is as great as I thought it was gonna be. Traceroute shows 2 hops between my routers, and 7ms pings through the tunnel. Now I need to upgrade the router here, maxing out CPU at 500mbit IPSec in one direction.

How did you pull the certs from your BGW210? Mine got installed yesterday and I was going to rig up an EAP proxy, but this sounds even better.

 

[mb300sd]

How did you pull the certs from your BGW210? Mine got installed yesterday and I was going to rig up an EAP proxy, but this sounds even better.

WIP: Supplicant Mode by aus · Pull Request #19 · aus/pfatt

 

[Corey Clingo]

WIP: Supplicant Mode by aus · Pull Request #19 · aus/pfatt

Got it, thanks! I'm upgrading my OpenBSD firewall at the moment, but will try the created files shortly.

I had a couple small hiccups, for any future travelers:

1. The device name for the USB stick is actually /dev/sda1 rather than /dev/sda

2. The BGW210 linux complained about not being able to mark the FAT fs as "dirty" after I unmounted and removed the stick. Windows 10 then wanted to "repair" the stick, and the .der files were 0 bytes. Formatting the stick in Windows (full format, not quick) before using it in the gateway allowed the files to be preserved, even though Windows still wanted to repair the stick. Bad stick maybe?