Crazy network issue - Windows + pfSense

[Patrick]

I along with @William and @Diavuno are experiencing a crazy network issue.

pfSense firewall (fairly open actually and no Squid or other advanced packages) installed and connected via an open switch to a Windows Server 2012 R2 box.

From the Windows Server 2012 R2 box, when we try going onto sites to download benchmarks, a lot of resources are not loading. It almost looks like it is doing DNS lookups and then not connecting to resources. When I use nslookup on the Windows box and on the pfsense the DNS lookups are resolving.

Strange to see half pages timeout on loads, often to external CDNs. Some sites are loading fine, even complex ones like Yahoo and even Google read-ahead suggested searches are working. Other sites like Aida64.com has netdna.bootstrapcdn.com where it is getting stuck.

Does anyone have ideas on what this might be? I have never seen anything like this before.

 

[bds1904]

Ipv6 mtu issue?

 

[PigLover]

Agree it sounds like a v6 issue. Do you have v6 address defined on the pfSense box that might be advertising as a router for autoconfig - but the route is dead?

I've had exactly these symptoms with Comcast's V6 service when addresses occasionally go dead.

 

[neo]

Strangely enough I've experienced this issue before on pfSense. Can't remember how I fixed it.

Are you doing any LAGG?

Might have been a firewall rules issues. Try temporary disabling the firewall to see if it fixed it?

Also I would check the "System: Advanced: Networking" section and try disabling some of the hardware offloading - as sometimes there is are driver issues.

 

[TechIsCool]

Also confirm MTU is not set weird on anything I have had issues where if the MTU was high on a switch it started breaking things.

 

[Danic]

a lot of resources are not loading.

Can you describe that a little more? I'm no networking guru but in my pfsense triple WAN setup I some times have issues with web pages loading because some external content (images, css files, ads, etc) is being feed by a different HTTP(S) server. In my case this was only few percent of the websites visited. Let us know what fixes it.

 

[Patrick]

Can you describe that a little more? I'm no networking guru but in my pfsense triple WAN setup I some times have issues with web pages loading because some external content (images, css files, ads, etc) is being feed by a different HTTP(S) server. In my case this was only few percent of the websites visited. Let us know what fixes it.

That is the strange thing. Similar to what is happening except that it is consistent on a few sites/ resources. E.g. loading AIDA64 | The Ultimate System Information, Diagnostics and Benchmark Tool the browser is hanging on:
netdna.bootstrapcdn.com

Servethehome.com loads but the Amazon ad on the right side does not load for a long time. Then it does load.

Comcast login page did not work and etc.

Very strange. IPv6 is configured the same as my home pfsense now and the issue persists.

 

[Patrick]

OK so I did a bit of a test today. Two different pfsense boxes and I just moved my laptop's cable between them.

It turned out that it is not the local Windows Server 2012 R2 that is the issue. Instead it is the pfsense box providing the VPN gateway services.

 

[Diavuno]

@Patrick

I seem to recall seeing ipv6 unchecked in pfsense vpn settings.

call me if you need a hand, work finally slowed

 

[Patrick]

So as of yesterday I think we were high 700mbps down and well over 800mbps up. Not bad at all. @Diavuno - am on the road but we should chat.

 

[Diavuno]

So as of yesterday I think we were high 700mbps down and well over 800mbps up. Not bad at all. @Diavuno - am on the road but we should chat.

That's some good speed!

 

[RchGrav]

What does your mbuf usage read on the system information screen, and what kind of NIC's are in your pfsense box.

Are you aware of this info... Tuning and Troubleshooting Network Cards - PFSenseDocs

You have to bump the mbufs on some nics to 1000000.

In /boot/loader.conf.local - Add the following (or create the file if it does not exist):
kern.ipc.nmbclusters="1000000"

My issues were similar to what you describe but it was usually when there was a lot of traffic flowing through pfSense.. DNS resolutions would fail.

Btw.. I believe Intel (igb) and Realtek (re) both require this tuning.. so even if it isn't the issue you had, it may be good info to be aware of.

 

[TType85]

I along with @William and @Diavuno are experiencing a crazy network issue.

pfSense firewall (fairly open actually and no Squid or other advanced packages) installed and connected via an open switch to a Windows Server 2012 R2 box.

From the Windows Server 2012 R2 box, when we try going onto sites to download benchmarks, a lot of resources are not loading. It almost looks like it is doing DNS lookups and then not connecting to resources. When I use nslookup on the Windows box and on the pfsense the DNS lookups are resolving.

Strange to see half pages timeout on loads, often to external CDNs. Some sites are loading fine, even complex ones like Yahoo and even Google read-ahead suggested searches are working. Other sites like Aida64.com has netdna.bootstrapcdn.com where it is getting stuck.

Does anyone have ideas on what this might be? I have never seen anything like this before.

I had an issue before with our PFSense setup at my old job. Initially, we had calls of some people having issues getting to one of our web sites while people on the same network as them could get in fine.

I was testing on one of the machines (Server 2008 R2) behind that PFSense box and had trouble getting to a lot of sites like you had. Some were missing assets, some wouldn't load.

After banging my head against it for a few days I brought up a second box, restored the configuration from the first one and everything worked like normal. I reset the old PFSense box and restored the configuration and when testing had the same issues getting out like before. I did a full format/reinstall of PFSense, restored the configuration and then it started working like normal.