Specific https site not working through pfsense
-
Morning All,
I have a bit of a strange problem that I could do with some assistance on.
pfSense 2.5.1 running on XCP-ng as a VM. wan on vlan 901 several LAN's on vlan 1100, 1200, 1300 and 1400 all with DHCP from the pfSense box and allow any from any to any firewall rules.
OpenVPN site to site setup from this site to my main office only routing required subnets (172.16.0.0/12) through the VPN.
General web traffic either http or https is working as expected but one specific site is not working through pfSense.
Currently testing from an ubuntu desktop VM on the same xcp-ng server but the problem exists on client machines out on the network as well. I'm changing the vlan tag on the VM and setting either a static public address or a DHCP depending on whether it's going through pfSense or going direct. When direct it works, when through pfSense it does not. Because of this I am happy that the VM, browser, connection and site are not the issue but it must be pfSense.
DNS is fine and I can ping the IP. Browsing to http://the.domain.com redirects to https://the.domain.com but the page then times out and does not load.
Any pointers on where to start here?
-
I would check the state table when testing to see if you have any traffic to/from that site.
A pcap will show you more if needed. It could be an MTU issue in the route to that site for example.https://docs.netgate.com/pfsense/en/latest/troubleshooting/website-access-issues.html
Steve
-
Thanks @stephenw10,
I'm working through the connectivity and website access troubleshooting guides now.
I also suspect MTU problems.
-
Ok, so checked through and I get an interesting result when doing the ping fragmentation test. I get good results at 1468, 4 short of the "normal" 1472 that I would expect. I think this is vlan tag related and that I have got something wrong in the hypervisor (xcp-ng).
Am I going to break anything if I set the MTU on the WAN interface to 1496 in interfaces>wan1 or should i put 1468 in the MSS
-
Interestingly I have just established that my own website is also not working through this connection. Checking the state tables shows the following;
LAN tcp 10.1.1.2:59812 -> 3.10.30.47:443 TIME_WAIT:TIME_WAIT 12 / 20 1 KiB / 17 KiBWAN1 tcp ...:14318 (10.1.1.2:59812) -> 3.10.30.47:443 TIME_WAIT:TIME_WAIT 22 / 20 3 KiB / 17 KiB
-
You should be able to set the MTU lower. PMTU should detect that and set TCP packet size accordingly. If it was a problem you have created locally I would expect it to affect all traffic not just one site. Seems more likely something in the route breaking PMTU in which case you might need to use MSS clamping.
Steve
-
@stephenw10 understood, thanks for the assistance.
What value would you suggest for the MSS? If I went really safe, say 1400 would that cause any serious problems?
-
It should be fine at lower values. It's common to clamp at 1350 for VPN traffic for example.
-
@stephenw10 fully understand the should. Don't worry I won't hold you responsible!!! (unless it works obviously).
I'm supposed to be on holiday this week and the job is about an 8 hour drive away so I want to avoid that if at all possible!
-
@stephenw10 Fingers crossed... that seems to have it sorted. I'll do some tuning once I'm able to be on site but for now all of the sites that I know were not loading are working fine.
Massive thanks, if you are every in Staffordshire let me know, I owe you beer or dinner or both! (genuinely thanks, you have no idea how stressed this has had me)
The learning opportunity for me now is to work out why I have never had this problem before. Another day, another lesson.
-
Ah nice! Something probably changed in the route. Fireware update, router swapped etc. You could probably find where it was failing with enough tracetroute and pinging but finding someone to admit it's a problem and fix it is a different matter!
Steve
