Some websites don't load, but all get through the ISP router
-
I just set up a pfSense router as a virtual machine, I have all the networks set up and the VLANs seem to be set correctly, but I cannot reach certain sites. When I ping Duckduckgo.com for example I get no reply, I do a tracert from the WAN side of pfSense and I see that it goes through my ISP router fine, and continues on for about 15 hops, then stops. But if I switch over one of my computer's interfaces to point directly at the ISP router not only can I get out, but I get a real response. Oddly enough however I still can't ping the addresses I've found broken when on the ISP network but they do load their pages.
Any ideas as to why with working DNS and a trail showing traffic past the ISP router/modem I wouldn't be able to reach some sites but can still reach others?
I've already gone through some of the basic stuff including turning off any offloading and making sure to have the bottom two things unticked on WAN.
-
@nhsep said in Some websites don't load, but all get through the ISP router:
But if I switch over one of my computer's interfaces to point directly at the ISP router not only can I get out,
Huh - how is that? That screams asymmetrical to me.. Could you give some details of how you have everything connected.
What are your rules on pfsense - not being able to ping could be your not allowing for icmp. Not all sites even answer isp anyway.
-
I'm not exactly sure. After testing on another network I found that indeed the site doesn't ping in general though that still doesn't explain why the traffic isn't getting through. I won't go into too much detail at the moment as I have a working solution right now which was putting the old router back in place while I figure this out and get things set up on a new switch which I'd been waiting for cables to come in for.
The Hypervisor is XCP-ng, and it's in a node server which has dual 10G NICs, so I'm not starved for bandwidth but I am very short of ports. I had been handing pfSense the port by itself and having it do all the VLANing instead of going that on the hypervisor side as I wanted as little interference as possible from the HV side.
Right now all my switches are Dell PowerConnect 2816's but I'm putting in a CRS326-24S+2Q+RM as my backbone and looking at getting another Mikrotik 24 port with 10G uplinks as a single switch for the rest of the network.
I may come back to this post later when I get my new ISP in if I'm still having issues (which I imagine I probably would) I'm also looking at just restarting my config from nothing and using a few things that I've learned from this as a starting point for testing before I do any major changes. (which I don't at the moment feel I've done from the default config)
If it's any help I can upload the config I've been using; but I think I'll wait until I at least test it on another piece of hardware that I have for the night.
-
@nhsep
Disable -> System / Advanced / NetworkingHardware Checksum OffloadingDisable hardware checksum offload
Hardware TCP Segmentation OffloadingDisable hardware TCP segmentation offload
Hardware Large Receive OffloadingDisable hardware large receive offload
also check MTU
-
@nhsep said in Some websites don't load, but all get through the ISP router:
if I switch over one of my computer's interfaces to point directly at the ISP router not only can I get out, but I get a real response.
As @johnpoz said this statement raises questions!
What exactly are you doing to 'switch over'?It implies you might be simply re-configuring it to use the pfSense IP as it's gateway rather then the ISP router. If that is the case and they are on the same subnet then you almost certainly have an asymmetric route which would explain the failure you're seeing entirely.
https://docs.netgate.com/pfsense/en/latest/troubleshooting/asymmetric-routing.html
Steve
-
@kiokoman those are all covered in the whole
"I've already gone through some of the basic stuff including turning off any offloading and making sure to have the bottom two things unticked on WAN."
Those two other things I was referencing are "Block private networks and loopback addresses" and "Block bogon networks" both of which were on for a while and are now off with the same results.
-
@nhsep said in Some websites don't load, but all get through the ISP router:
I had been handing pfSense the port by itself and having it do all the VLANing
There was just another thread about this - where XCP-ng which is ZEN, sets a mtu of 1496 and causing sites not to work.
https://forum.netgate.com/topic/172390/issue-accessing-certain-websites
-
@stephenw10 Switch over as in I switched from my RJ45 plugged into my switch on VLAN 10 running off my pfSense to my SFP+ card going through my TrueNAS server in bridge mode to VLAN 1 and manually setting the SFP on VLAN 1 to go straight to the ISP router which worked fine and passed all the traffic without issue.
So the process is VLAN 10 client to pfSense host at 10.0.10.1, over to VLAN 1 on the same cable reaching out to 192.168.0.1 which is the ISP router on VLAN 1. That does not work with some sites, but works fine for many.
Then I disabled that NIC, reenabled the other one, and went manually from 192.168.0.37 or something to 192.168.0.1 all on the native LAN and got through just fine.
Even cutting out any of the VLAN stuff when I had pfSense hosting DHCP for 10.0.75.x on the native VLAN as soon as any traffic passed through pfSense it lost its ability to go to several sites but not all. In particular the sites that gave me issues were DuckDuckGo.com and idmsa.apple.com.
Bad
10.0.10.100 VLAN 10 > 10.0.10.1 VLAN 10 > 192.168.0.1 VLAN 1Bad
10.0.75.100 VLAN 1 > 10.0.75.1 VLAN 1 > 192.168.0.1 VLAN 1Fine
192.168.0.2 VLAN 1 > 192.168.0.1 VLAN 1Fine
10.0.75.100 VLAN 1 > 10.0.75.1 (Airport Extreme router) VLAN 1 > 192.168.0.1 VLAN 1 -
@johnpoz
Took me a minute to understand how they laid out that solution (and I'm also rather new to this level of networking), but that makes a lot of sense. I will give that a go and see how things end up. I really appreciate it. -
Yup, that was it. I at least have most things acting normally now. I'll find out as I keep going if anything else pops up, but I'm thinking that was probably it. Now I just need to migrate my whole network to new VLANs...
