Secondary WAN can ping & resolve hostnames, can't browse

sef1414

I'm quite stumped. I moved my WAN interfaces to a switch. So I've got WAN1 / WAN2 / WAN3 modems all connected to a switch, with a trunk going to pfsense. Each interface is tagged with a corresponding VLAN in pfsense. WAN1 / WAN2 are working fine. However, I'm running into trouble with WAN3.

WAN3 is a different ISP. It's a CenturyLink PPOE connection. Pfsense successfully obtains an IP from WAN3. When traffic is routed through WAN3, I am able to ping an external hostname and receive a reply, so there is connectivity, and DNS is working. However, I can't curl (or browse the internet). I've tried adjusting MTU, toggling automatic outbound NAT rules, rebooting modem, rebooting pfsense, etc. Firewall rules permit any traffic from LAN and work fine with other WAN interfaces. I suspect it has something to do with introducing a VLAN to PPOE. Anyone have any suggestions?

viragomann

@sef1414 said in Secondary WAN can ping & resolve hostnames, can't browse:

When traffic is routed through WAN3, I am able to ping an external hostname and receive a reply, so there is connectivity, and DNS is working.

Did you verify that the traffic went through WAN3 both ways, requests and responses?
Use packet capture to get sure.

Is the gateway IP of the new connection unique?

sef1414

@viragomann

I can see it with packet capture, and can see the curl request but not response.

The IP is unique as far as I can tell.. its the only one I have from that ISP and I didn't get to choose it. But its not CGNAT or anything.

viragomann

@sef1414
I assume you're trying this with IPv4 traffic.
So presumably ping replies come back to one of your other WANs. This could be the case if the outbound NAT isn't configured correctly.

If your WAN have no RFC 1918 or CG-NAT IPs they must be unique anyway. But was suspecting, that you have got private IPs.

So check the outbound NAT settings. If it's in automatic mode, is there a rule for WAN3 and pfSense itself?

sef1414

@viragomann

Yep, definitely have outbound NAT rules set up. Pretty stumped here.

sef1414

@sef1414

So I am testing on a single device

And I logged the traffic from this rule. I notice firewall entries show the LAN device as the source, going directly to destinations on outside internet. So its as if traffic is skipping WAN3 entirely.

I don't think this is expected behavior. Source should be the WAN3 address I believe?

viragomann

@sef1414
No, that's expected. In the firewall log you see the origin source address.

You can better verify this in Diagnostic > States. If you filter for the destination IP you should see an entry on LAN with the devices LAN address and one on the WAN with the public address.

sef1414

@viragomann

Ok yeah, that makes sense, now that you mention it, I've seen that before. Just not something I typically pay attention to. Guess that leaves me pretty well stumped here.