Port Forwarding Multi-Wan Issue on 1 Wan

james416 · Jan 26, 2020, 5:03 AM

I posted something similar in another group, but think this is the better area and I've got a bit more info.
First I should say my overall setup was working a week ago. In trying to fix something I've apparently broken something.
My setup involves 3 interfaces, WAN, LAN and one named BellFibe which is a fiber internet connection. Backstory here is WAN is my original internet connection which I've dropped to minimal speeds but kept for now as a backup, BellFibe is new GB connection.

WAN has the router in Bridged mode, so PFSense sees the external IP.
BellFibe doesn't have bridged, so it's in DMZ. Router is 192.168.2.1 and PFSense interface is at 192.168.2.25.
LAN is 192.168.1.0/24.

BellFibe is the default Gateway. It was setup using a failover gateway, however I just changed that today direct to BellFibe instead.

Here are my NAT Rules. Key point at the moment is that none of the BellFibe ones are working for some reason, though right now I'm only worrying about the first rule. The Destination address is my external IP, but I've also had it as *, which is what i'm sure worked a week ago. Another note here, I've essentially got the same rule on the WAN for testing the connection.

So there are all my rules.
If I externally access BellFibe at 443, I get timeouts and the webpage doesn't load. I wasn't sure if the traffic was getting through, so I turned logging on the connection and I could see the traffic passing. I was able to run iftop on the 192.168.1.223 box and when I try hitting the webpage I see the source IP appear. So it looks like traffic is getting through.
The debugging suggestions also say to check States. These 4 are relevant and show the connections are reaching the destination. But these 4 are all I get. If I try to connect via WAN, I get many more items appear and the page actually loads.

So further I tried packet captures to see if I could see anything.

And to compare, I did the same via the WAN.

I simply cannot figure out what else to try or look at. To this point I've actually removed all the port forwards and rules and aliases and added them back to no avail. I imagine the problem is caused by something I changed as I said it was working a week back but unfortunately I don't have backups from then.

Best I can think of is traffic thru bellfibe is getting to the destination, but somehow getting blocked on it's way back, or maybe going back to the wrong gateway, but since bellfibe is the default I don't see how. It'd make more sense if WAN didn't work.
I've also thought maybe I'm missing a LAN rule, as my bellfibe connection is 192.168.2.x but my lan is 192.168.1.x. Except that all my lan machines have no issues hitting the internet.
Looking for any other suggestions. At this point I may have to reinstall, and when I do swap BellFibe to WAN, and probably get rid of that service, but that'll take me a while and I'd like to avoid.

james416 · Jan 27, 2020, 3:59 AM

Update. Solved the issue, turns out it wasn't any PFSense setting at all. The provider changed a setting in my modem that made the DMZ be able to act like a router in bridged mode and I turned it on I guess. Oddly this mode didn't seem to conflict much with my static IP configuration in PFsense. I could still get out on the internet, just had issues with connecting via my port forwards. A quick change of PFSense to have the interface use dhcp and things are good.

chpalmer · Jan 27, 2020, 4:03 AM

@james416 said in Port Forwarding Multi-Wan Issue on 1 Wan:

BellFibe is the default Gateway

Yeah.. That would be something to post in bold in your case..

I wish ISP's would get a clue..