Inbound NAT with Multi WAN broken with 21.02?
I just upgraded my SG-3100 to 21.02. I have a setup with dual-wan connection which worked for years now.
Setup (simplyfied) is:
WAN1: DSL-PPOE (50 Mbit, fixed IPv4 address)
WAN2: Fiber-Connection with modem in full bridging mode between Internet an pfSense (900 Mbit, dynamic IPv4 address)
LAN: with mailserver
The default gateway is WAN2 because everything that not has to be reachable from outside should use the faster fiber connection.
I have inbound nat rules for accessing e.g. mailserver from the internet on WAN1.
After upgrade to 21.02 yesterday the Inbound NAT seemed to stop working. I can see incoming traffic over WAN1->pfSense->nat destination, but packets never travel back over WAN1 . They use the "default gateway" on WAN2. So no server is reachable from the internet anymore.
Is this a bug in 21.02 or do I have to add additional things/rules somewhere, which was not necessary with old release?
You help is very appreciated.
I have nearly the same problem after upgrading my XG-7100 1U to 21.02 :
The difference for me is that the answer from LAN server try to goes out to the correct interface for the correct WAN, but the outgoing answer is blocked by 1000000104 label "Default deny rule IPv4"...
The same exact same setup of course worked correctly in the previous version.
I have tried some points of : http://doc.pfsense.org/index.php/Port_Forward_Troubleshooting with no success.
If i force a gateway on the interface rule, the block is not logged anylonger, but the flow still not work and it seems cant find it on tcpdump of interfaces, so it seems it's still blocked somewhere.
Strangely, it works directly if i set the same Port Forward rule on an OpenVPN client interface (with direct public IP on it).
I did not find much differences on /tmp/rules.debug when i switch interface for the same rule (except the interface name of course ^^)
Outbound NAT is pretty classic, manual rules for LAN to VPN Access, and automatic rules for WAN1/WAN2.
So im also interrested to know if there are extra rules needed by this version^^
Thanks in advance,
Did you try deleting and recreating the inbound NAT rules?
I have found sometimes that pFSense gets confused over some of the rules if an interface changes, and solution is to delete and re-add. Maybe this update changed the way it reference interfaces which do not update some NAT rules correctly.
Thanks for your answer.
Sadly, i tried to remove all port forwards, and recreate them and it still the same : outgoing answer to the interface of correct WAN blocked by the default deny out ...
To summarize the flow :
WAN1>PfSense RDR:443>HTTPS_Server->pfSense-> blocked OUT default deny on WAN1.
(exactly same behavior on WAN2. but not on ovpnc....)
Edit : The WAN1's GW is the pfSense default GW
I have also tried a completely new port forward, and its the same. Works only on ovpnc interface and not on WAN1/2.
I also tried switching to fully "Automatic outbound NAT" with no luck neither...
I dont understand why after upgrade to 21.02 it get "blocked" by the default rule, maybe the firewall dont recognize those answer packets as "RELATED,ESTABLISHED" or something like that ?
i kept verifying, the /tmp/rules.debug only move interface names when i change the a port forward rule, so no clues from here to me.
Note : my WAN1/2 interface addresses are private addresses to communicate with internet boxes that are configured as "DMZ (Nat1:1)" to the pfsense so the inbound public traffic is relayed as-is to pfsense. Obvously, the block private traffic is NOT CHECKED on theses interfaces^^, bogons block still "ON".
Note2 : ovpnc client that i can test have a direct public IP adress on it, that's the only difference with WAN1/2 i see right now.... (except from the interface names, lagg0.4090 vs ovpnc2)
Note3 : internet boxes configuration/FW was not changed, and the so called "DMZ"(nat1:1) feature work as intented.
This pfsense masquerade outgoing LAN traffic correctly to boths WANs, VPNc, GW_GROUP so it seems not a "basic connectivity" issue
Thanks for your help,
I am having the same problem except it's not an upgrade I just purchased an xg7100 running 21.02 and port forwarding is not working. I am using dual wan. Wan1 is a Starlink satellite so port forwarding won't work on it "CGNAT" so I have wan2 a 20MB WISP connection with static IP set up as a failover ( gateway settings Tier 1 Starlink tier 2 WISP) if I set them both to tier 1 port forwarding on WAN2 works or if I set the WISP to tier 1 and Starlink to Tier 2 then port forwarding will work. But I can't get port forwarding to work on WAN2 set to tier 2.
But my desire is to have Starlink be the primary and the wisp be backup and NAS access.
Any help would be much appreciated!
Big update since my precedent post : (I now run again 2.4.5p1 and evrything work fine)
1 : in fact, i got the exact same problem than the OP, the "Port Forwarding" works only for interface for the default GW interface (or higher priority on an GW_GROUP).
exactly like Nalasko and https://forum.netgate.com/topic/161157/multi-wan-port-forwarding-is-broke-in-pro-version ...
2 : I was mistaken because i found another "bug" : OpenVPN client in pfsense getting by default the "push route defaut GW" from the VPN provider ; but the GUI general "routing" DID NOT SHOW IT !! (curl -o - http://checkip.dyndns.org to check it). Once checked the correct option in ovpnc ("dont pull routes" by memory) the GUI "routing" has correct default gw indication and modification.
Now i can confirm the working "port forward" follow the default GW in 21.02
(And i can confirm with the same config file the 2.4.5p1 work fine^^)
3 : Finally I reinstalled (after asking the techsupport the installation image for my device : answer less than 2 minutes !!) the 2.4.5p1 image : after resinstallation, you will want to restore your backup config, but BEWARE : unplug all WANs/routed interface to the net because the automatic 'reinstall all packages' after a config upload/restore will try to install again the 21.02 release !! For my case, i tried to understand why i had several errors of missing libs to figure out that an update was ongoing without my intervention (php74, etc,etc).
What i done to prevent after a 3rd reinstall/restore : Restore config with only LAN/admin interface, wait long to loggin to pfSense (cause DNS not reachable, etc), switch update channel to "previous", cancel the automatic package resinstall, then reconnect WAN interfaces and eventually switch to stable (to see the "new 21.02" branch, and select back 2.4.5 DEPRECIATED until this NAT regression is fixed ^^
@nalasko Ask the support for the 2.4.5p1 image for XG-7100 1U/DT throught the portal support as said in : https://docs.netgate.com/pfsense/en/latest/solutions/xg-7100/reinstall-pfsense.html : multiwan portforwarding work in this version
The "tiers" in pfSense has only sense (to my understanding^^) for outgoing/replies traffic.
Without the multi WAN bug we encounter, the portforward work on all WANs : its up to your client to define what is a primary and a backup (client side ? DNS roudrobin ? DNS update only? failover?)
-> deleted all port forwarding & nat rules (incoming and outgoing) and checked that all corresponding FW rules are also deleted -> reboot -> added new port forwarding rules with "add associated firewall rules" -> same behavior (not working if I'm not on the default gateway)
-> deleted again all port forwarding & nat rules and checked that all corresponding FW rules are deleted -> added new port forwarding rules without "add associated firewall rules" -> added the firewall rules manually -> same behavior (not working if I'm not on the default gateway)
Even outbound nat is not working if I'm not on the default gateway.
Even more: system update shows "unable to check for updates" because my SG-3100 is searching for armv7 packages, which are definitely not existing on Netgate file server (checked this via browser an from ssh console of the SG-3100).
Too bad. For the meantime I've switched my default gateway to the slow DSL connection (with static IP) so I have the chance to get my eMail server up and running again. But now my 20x faster fiber connection is useless :-(
Glad to see I am not alone on this. I had heard some OpenVPN client issues could be present, so when that failed at first I thought I was going to have to go down that hole. Then I started getting down notices for my external services. That is when I realized none of my NAT rules were working properly.
Dual Wan config like the others. It looks like deleting and adding rules back is not working for anyone else so I guess we just sit and wait for a response?
For now I disabled CARP so that it would fail back to my older version unit.
Hmmm... I've seen that there is a hotfix (pfSense Plus 21.02-p1) for the nat problem... Installed it, but still the same error :-(
Someone else who already installed the hotfix?
I have not read the hotfix release notes, but the ticket for this NAT regression is not assigned yet / still open : https://redmine.pfsense.org/issues/11436
So i doubt it is corrected in the hotfix...
(and the post https://forum.netgate.com/topic/161058/nat-issue-after-21-02-upgrade/4 seems to confirm that the hotfix still have the problem)
For my setup, I will wait at least this ticket closed before trying again the 21.02 branch as the multi WAN inbound is mandatory for me.