Help Needed: Unexpected Results NAT port forward is not working for some reason

[AllGamer]

1st time I try to implement pfsense in a real environment.

I've tried it before in a test and learn environment without much problem,
so I'm little baffled as to why something so simple as to add a NAT port forward for port 80 a web server behind pfsense,
connected to the WAN is not working.

VDSL modem (PPPOE done by modem /29) <–-> pfsense (as router firewall for internal LAN) <---> Web server (regular port 80 web)

VDSL modem (PPPOE done by modem /29) <---> Web server (regular port 80 web)

If I connect the Web server directly to the VDSL modem, it works just fine I can access the web server from outside (from the internet),
but even with the proper (or so I believe) NAT configured, I can not connect to the Web server from the outside.

It's actually the same problem with every other port (FTP, HTTPS, SMTP, POP, etc) I try to NAT port forward they are all blocked.

I followed a lot of the step by steps guides, and even video tutorials, I do exactly the same thing and... it doesn't work, which I know it should have worked, as I had them working before in my test environment.

Currently running pfsense v2.3, but the problem was the same on v2.2.6, running on physical hardware.
Core Duo Intel
4 GB RAM
msk0: on board NIC Marvell [LAN]
em0: pci intel 1000 [WAN]
em1, em2, em3, em4: pci-e intel I340-T4 [not in use, until NAT issue is resolved]

This is exactly how my rules looks like, as I followed this guide https://calvin.me/port-forward-web-servers-in-pfsense-2/

Additional info:
pfsense is working fine as UPnP, if I run programs like Skype, Games, or any App that will initialize communication to the outside, it will create the proper port forwards automatically, it works fine without any problem.

However UPnP doesn't work for stuff like Web / FTP / Mail which is why the NAT port forward is necessary for these services to accept external connections.

[cmb]

The config shown is correct. Go through the troubleshooting steps.
https://doc.pfsense.org/index.php/Port_Forward_Troubleshooting

[AllGamer]

Common Problems:

1. NAT and firewall rules not correctly added (see How can I forward ports with pfSense?) Hint: Do NOT set a source port
Currently set to any (*), I did try during troubleshoot to set it to specific port 80, and it made no difference, set it back to any

2. Firewall enabled on client machine
nope firewall is disabled in the webserver, that's the first thing I check for connection issues, the whole idea was to use pfsense as the firewall

3. Client machine is not using pfSense as its default gateway
pfsense is set as the gateway, as pfsense is both the  DHCP server and DNS server, also verified in the Web Server ifconfig result

4. Client machine not actually listening on the port being forwarded
web server is accepting connection, it works when it is not using pfsense

5. ISP or something upstream of pfSense is blocking the port being forwarded
not the issue, else the web server would have not displayed the web page when connecting to the internet without pfsense as the firewall

6. Trying to test from inside the local network, need to test from an outside machine
Web server answer fine from machines within the same LAN, also answer fine from the Outside, when pfsense is not in the middle

7. Incorrect or missing Virtual IP configuration for additional public IP addresses
Hmm… the modem is on aaa.bbb.ccc.41
pfsense is on aaa.bbb.ccc.42
I was under the impression I could also run the web server in the same aaa.bbb.ccc.42 with NAT port forward
I thought Virtual IP were only needed for the 1:1 NAT translation.

8. The pfSense router is not the border router. If there is something else between pfSense and the ISP, the port forwards and associated rules must be replicated there.
The modem (gateway) aaa.bbb.ccc.41 is wide open, no firewall, no NAT, no filter or block of any kind, else the web server would not have answered, when connecting from the outside, when not using pfsense

9. Forwarding ports to a server behind a Captive Portal. An IP bypass must be added both to and from the server's IP in order for a port forward to work behind a Captive Portal.
pfsense is running on HTTPS port 444, so port 80 and 443 should not be in use.

10. If this is on a WAN that is not the default gateway, make sure there is a gateway chosen on this WAN interface, or the firewall rules for the port forward would not reply back via the correct gateway.
VDSL modem (aaa.bbb.ccc.41/ ISP gateway IP) <–-> pfsense (WAN IP aaa.bbb.ccc.42 / LAN IP 10.0.0.1 / HTTPS port 444 / gateway aaa.bbb.ccc.41) <---> Web server (10.0.0.13 / HTTP port 80 HTTPS 443 / gateway 10.0.0.1)

11. If this is on a WAN that is not the default gateway, ensure the traffic for the port forward is NOT passed in via Floating Rules or an Interface Group. Only rules present on the WAN's interface tab under Firewall Rules will have the reply-to keyword to ensure the traffic responds properly via the expected gateway.
I'm 100% sure there is no floating rules, or interface groups, it's clean brand new pfsense setup, there are no other rules, other than HTTP 80 and HTTPS 443

12. If this is on a WAN that is not the default gateway, make sure the firewall rule(s) allowing the traffic in do not have the box checked to disable reply-to.
Both rules
Not applicable

13. If this is on a WAN that is not the default gateway, make sure the master reply-to disable switch is not checked under System > Advanced, on the Firewall/NAT tab.
I need to verify this later when I get back to my pfsense box
Confirmed, it's not checked

14. WAN rules should NOT have a gateway set, so make sure that the rules for the port forward do NOT have a gateway configured on the actual rule.
Interesting… this is also something I need to verify when I get home
Confirmed, it's using default, as in not specified, it'll use whichever is the system default

15. If the traffic appears to be forwarding in to an unexpected device, it may be happening due to UPnP. Check Status > UPnP to see if an internal service has configured a port forward unexpectedly. If so, disable UPnP on either that device or on the firewall.
While this might be the case, shouldn't it at least load up the web page of whichever device it is connected to, like a security camera?
Anyway, this is also something I need to verify when I get home
… now this is interesting, it says "UPnP is currently disabled."
yet i have no problems with Skype, or video conferencing apps, or games... weird...

---

Lets assume all of the above checks out fine, what else could cause this weird behaviour with the NAT ?

I'm contemplating moving back to DD-WRT, with DD-WRT usually the problem I have, are the hardware devices dying every 3 years give or take.
I'm kind of tired of replacing routers just to run DD-WRT.

That's why this time around when my previous router started acting up, I thought I'll give pfsense a go to see how it performs over the DD-WRT, I was really looking forward to use the advanced features available in pfsense which DD-WRT can hardly compare.

It was really unexpected for a simple NAT port forward rule to not work as expected, I must be missing something really ridiculously simple and I'm probably over thinking it, as most guides and tutorials uses the exact same steps and it works for them. I need to figure out what I'm missing.

[johnpoz]

So per the toubleshooting doc did you do a packet capture??  This really is like 1 min of troubleshooting to find your problem..

Simple packetcapture to verify your traffic sending the packets to your device and it answers back..

What is in front of pfsense?  Does pfsense get a public IP on is wan?  Or is it behind a nat?  Post up your wan rules..  Your not doing any oddball outbound nats are you?  Are you using the captive portal?

What I can tell you is in the long time I have been here over multiple version of pfsense, I don't recall ever seeing a port forwarding issue that was not PEBKAC..  Can tell you it should take all of 5 seconds to create a port forward in pfsense.

You state when you take pfsense out of the middle - sounds like to me pfsense is behind a NAT..  So on that device doing nat are you forwarding to what pfsense wan IP is?  Do you have any other packages like snort or pfblockerng? etc..

[AllGamer]

Hey thanks for taking the time.

I forgot to update.

Issue solved, problem was ISP modem got reset, or ISP came in and resetted it.
So the firewall was turned back "ON"

after logging back into the modem and changing it back to OFF, then everything worked, as predicted when playing with the pfsense in a test environment.

Long story short, to avoid further unexpected ISP management intrusion, I disabled all the factory and ISP default accounts, changed Admin passwords, create new account for myself, and …. to really avoid further modem woes....

Set the modem in bridge mode, and now I'm using pfsense for PPPOe as I was planning to do from the beginning, that being said, Now I need to build probably a few more pfsense boxes to go behind this box, for the network management stuff, since I was planning to do Fail Over, load balance 2 WAN using pfsense in 2 physical boxes, so if one physical machine dies, the over one keeps going.

I was contemplating running 2 VM but unsure if the lag in VMware might cause network delay or not. I've seen such delay elsewhere before with other Network Apps that are VMmachine sensitive.

Anyway, that's topic for another thread.