NAT/Port forwarding doesn't appear to be working whilst HAProxy is

  • Hi all,

    I have a pfsense installation v2.4.4, which has a single WAN with 8 additional IPv4 addresses on top of the normal IPv4 address for the WAN. It also has 4 LAN connections (These are VLANs configured to present as separate ethernet ports).

    All of this is running on a virtual machine. I have HAProxy installed and configured to load balance a Galera Cluster without issue, and had at one point setup HAProxy to forward ports from a single virtual IPv4 on the WAN to one of the clients connected to one of the LANs and this worked as expected, with the exception of the IP address presented by the client connecting to the forwarded port being replaced by the pfsense IPv4 address. This caused a few issues, so I deleted the rules from HAProxy and set up NAT based port forwarding to do the same job.

    I have applied the rules, restarted the pfsense server, and still the ports don't appear to be being forwarded.

    Images of NAT and firewall rules below.

    Running netstat -a via the command prompt doesn't show the ports being listened to at all, which would explain why they aren't open to anyone trying to connect, but that shouldn't be happening.

    I have been through the port forwarding diagnostics page with no success.

    netstat -a running via the command prompt page on the website doesn't return any of the ports selected being listened to for some reason. Which indicates that the ports aren't open or the firewall isn't being respected when it comes to setting up the WAN connection.

    Any ideas what I can do?

    NAT Rules;
    Current NAT Rules inside the firewall

    Firewall Rules;
    Current firewall rules

    HAProxy Frontend Rules;
    alt text

    HAProxy Backend Rules;
    alt text

  • LAYER 8 Global Moderator

    @ceastdep said in NAT/Port forwarding doesn't appear to be working whilst HAProxy is:

    I have been through the port forwarding diagnostics page with no success.

    So you sniffed on the wan and see the traffic hitting that IP.. Then you sniff on the lan interface and don't see or do see the traffic being forwarded?

    If you went through the troubleshooting guide you would know exactly what the problem is.. So lets see your sniff on the wan, and the sniff on the lan side interface that is suppose to send the port on to the server..

    BTW still looks like you have the HA proxy up and running.. Thought you said you turned that off and wanted to do just port forwards?

  • Hi @johnpoz

    HAProxy is still being used for the Galera cluster environment.

    I tested with the packet sniffer, and it seems to come through to the LAN side, but never makes it to the machine itself. I get the retransmission immediately after the first packet being sent because things don't connect. I can run the captures again with Nmap running if you don't want to take my word for it.

    If I run Nmap on the internal IP addresses from the pfsense machine, then the relevant ports seem to be open, but if I nmap from an external source the ports aren't open.

    The NAT/Port forwarding just doesn't seem to be working.

    It's worth saying again, as I clearly wasn't clear enough the first time. When I had these ports running via HAProxy, it worked fine, except for replacing the IP address coming in with the IP address of the pfsense machine. That was causing issues so I decided to go back to how the system worked on another pfsense server I have, and use NAT/Port forwarding to do the same job.

    It works on another system just fine, but it's v2.4.2, and not v2.4.4. Everything except that and the external IPv4s are the same.

  • LAYER 8 Global Moderator

    @ceastdep said in NAT/Port forwarding doesn't appear to be working whilst HAProxy is:

    but never makes it to the machine itself

    Well pfsense is doing what it was told to do then.. So why does the other machine not answer? Wrong port, its not actually listening? Different gateway other than pfsense, firewall on that machine, etc. etc..

    Now sniff on the dest machine - does it see the traffic?
    Have you validated that the traffic being sent by pfsense is actually going to the correct machine via looking at the mac address in the forwarded traffic..

  • Hi @johnpoz

    I'm currently trying to move it all to another pfsense instance, which is actually the gateway for these servers, and hopefully that'll help. However, it doesn't seem to be working, I can't even get NAT to work on ICMP for a specific IP address, but if I open ICMP to work across the whole WAN interface it works.

    It's also causing the same issue with the NAT port forwarding not working. So I'm sitting here bashing my head against a brick wall.

  • To answer your questions, the Mac addresses match, the IPv4 addresses match but the traffic never hits the machine. There's no firewall, it's switched off (firewalld on CentOS v7, disabled).

    Everything seems to be in order, I can ping and the ports are open from every other machine on that network, and even when I ping/nmap from the pfsense servers, it reports the ports open.

    It's something to do with the firewall, I'm certain of it, even if the traffic is reporting as being sent out of the pfsense box, something is going wrong.

    It's worth adding, it seems to me at least that anything I'm doing via NAT/Port forwarding isn't working. At this point I am happy to pay someone to spend an hour to resolve this.

  • LAYER 8 Global Moderator

    So you sniffed on the machine your sending too?

    which is actually the gateway for these servers

    Does it see the traffic and answer it? If it doesn't answer it but sends to wrong gateway?

    It NOT going to work if the gateway for the machines traffic being sent to is not the pfsense forwarding on the traffic..

    If you want actual support then

    Here is what I can tell you in the 10+ years I have been on this site and using pfsense port forwarding issues are always PEBKAC...

    BTW your haproxy is prob working because it does a source nat.. So traffic would be coming from pfsense IP, ie the whole proxy thing ;)

Log in to reply