Port forwading using NAT dropping packets issue
-
Hi to all,
I have a fairly simple network configuration with an internal LAN (192.168.1.x), a pfsense (2.4) box (10.0.0.20) and only one external IP (using a simple router) from my ISP provider.
While I am migrating our old ISA solution to pfsense I face the following problem:
I have to port forward multiple Nagios (nrpe) clients from the internal network to my unique external IP (different external ports) in order for our external Nagios server to communicate properly, but the nagios (nrpe) test command shows CHECK_NRPE: Socket timeout after 60 seconds.So with a straight forward process, I have set up a NAT rule from the external port to forward to an internal IP and port (internal Linux server) + automatic rule to firewall rules to allow traffic for this.
So far :
-I am running a test nmap command from my external server for both solutions (old working ISA and new pfsense)
-I am running a test command from my external server for nagios (nrpe) communication for both solutions (old working ISA and new pfsense)
-The pfsense log shows that the firewall rule works and pass the traffic to internal Linux client
-I am running a tcpdump command to my internal Linux for both solutions (old working ISA and new pfsense)the results:
for nmap from external server at working old ISA setup:
nmap --reason xx.xx.xx.xxx -p 15667
Starting Nmap 6.40 ( http://nmap.org ) at 2019-08-09 14:21 EEST
Nmap scan report for xx.xx.xx.xxx
Host is up, received echo-reply (0.077s latency).
PORT STATE SERVICE REASON
15667/tcp open unknown syn-ackfor nmap from external server at new pfsense setup:
nmap --reason xx.xx.xx.xxx -p 15667
Starting Nmap 6.40 ( http://nmap.org ) at 2019-08-09 14:23 EEST
Nmap scan report for xx.xx.xx.xxx (62.169.208.109)
Host is up, received syn-ack (0.078s latency).
rDNS record for xx.xx.xx.xxx: ipa109.211.myprovider.com
PORT STATE SERVICE REASON
15667/tcp filtered unknown no-responsefor tcpdump to internal linux box at working old ISA setup:
root@MYINTERALLINUXSERVER:/etc/nginx/sites-available# tcpdump port 5666
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
22:48:51.856839 IP nagios.myexternalserver.com.56233 > MYINTERALLINUXSERVER.nrpe: Flags [S], seq 1087712600, win 29200, options [mss 1304,sackOK,TS val 1625546298 ecr 0,nop,wscale 7], length 0
22:48:51.856904 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.56233: Flags [S.], seq 1775728356, ack 1087712601, win 14480, options [mss 1460,sackOK,TS val 70947103 ecr 1625546298,nop,wscale 6], length 0
22:48:51.939986 IP nagios.myexternalserver.com.56233 > MYINTERALLINUXSERVER.nrpe: Flags [.], ack 1, win 229, options [nop,nop,TS val 1625546319 ecr 70947103], length 0
22:48:51.946013 IP nagios.myexternalserver.com.56233 > MYINTERALLINUXSERVER.nrpe: Flags [P.], seq 1:126, ack 1, win 229, options [nop,nop,TS val 1625546319 ecr 70947103], length 125
22:48:51.946047 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.56233: Flags [.], ack 126, win 227, options [nop,nop,TS val 70947125 ecr 1625546319], length 0
22:48:51.967912 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.56233: Flags [P.], seq 1:217, ack 126, win 227, options [nop,nop,TS val 70947130 ecr 1625546319], length 216
22:48:52.053080 IP nagios.myexternalserver.com.56233 > MYINTERALLINUXSERVER.nrpe: Flags [.], ack 217, win 237, options [nop,nop,TS val 1625546347 ecr 70947130], length 0
22:48:52.053512 IP nagios.myexternalserver.com.56233 > MYINTERALLINUXSERVER.nrpe: Flags [P.], seq 126:252, ack 217, win 237, options [nop,nop,TS val 1625546347 ecr 70947130], length 126
22:48:52.055051 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.56233: Flags [P.], seq 217:443, ack 252, win 227, options [nop,nop,TS val 70947152 ecr 1625546347], length 226
22:48:52.142924 IP nagios.myexternalserver.com.56233 > MYINTERALLINUXSERVER.nrpe: Flags [P.], seq 252:1317, ack 443, win 245, options [nop,nop,TS val 1625546369 ecr 70947152], length 1065
22:48:52.144689 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.56233: Flags [P.], seq 443:1508, ack 1317, win 260, options [nop,nop,TS val 70947174 ecr 1625546369], length 1065
22:48:52.238864 IP nagios.myexternalserver.com.56233 > MYINTERALLINUXSERVER.nrpe: Flags [P.], seq 1317:1348, ack 1508, win 268, options [nop,nop,TS val 1625546393 ecr 70947174], length 31
22:48:52.238904 IP nagios.myexternalserver.com.56233 > MYINTERALLINUXSERVER.nrpe: Flags [F.], seq 1348, ack 1508, win 268, options [nop,nop,TS val 1625546393 ecr 70947174], length 0
22:48:52.238963 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.56233: Flags [P.], seq 1508:1539, ack 1348, win 260, options [nop,nop,TS val 70947197 ecr 1625546393], length 31
22:48:52.240077 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.56233: Flags [F.], seq 1539, ack 1349, win 260, options [nop,nop,TS val 70947198 ecr 1625546393], length 0
22:48:52.322994 IP nagios.myexternalserver.com.56233 > MYINTERALLINUXSERVER.nrpe: Flags [.], ack 1540, win 268, options [nop,nop,TS val 1625546414 ecr 70947197], length 0for tcpdump to internal linux box at pfsense setup:
root@MYINTERALLINUXSERVER:/etc/nginx/sites-available# tcpdump port 5666
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
22:49:39.531863 IP nagios.myexternalserver.com.52732 > MYINTERALLINUXSERVER.nrpe: Flags [S], seq 2333740221, win 29200, options [mss 1452,sackOK,TS val 1625558219 ecr 0,nop,wscale 7], length 0
22:49:39.531925 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.52732: Flags [S.], seq 2709812462, ack 2333740222, win 14480, options [mss 1460,sackOK,TS val 70958960 ecr 1625558219,nop,wscale 6], length 0
22:49:40.529125 IP nagios.myexternalserver.com.52732 > MYINTERALLINUXSERVER.nrpe: Flags [S], seq 2333740221, win 29200, options [mss 1452,sackOK,TS val 1625558469 ecr 0,nop,wscale 7], length 0
22:49:40.529165 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.52732: Flags [S.], seq 2709812462, ack 2333740222, win 14480, options [mss 1460,sackOK,TS val 70959208 ecr 1625558219,nop,wscale 6], length 0
22:49:40.935211 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.52732: Flags [S.], seq 2709812462, ack 2333740222, win 14480, options [mss 1460,sackOK,TS val 70959310 ecr 1625558219,nop,wscale 6], length 0
22:49:42.532796 IP nagios.myexternalserver.com.52732 > MYINTERALLINUXSERVER.nrpe: Flags [S], seq 2333740221, win 29200, options [mss 1452,sackOK,TS val 1625558970 ecr 0,nop,wscale 7], length 0
22:49:42.532831 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.52732: Flags [S.], seq 2709812462, ack 2333740222, win 14480, options [mss 1460,sackOK,TS val 70959707 ecr 1625558219,nop,wscale 6], length 0
22:49:42.945051 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.52732: Flags [S.], seq 2709812462, ack 2333740222, win 14480, options [mss 1460,sackOK,TS val 70959810 ecr 1625558219,nop,wscale 6], length 0
22:49:46.540642 IP nagios.myexternalserver.com.52732 > MYINTERALLINUXSERVER.nrpe: Flags [S], seq 2333740221, win 29200, options [mss 1452,sackOK,TS val 1625559972 ecr 0,nop,wscale 7], length 0
22:49:46.540684 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.52732: Flags [S.], seq 2709812462, ack 2333740222, win 14480, options [mss 1460,sackOK,TS val 70960703 ecr 1625558219,nop,wscale 6], length 0
22:49:46.968420 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.52732: Flags [S.], seq 2709812462, ack 2333740222, win 14480, options [mss 1460,sackOK,TS val 70960810 ecr 1625558219,nop,wscale 6], length 0
22:49:54.556556 IP nagios.myexternalserver.com.52732 > MYINTERALLINUXSERVER.nrpe: Flags [S], seq 2333740221, win 29200, options [mss 1452,sackOK,TS val 1625561976 ecr 0,nop,wscale 7], length 0
22:49:54.556591 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.52732: Flags [S.], seq 2709812462, ack 2333740222, win 14480, options [mss 1460,sackOK,TS val 70962697 ecr 1625558219,nop,wscale 6], length 0
22:49:55.208335 IP MYINTERALLINUXSERVER.nrpe > nagios.myexternalserver.com.52732: Flags [S.], seq 2709812462, ack 2333740222, win 14480, options [mss 1460,sackOK,TS val 70962860 ecr 1625558219,nop,wscale 6], length 0As you can see at the pfsense setup missing Flags [.], ack and Flags [P.], seq that may indicate dropping packets from pfsense.
The NAT and firewall rules are all straight forward with defaults and the pfsense logs shows that are working and pass the traffic.
It's unlikely the simple hw router to cause the problem because it passes without problem traffic for 80 and 443 portsAny help is welcome ....
Thanks,
Nick -
Ok, solved
Due to the test environment, my client MYINTERALLINUXSERVER was set to wrong getaway.