Can't get NAT to work: ISP modem >> (WAN) pfSense (LAN) >> (WAN) home router
-
Good day. I am sure you get asked this question a lot, and I scoured through the forum and couldn't find anything matching exactly. Here is my situation, I am trying basic tests for now.
I am familiar with firewalls to an extent, but never with a home router on the inside and having to port-forward. Any help someone may have would be greatly appreciated, as I have run out of ideas of what to try next. Thank you in advance.
Physical setup:
ISP modem --> pfSense WAN port
pfSense LAN port --> Asus RT-AX88U router WAN portTest case
I have an FTP server running to test. With just my Asus router, everything works
When I put the pfSense between modem and router it does not workIP settings
pfSense WAN (DHCP, modem-assigned public IP)
pfSense LAN (static, 192.168.1.1)
router WAN (static, 192.168.1.2, gw: 192.168.1.1)
router LAN (static, 172.16.10.1)
router assigns DHCP to internal network (172.16.10.2--172.16.10.50)
FTP server (172.16.10.46)Settings on pfSsense
static route > 172.16.10.0/24 - gateway 192.168.1.2 - LAN interface
NAT port-forward >
interface: WAN
protocol: TCP
Source: Any
Destination: WAN address
Destination port range: (from) FTP (to) FTP
Redirect target IP: tried both 172.16.10.46 and 192.168.1.2
Redirect target port: FTP
NAT reflection: use system default
Filter rule association: Rule NATTests
I have done packet captures from pfSense for many different configurations, including every time I changed a setting on my Asus router. On the Asus router, I noticed I have to keep "Enable NAT" on in the WAN settings, otherwise I can't get out to the Internet. I have added port-forwarding there as well and same results. Packet captures only see traffic coming in from the Internet but never getting a response.Packet capture
24.XXX.XXX.XXX is my test machine on the Internet
184.XXX.XXX.XXX is my pfSense WAN IP20:34:25.917974 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61261, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18749 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0xf0ff (correct), seq 1122232626, win 32120, options [mss 1390], length 0
20:34:27.761341 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61261, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18749 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0xf0ff (correct), seq 1122232626, win 32120, options [mss 1390], length 0
20:34:28.761604 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61261, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18749 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0xf0ff (correct), seq 1122232626, win 32120, options [mss 1390], length 0
20:34:29.760841 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61261, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18749 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0xf0ff (correct), seq 1122232626, win 32120, options [mss 1390], length 0
20:34:30.760111 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61261, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18749 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0xf0ff (correct), seq 1122232626, win 32120, options [mss 1390], length 0
20:34:31.761220 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61261, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18749 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0xf0ff (correct), seq 1122232626, win 32120, options [mss 1390], length 0
20:34:32.762201 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61261, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18749 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0xf0ff (correct), seq 1122232626, win 32120, options [mss 1390], length 0
20:34:35.761721 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61261, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18749 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0xf0ff (correct), seq 1122232626, win 32120, options [mss 1390], length 0
20:34:37.761090 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61261, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18749 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0xf0ff (correct), seq 1122232626, win 32120, options [mss 1390], length 0
20:34:40.958665 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61262, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18734 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0xf10e (correct), seq 1122232626, win 32120, options [mss 1390], length 0
20:34:42.761604 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61262, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18734 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0x6e87 (correct), seq 1122232626, win 0, options [mss 1390], length 0
20:34:43.762191 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61262, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18734 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0x6e87 (correct), seq 1122232626, win 0, options [mss 1390], length 0
20:34:44.760944 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61262, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18734 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0x6e87 (correct), seq 1122232626, win 0, options [mss 1390], length 0
20:34:45.761829 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61262, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18734 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0x6e87 (correct), seq 1122232626, win 0, options [mss 1390], length 0
20:34:46.761711 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61262, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18734 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0x6e87 (correct), seq 1122232626, win 0, options [mss 1390], length 0
20:34:47.760884 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61262, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18734 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0x6e87 (correct), seq 1122232626, win 0, options [mss 1390], length 0
20:34:50.760833 00:17:10:11:85:72 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 114, id 61262, offset 0, flags [none], proto TCP (6), length 44)
24.XXX.XXX.XXX.18734 > 184.XXX.XXX.XXX.21: Flags [S], cksum 0x6e87 (correct), seq 1122232626, win 0, options [mss 1390], length 0 -
@SquirrelSloth said in Can't get NAT to work: ISP modem >> (WAN) pfSense (LAN) >> (WAN) home router:
Test case
I have an FTP server running to test. With just my Asus router, everything works
When I put the pfSense between modem and router it does not workSo you say, if you connect the ASUS router to the internet directly and forward port 21 it works?
Settings on pfSsense
static route > 172.16.10.0/24 - gateway 192.168.1.2 - LAN interface
NAT port-forward >
Redirect target IP: tried both 172.16.10.46 and 192.168.1.2When forwarding to the router, you need also a forwarding on the router to the destination device behind it.
Packet captures only see traffic coming in from the Internet but never getting a response.
The capture was taken on the WAN obviously.
What do you see on the LAN?
Also sniff the traffic on the destination device.Remember that you have to allow the traffic to pass on the ASUS as well.
On the Asus router, I noticed I have to keep "Enable NAT" on in the WAN settings, otherwise I can't get out to the Internet.
This requires, that you set the outbound NAT to hybrid mode and add a rule on WAN for the source: 172.16.10.0/24.
-
@SquirrelSloth your going to have a hard time with ftp in a double nat. Are you wanting to use active or passive. Keep in mind with ftp 21 is the control channel. Then there is the data channel. And depending if your active or passive would determine which side will make the connection from, etc.
https://slacksite.com/other/ftp.html
If your not going to do nat on the asus router, then your rules on pfsense lan would have to be set to allow your lan side asus network. And you would have to setup routing on pfsense to send traffic to the asus wan IP to get to its lan, and yeah you would to make sure pfsense outbound nat is handling the asus lan network.
Normally you would want to avoid double nat. Why do you not just use your asus router as AP if you need it to provide wifi? Then your only natting once.
internet ---- 21 ---> public IP (pfsense) -- portforward1 asus wanIP--> (asus) -- portrward to ftp server IP--->
That is simple enough to do when both pfsense are natting.
But again that is just the control channel.. I would never attempt to get such a setup working.. Use sftp (ie over ssh), now there is only the 1 port to have to deal with. Or only single nat.
-
Thank you both @viragomann and @johnpoz for your insight.
To clarify, the intended purpose will not be FTP, I was just using this to be able to easily test my solution. Here's the full story of what is required:
Our small office currently has custom software that is accessed from the outside through port-forwarding on a small port range, and that works great. Now we also need to have 1 specific external IP print to our printer on port 9100, but when I open that port we see a lot of unwanted traffic, and yes it works also.
The router at our office is actually a TP-Link Deco mesh router and it let's me forward ports but does not let me lock it down to that 1 specific public source IP, so I bought the pfSense to help with that and keep us secure.
I couldn't get it figured out so I brought this pfSense home to play around with it with my Asus router I have and just trying with FTP server because I know that works from my home router without the firewall.
Not sure if the best way is to use only the pfSense as our router and somehow let the TP-Link just take care of DHCP and the wifi, so that we don't need to re-ip or change everything up too much. I'm just not really sure how to go about that
If I could avoid having to double-NAT or do whatever is the recommended way, I would like to stay with what the standard method should be. Would you have any suggestion for that?
(sorry I didn't give this full explanation originally)
-
@SquirrelSloth already went over how to not double nat..
Why would you want/have to expose a printer to the public internet? That is a good way to have prints of "goatse" start popping out of the printer.
Who is accessing this printer and software.. Why would you not just create a vpn between this location or users? Now your not exposing anything to the public internet.
-
@johnpoz , the printing is required because the software they are currently connecting to on-site is being migrated to AWS and that service needs to send the print jobs directly to our printer in the office, and our web developer told us this is what we need to do. We tried without the pfSense and the basic port-forwarding on 9100 open to everyone and we came in the next morning with a bunch of random garbage printed out lol (hence needing the firewall to limit to only that cloud server public IP). After the migration, the other port forwarding for the software will be removed and the ONLY this we need that firewall to do is let us print to the printer on port 9100 from 1 public IP only. Let me know if I am being confusing lol.
-
@SquirrelSloth you don't need to expose your printer to anything other than your instance in aws via a vpn you setup. Or you could use something like cirrusprint
-
@SquirrelSloth said in Can't get NAT to work: ISP modem >> (WAN) pfSense (LAN) >> (WAN) home router:
The router at our office is actually a TP-Link Deco mesh router and it let's me forward ports but does not let me lock it down to that 1 specific public source IP, so I bought the pfSense to help with that and keep us secure.
I couldn't get it figured out so I brought this pfSense home to play around with it with my Asus router I have and just trying with FTP server because I know that works from my home router without the firewall.
Not sure if the best way is to use only the pfSense as our router and somehow let the TP-Link just take care of DHCP and the wifi, so that we don't need to re-ip or change everything up too much. I'm just not really sure how to go about that
If I could avoid having to double-NAT or do whatever is the recommended way, I would like to stay with what the standard method should be. Would you have any suggestion for that?
Ultimately you should probably put the Deco in AP mode, and let pfsense handle DHCP and thereby avoiding double NAT.
But even in the double NAT situation, simply forwarding port 9100 to 192.168.1.2 should "do it"?
Since you already have that port forwarded on the Deco and you know that it works...@johnpoz said
internet ---- 21 [9100] ---> public IP (pfsense) -- portforward1 asus wanIP--> (asus) -- portrward to ftp [print] server IP--->In your description you seem to have done a lot of other things, like static routing and outbound NAT? None of that should be necessary...
The only thing you should need to do then, is to make sure the NAT rule you created has your desired single source IP that will be allowed to use that port 9100. -
@johnpoz both S2S VPN and Cirrusprint on AWS have a monthly cost. All we need is to let port 9100 through pfSense for that specific IP but not have the TP-Link router stopping it. I did the test again and ran packet capture on the pfSense LAN port as well. Further troubleshooting, I found that when I disable the "firewall" on my Asus router, the pfSense packet captures now show the router replying, but still can't make a connection (and this is the same output whether I enable or disable port-forwarding on the Asus router):
pfSense LAN port packet capture
09:20:31.946565 f0:ad:4e:3a:e1:cb > 04:42:1a:0f:a4:a8, ethertype IPv4 (0x0800), length 58: (tos 0x0, ttl 113, id 50626, offset 0, flags [none], proto TCP (6), length 44, bad cksum 0 (->6264)!)
24.XXX.XXX.XXX.17099 > 172.16.10.46.21: Flags [S], cksum 0x5487 (correct), seq 1229658268, win 32120, options [mss 1390], length 009:20:31.948053 04:42:1a:0f:a4:a8 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 63, id 15116, offset 0, flags [DF], proto TCP (6), length 40)
172.16.10.46.21 > 24.XXX.XXX.XXX.17099: Flags [R.], cksum 0xe962 (correct), seq 0, ack 1229658269, win 0, length 009:20:38.536745 f0:ad:4e:3a:e1:cb > 04:42:1a:0f:a4:a8, ethertype IPv4 (0x0800), length 58: (tos 0x0, ttl 113, id 50628, offset 0, flags [none], proto TCP (6), length 44, bad cksum 0 (->6262)!)
24.XXX.XXX.XXX.17123 > 172.16.10.46.21: Flags [S], cksum 0x546f (correct), seq 1229658268, win 32120, options [mss 1390], length 009:20:38.538169 04:42:1a:0f:a4:a8 > f0:ad:4e:3a:e1:cb, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 63, id 21311, offset 0, flags [DF], proto TCP (6), length 40)
172.16.10.46.21 > 24.XXX.XXX.XXX.17123: Flags [R.], cksum 0xe94a (correct), seq 0, ack 1229658269, win 0, length 0So it looks like it passes the pfSense but there's still something on the router I need to configure. These "home routers" are pretty basic and I can't even get tcpdump to work on my Asus. Is it possible to even change the router mode from "router" to "AP" or whatever that brand calls it so that I can make the pfSense be the firewall+router, have it do DHCP on the LAN and just have this TP-Link in the office act as the wireless AP?
Thinking if so, this may be what I need to change:
- change pfSense LAN IP to the address the router LAN was using (172.16.10.1) and enable DHCP with the same range, that way there's nothing to do on any of the devices in the network
- set router to AP-mode (not sure whether that router would be cabled the same-- right now pfsense(LAN) > (WAN)-router-(LAN) > switch > all devices... so I'm thinking to keep everything in the same subnet then I will not be using WAN interface of router, then I would just connect pfsense to switch and then the router LAN to any switchport???
I think I'm almost there lol. I do thank you for all this time helping me barrel through my dilemma.
-
@Gblenn I just read your reply after posting my last message, but ya I'm thinking the same. If I put it in AP mode, then there's no forwarding I need or any configuration on the router correct? Simply use this setup?
Internet > pfSense WAN (public IP) > pfSense LAN (172.16.10.1 + use as DHCP server + NAT rule on WAN side only) > switch > AP LAN port?
-
@SquirrelSloth using your current wifi router as just an AP is almost always going to be a better choice. And yeah there would only be single nat. And only need to forward on pfsense that has public IP on its wan.
-
@johnpoz So I did that and still nada. Since the Asus router is now only taking care of wireless, and the FTP server wired to the switch, I even just disconnected the router, so now the only setup is;
Internet > pfSense > switch > 2 devices (desktop + NAS running FTP server)
capture looks the same, seems the host is responding back but not making a connection.
Yet before this entire endeavor, I was connecting no problem from outside to the FTP server with only my router.
Thinking it's a return-traffic issue. Do I need some LAN rule to allow outbound? Only thing configured right now is
with below details
Tried NAT reflection as "use system default" and also changed to "enable NAT+Proxy" to test if that does anything. Outbound settings under that menu shows as follows
Creating that NAT automatically created a firewall rule (didn't specify any source IP yet... once I have it working I will do that)
and drilling down to that rule is below configuration
Is there anywhere else I need to be looking at? LAN rules are only the default ones.
-
@SquirrelSloth when you say nada.. What exactly do you mean? Your ftp client didn't get prompted for login? From what you posted your firewall rule it was evaluated.. That 0/14 KiB shows the rule was evaluated.
It very well could be return traffic, your now 172.168.10.46 is using pfsense as its gateway.. Sniff on pfsense on your interface when you do a test.. Do you see pfsense send the traffic on?
Here I just forwarded 21 on pfsense to something that doesn't even listen on 21.. But as you can see pfsense sends the traffic on.
Why I didn't get an answer is because 9.100 isn't listening on 21.. But from this pfsense job is done, it saw traffic on its wan IP and sent it on to the IP I told it too.. (this IP is valid - if pfsense can not arp for the ip it wont send traffic to it, etc) If you don't get an answer that is not on pfsense.. Pfsense is doing what it was told, send traffic you see on your wan port X to this IP on port X.. It has no control if that device your sending to answers. Or to be honest even got the traffic, pfsense just arps for the IP, ok there is mac and sends it to the mac, it has no idea if there might be filtering between it and the IP, etc.
You can see from my wan firewall rule, it was evaluated (I took that screenshot after I did the test so it wouldn't be 0/0)
Do the same test - if you see pfsense sending on the traffic when you test from say can you see me . org, then your problem is your device isn't even listening on 21, or it sent its answer elsewhere, or it running its own firewall, etc.
-
@johnpoz you are a genius! I went to that website and it told me it could not connect... I disabled AV software on that machine and it connected... not sure why without the PfSense I was able to connect to FTP server without playing around with my AV software, but hey at least that is resolved now.
Next test during the week is to bring this pfSense to the office, turn that TP-Link into AP mode and test port 9100 from that AWS server. I'll post back here if all is a success. Thank you again for your patience with me on this forum, I know this isn't how people want to be spending their weekends lol.
-
@SquirrelSloth said in Can't get NAT to work: ISP modem >> (WAN) pfSense (LAN) >> (WAN) home router:
not sure why without the PfSense I was able to connect to FTP server without playing around with my AV software
possible when the pc noticed a new connection it flipped back on? Windows for example will flip its firewall profile from private to public if the mac of the gateway changes, even if same IP scheme, etc.
To be honest in all the years have been here and using pfsense, I really don't recall ever seeing an actual problem with pfsense and port forwarding.. It always comes down to pebkac of some sort. The traffic not even getting to pfsense wan to be forwarded, client running some security software or not even using pfsense as its gateway, user set source port in the forward (this is very common). etc..
Glad you finally got it sorted, and yes football is starting ;)
-
@johnpoz / @viragomann / @Gblenn Thanks for all your help. I set it up that way and it was much easier, worked right away. I appreciate the time you spent helping me out on this :)
