AWS on PFSense. What's the proper config for routing?
-
I'm having a tough time getting a pfsense firewall configured correctly on AWS. I have a public subnet and private subnet configured on AWS and have attached the interfaces (eth0 172.16.2.10 & eth1 172.16.3.10) as outlined in https://www.netgate.com/docs/aws-vpn-appliance/vpc-guide.html. I then threw up a generic Windows machine (ip address 172.16.3.50) on the private subnet so that I could test basic connectivity (ping and RDP). I can't ping from the Windows 2012 box from LAN to WAN out to the Internet. I've set the default route for the RT on the 172.16.3 subnet to point to the interface ID of the eth1 subnet interface. I can ping the WAN address from the server but not out to 8.8.8.8 on the Internet. I even set a static route in the PFSense instance for 8.8.8.8/32 to the subnet gateway, currently 172.16.2.1. I've got the rules basically saying any any any trust on both the LAN & WAN side. I've verified the source/destination check is selected for both interfaces on the firewall. What am I missing?
-
You mean source/dest check is disabled right?
That has to be disabled on any VM (or at least interface) that needs to pass traffic that does not include that interface's IP address in either the source or destination address of the IP packet.
I would delete whatever static route for 8.8.8.8 you made. Likely unnecessary.
Be sure outbound NAT on the pfSense WAN includes the 172.16.3.0/24 source. pfSense treats DHCP interfaces as WANs so they will not be included in Automatic Outbound NAT. You might need to statically address the LAN side with the address assigned in AWS/EC2 to eliminate that behavior. Interfaces > LAN
-
Yes, I meant source/dest check is disabled....on both Pfsense interfaces. I even disabled it for grins & giggles on the windows box as well just to see if anything would happen.
Static route to 8.8.8.8 is gone. Again, just trying to make that magic ping box start working.
I've statically assigned the LAN interface to the address assigned to me by AWS. I've enabled AON and I'm still not able to ping out from the Windows box.
One thing I noticed is the automatically generated rules show a 172.25.53.0/24 (along with 127.0.0.0/8) network in the source networks. That subnet isn't anywhere I can see in this VPC. At first I thought it somehow carried over from another VPC my company has but that VPC is in 172.25.0.0/20.
-
pcap on LAN for icmp host 8.8.8.8. ping from windows. Do you see it?
Then do the same thing but pcap on WAN. Do you see it?
-
Yes, I see it on both interfaces.
20:14:53.964442 IP 172.16.3.50 > 8.8.8.8: ICMP echo request, id 1, seq 629, length 40
20:14:58.964264 IP 172.16.3.50 > 8.8.8.8: ICMP echo request, id 1, seq 630, length 40 -
If that is on WAN it is not getting outbound NAT. Traffic out WAN should be translated so the source address is the WAN address so replies come back to it.
-
Right, I agree. If I enable AON in the NAT settings I'm seeing that strange subnet again. Perhaps I did something wrong when I setup the VPN?
WAN 127.0.0.0/8 172.25.53.0/24 * * 500 WAN address * Auto created rule for ISAKMP
WAN 127.0.0.0/8 172.25.53.0/24 * * * WAN address * Auto created rule
LAN 127.0.0.0/8 172.25.53.0/24 * * 500 LAN address * Auto created rule for ISAKMP
LAN 127.0.0.0/8 172.25.53.0/24 * * * LAN address * Auto created rule -
If you changed LAN to static you should not be seeing that outbound NAT there. Unless you changed to Advanced, manual NAT before making that change.
So look at the routes and see what interface that "strange" network is on. Probably OpenVPN or something.
Manually add a rule for 172.16.3.0/24 on WAN then figure out what that other subnet is.
-
Confirmed LAN is on static and set to 172.30.3.10. I might have switched to manual NAT before choosing Advanced during this debugging process. Very possible. Not seeing anything that corresponds to that subnet anywhere in the Routing section under System. No idea where that's coming from.
Manually added and still no ping out.
Int Source Source Port Dest Dest Port NAT Address NAT Port Static Port Desc Actions
WAN 172.16.3.0/24 * * * WAN address *Also ran another packet capture and it's still not translating.
-
@derelict Shouldn't the WAN address be the elastic IP address AWS assigned me instead of the public subnet IP address?
-
No. The VPC NATs from the interface IP address to the Elastic IP on the igw.
-
@derelict Well, then I'm out of ideas. Thanks for trying.
-
How about you post screen shots. Something might not be set how you think it is.
LAN, WAN, Outbound NAT.
Maybe a screen shot of the states filtered on the interesting traffic.
Are you getting any alerts on the dashboard that the rule set isn't loading or anything like that?
-
@derelict I really hate stupid things that defy explanation. I just changed the manual NAT rule source address to any from the 172.16.3.0/24 network and ping started going through. I changed it back to 172.16.3.0/24 and it's still working. It's times like these I wish I had picked a different career
-
Maybe didn't hit apply? Were you running a continuous ping and didn't stop/start it after changing outbound NAT?
-
@derelict Yes, it was continuous.
-
Then changing NAT would not affect the already-established state. You would have had to stop and restart the ping or kill states.
-
@derelict Yup, rookie move. Thanks for sheparding me around. Now on to 1:1 NAT. I hope it's not as complicated as this process took. Any configuration guides you can point me too?
-
@joshuamichaelsanders 1:1 NAT for what?
-
@derelict Host a web server, mail server, etc.
