N00b help with NAT



  • Hi all,
      I'm new to pfSense, but not new to *nix routing in general.  I've been doing it for a few years, though I've never used a BSD distro.  So far I'm very impressed with how well pfSense setup was thought out.  I was able to install pfSense to an 8 GB usb, and then copy it to 2 other backups.  We have a very low wattage tower with 4 nics to support our infrastructure.

    Basically I'm trying to perform some NAT, and I'm getting some errors that I can't seem to suss without some help.  I've read the doc here and here.  Here is my physical device to Interface mapping

    LAN sk1
    WAN fxp0
    DMZ sk0
    WAN-Wireless dc0

    Now, I basically want to NAT port 80 and 443 on the WAN port to 10.0.1.11 within our LAN.  Eventually this virtual machine will be moved to the DMZ, but we currently don't have the hardware to put it on.  I have the following 2 rules for NAT.

    WAN  TCP  80 (HTTP)  10.0.1.12(ext.: any) 80 (HTTP) Redmine http
    WAN TCP         443 (HTTPS) 10.0.1.12(ext.: any) 443 (HTTPS) Redmine HTTPS

    I also used the "auto create firewall rules" and I have these corresponding firewall rules

    TCP  *  *  10.0.1.12  80 (HTTP)  *      NAT Redmine http 
    TCP * * 10.0.1.12 443 (HTTPS) *   NAT Redmine HTTPS

    However no one from outside the system can access the server.  If I use the internal IP of 10.0.1.12, the server responds.  When I use the terminal and tail the filter logs, I always see this when someone tries to hit port 80 or 443 (as port 80 redirects to port 443)

    3. 105219 rule 364/0(match): block in on sk1: 10.0.1.12.443 > 60.234.165.169.1508: [|tcp]

    With our old crappy Linksys, we were able to NAT this connection, so I don't believe the problem is with Redmine as it hasn't changed.  I'm positive I just have some rule I've screwed up that is accidentally blocking the Redmine response, but I can't seem to find it.  Any help would be greatly appreciated!



  • Set the external address to the interface IP and not to any.
    Make sure you dont have the pfSense webinterface on a port you're trying to forward.



  • Thanks for the help, but unfortunately that doesn't appear to have fixed my problem.  I deleted the firewall exceptions as well as the NAT, then re-added them from scratch.  I can ssh into my firewall then use telnet to test that I can reach port 443 on address 10.0.1.12, which works.  Is there a way I can use the tcpdump command to filter only on forwards to 10.0.1.12?  Here are my new rules

    NAT

    WAN  TCP  80 (HTTP)  10.0.1.12(ext.: 201.114.178.182) 80 (HTTP) Redmine HTTP 
    WAN TCP          443 (HTTPS) 10.0.1.12(ext.: 201.114.178.182) 443 (HTTPS) 10.0.1.12

    Firewall
    TCP  *  *  10.0.1.12  80 (HTTP)  *      NAT Redmine HTTP   
    TCP * * 10.0.1.12 443 (HTTPS) *   NAT 10.0.1.12

    I've also tried changing the origin Firewall rule to use the WAN interface instead of the internal IP, but I haven't had any luck with either.  In both scenarios for the firewall, nmap run on an external device (my server at home) doesn't detect an open port on 80 or 443.  Just an update, I know the port forwarding is working correctly as I have enabled NAT reflection and can get to my host via the public IP and port 443.

    Thanks,
    Todd



  • After a bit more testing, I believe something is horribly wrong with with my installation, but I'm not sure what.  All the client computers on my LAN and DMZ can get an IP from DHCP.  My LAN computers have access to the WAN and internet, but not DMZ.  My DMZ can't access the WAN.  I have removed all NAT rules and all firewall rules and started with basic rule definitions, but these do not appear to work.  Any ideas on what is going wrong?








  • Just a follow up.  After rebooting this solves my issue with routing, but I'm unclear why I have to reboot.  I'm also still experiencing the firewall issues I discussed before.  Any ideas?



  • I experienced a similar issue with routing / NAT yesterday.  I played around with the configuration for hours trying to get it to work, reboots in between etc and had no luck.  Finally I reinstalled and everything seems to be working as designed, since!!

    Seems as though sometimes a buggy installation creeps in and messes things up…..  Try just booting up with the CD version and see if your rules work the way you have been creating them.



  • Did you load your previous config file, or did you manually re-define the rules?  Every time I try to connect via external IP, I always receive this error in my logs.

    Apr 6 08:49:58 kernel: arpresolve: can't allocate route for 203.114.178.182
    Apr 6 08:49:58 kernel: arplookup 203.114.178.182 failed: host is not on local network

    That's my WAN IP, trying to route to my LAN with NAT.



  • After a bit more testing, I believe that the problem is not with pfsense blocking the incoming connection, but rather than Apache on my internal server can't respond to the request.  I see this in my filter log (from ssh session) when I connect via a Cellular modem that's outside our network.

    firewall:~#  tcpdump -vv -i pflog0 -n port 443
    tcpdump: WARNING: pflog0: no IPv4 address assigned
    tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 96 bytes
    14:01:06.897807 IP (tos 0x0, ttl 121, id 34339, offset 0, flags [DF], proto TCP (6), length 52) 166.179.147.4.3371 > 10.0.1.12.443:  tcp 32 [bad hdr length 0 - too short, < 20]

    EDIT
    I was using the default arguments on tcpdump, so it couldn't read the packets.  After increasing the size to 256, this is what I get

    tcpdump -vv -n -s 256 -i pflog0 port 443

    tcpdump: WARNING: pflog0: no IPv4 address assigned
    tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 256 bytes
    15:12:52.625642 IP (tos 0x0, ttl 64, id 18100, offset 0, flags [DF], proto TCP (6), length 52) 10.0.1.12.443 > 166.179.147.4.3846: S, cksum 0xec83 (correct), 1905843377:1905843377(0) ack 4067546480 win 5840 <mss 3="" 1460,nop,nop,sackok,nop,wscale="">This usually appears in the log about 1 to 2 minutes after I make my request over my 3G wireless.  The apache server at 10.0.1.12 only takes < 1 second to respond to requests on the lan, so it shouldn't take that long via NAT.

    Attached is a screenshot of the firewall log that shows my rules are set up correctly.  Is it possible that pfSense is closing the socket before a response has been sent back from Apache?

    ![Firewall Entry.jpg](/public/imported_attachments/1/Firewall Entry.jpg)
    ![Firewall Entry.jpg_thumb](/public/imported_attachments/1/Firewall Entry.jpg_thumb)</mss>



  • So, full round trip, this is what I've been able to diagnose.

    1. Outside client requests connection via https:// <my wan="" ip="">2. pfSense performs this operation (according to tcp dump)

    tcpdump -n -vvv -s 256 -i pflog0 host 10.0.1.12

    tcpdump: WARNING: pflog0: no IPv4 address assigned
    tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 256 bytes
    17:32:18.326266 IP (tos 0x0, ttl 121, id 53974, offset 0, flags [DF], proto TCP (6), length 52) 166.179.169.245.1516 > 10.0.1.12.443: S, cksum 0x80cd (correct), 582053297:582053297(0) win 64240 <mss 1360,nop,wscale="" 0,nop,nop,sackok="">3. Cent OS performs these network calls

    [root@localhost ~]# tcpdump -n -vvv -s 256 -i eth0 port 80 or port 443
    tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 256 bytes
    17:32:17.420932 IP (tos 0x0, ttl 120, id 41407, offset 0, flags [DF], proto: TCP (6), length: 52) 166.179.169.245.vpad > 10.0.1.12.https: S, cksum 0x80cd (correct), 582053297:582053297(0) win 64240 <mss 1360,nop,wscale="" 0,nop,nop,sackok="">17:32:17.421136 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: TCP (6), length: 52) 10.0.1.12.https > 166.179.169.245.vpad: S, cksum 0x45e8 (correct), 2244581572:2244581572(0) ack 582053298 win 5840 <mss 3="" 1460,nop,nop,sackok,nop,wscale="">17:32:20.419555 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: TCP (6), length: 52) 10.0.1.12.https > 166.179.169.245.vpad: S, cksum 0x45e8 (correct), 2244581572:2244581572(0) ack 582053298 win 5840 <mss 3="" 1460,nop,nop,sackok,nop,wscale="">17:32:26.416671 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: TCP (6), length: 52) 10.0.1.12.https > 166.179.169.245.vpad: S, cksum 0x45e8 (correct), 2244581572:2244581572(0) ack 582053298 win 5840 <mss 3="" 1460,nop,nop,sackok,nop,wscale="">17:32:38.411045 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: TCP (6), length: 52) 10.0.1.12.https > 166.179.169.245.vpad: S, cksum 0x45e8 (correct), 2244581572:2244581572(0) ack 582053298 win 5840 <mss 3="" 1460,nop,nop,sackok,nop,wscale="">17:33:02.599279 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: TCP (6), length: 52) 10.0.1.12.https > 166.179.169.245.vpad: S, cksum 0x45e8 (correct), 2244581572:2244581572(0) ack 582053298 win 5840 <mss 3="" 1460,nop,nop,sackok,nop,wscale="">4. My state table shows this.

    Proto      Source -> Router -> Destination      State     
    tcp 10.0.1.12:443 <- 203.114.178.182:443 <- 166.179.169.245:1516 SYN_SENT:ESTABLISHED
    tcp 166.179.169.245:1516 -> 10.0.1.12:443 ESTABLISHED:SYN_SENT

    5. The browser shows a network timeout

    Ive tried this from several physical connections.  My 3G wireless, home, and a friend in the US.  All of the connections end with the same result.  I've also tried routing requests to an IIS Windows Server we have in the DMZ just to see if it's an issue with our CentOS Server, but I get the same results.  All NATs  just don't seem to get replies.  I've also had this problem with ssh NAT</mss></mss></mss></mss></mss></mss></mss></my>



  • Anyone?  I could really use the help.  I don't want to give up on pfsense, and go to a vanailla unix/shorewall.  pfsense has some great features, but I need to get this running.  I've been struggling with this for 3 business days now…



  • Well after several hours with tcpdump I finally got it running.  The issue was my DSL modem.  It's a D-Link DSL-502T, I don't recommend it.

    I tried using PPPoE on pfSense to authenticate as it seems some people have had success authenticating against an actual PPPoA authentication.

    I had to set the modem to "half-bridge" mode as pfSense can't auth directly due to PPPoA.  In this mode the modem authenticates to our ISP and puts the public IP on the ethernet port in the modem instead of setting up a LAN.  For some reason, it was blocking responses, and there was no firewall our routing enabled on the modem.  As a workaround, I set the Ethernet to a static IP on the modem, then put the WAN port of pfSense as the DMZ until I get a new modem.  Once I did this, everything works as expected with responses routing correctly back to our ISP.

    We have a significant performance boost in our routing and NAT even with Snort running.  The box we turned into our Firewall is an old P4 2 GHZ with 1 GB of ram.  Thanks to everyone for all the hard work put into pfSense.  It's the best firewall distro I've used.


Log in to reply