Double NAT



  • It has been driving me insane the last few days, but here it is.

    I've got a second cable modem with isp setup the isp for some reason doesn't want to provide bridge, they've setup NAT on the modem, I'm able to portforward dmz and stuff like that but that's it.

    So what i did was i've setup the router with a static LAN IP 192.168.10.10 the modem is 192.168.10.1 and ofcrouse it has a wan IP and nat enabled, .. I've setup DMZ to 192.168.10.10 so you would expect that portforwards would work but for some reason they don't.

    I've tried doing this with NAT 1:1 mapped 192.168.10.40 to 192.168.1.40, on my other router this works fine with real IP's however on the one with double NAT this doesn't work at all, ..

    Anyone any Idea's, ..

    when I nc -l <port>on the router I can nmap that port from the internet  however if i portforward it or nat1:1 it doesn't work.

    short:

    ipalias or ip arp proxy
    1:1 nat
    portforward isp
    
    

    doesn't work

    dmz isp
    portforward on pfsense
    

    doesn't work

    i've done a tcpdump all i can understand from it is that it stops at the 192.168.10.10 IP, which is the so called WAN IP</port>


  • Rebel Alliance Global Moderator

    And when your box your forwarding to answers.. What wan connection will it go out of - the normal wan, or this new double nat wan?

    You need to make sure your outbound nat is setup to use the correct connection.

    I would sniff on your lan interface to see that traffic is forwarded to your server, and the server answers.



  • I did but all stops at the router, .. if i make a socket on the router for instance

    nc -l 5060 then i can connect to it, but if i add a rule then it doesn't, the strange thing i just discovered however is, that UDP does pass through? only TCP is having issues and yes it's a new pfsense box and modem, ..

    and all outbound rules are on the WAN interface

    WAN	 	127.0.0.0/8	*	*	500	WAN address	*	YES	 	
    WAN	 	127.0.0.0/8	*	*	*	WAN address	*	NO	 	
    WAN	 	192.168.1.0/24	*	*	500	WAN address	*	YES	 	
    WAN	 	192.168.1.0/24	*	*	*	WAN address	*	NO	 	
    

    same for localhost ofcourse


  • Rebel Alliance Global Moderator

    "and all outbound rules are on the WAN interface"

    Huh??  There is no outbound rules on the wan..  All rules in pfsense are INBOUND to the interface..  Other than floating rules.

    Oh your outbound nat –- well how is that going to work??  If your traffic comes in in isp 2, and you send out the traffic in answer out isp 1?

    Your going to need an outbound nat rule for the traffic to go back out the correct wan connection.



  • Could you provide a sample cause it doesn't really make sense what it is you're saying.

    lets say i want to forward 5060, .. then i would make a forward rule however in the outbound what am i supposed to do there?


  • Rebel Alliance Global Moderator

    You need the nat to go out the interface you came in on.. You have 2 wan connections right..

    Do you need a picture??  So something comes in wan2, and then when try to talk back it goes out wan1 how do you expect that to work?




  • actually I've got 4 connections and 2 pfsenses, .. the thing is one of 2 works kinda, it has also nat issues, with your post this makes sense, however the new pfsesne has is new and has only one wan for now, .. the point is that it is behind a nat enabled modem, I can't disable it since it's the isp who delivers it like this, ..

    However I can setup portforwarding and dmz, which I did, on the new pfsense the one I'm having issues with, .. I can make a port listen nc -l 5060 for example and connect to it from the internet, however once i setup portforwarding on the pfsense box it self to forward that traffic to a IP within the lan it doesn't work.

    so basicly I can connect to the port however once I want to forward it, it doesn't work. So i think something is wrong in the NAT of the new pfsense, …

    so basicly this pfsense has wan IP 192.168.10.10 and I need to forward some ports to 192.168.1.40 all the traffic arrives only at the point of portforwarding it all goes wrong but i can't wrap my head arround it why.

    This is a forward, what would the corresponding outbound rule be.

    WAN	TCP/UDP	*	*	WAN address	5060 (SIP)	192.168.1.40	5060 (SIP)
    

    I thought these where those rules

    WAN	 	192.168.1.0/24	*	*	500	WAN address	*	YES
    WAN	 	192.168.1.0/24	*	*	*	WAN address	*	NO
    

  • Netgate

    You do not need outbound rules with only 1 WAN/default gateway.  pfSense is stateful.

    Reworded: The default rules are all you need.

    What's the corresponding firewall rule on WAN?



  • Euhm, I allow all traffic there's no rules needed, I guess I don't filter anything at this point.

    Edit:

    I need to verify something, .. maybe I thought of the reason, .. but not sure if this could be the cause. As  i said I have 2 pfsenses

    WAN => pfsense1 => 192.168.1.1/24
    WAN => pfsense2 => 192.168.1.2/24

    the pfsense1 does nat on that range and pfsense2 as well is and forwards traffic to the same range is that possible.


  • Netgate

    No rules means no connections allowed into WAN.



  • No I have rules but allow any.

    IPv4 *	*	*	*	*	*	none
    IPv6 *	*	*	*	*	*	none	
    

    So I don't really see the point in adding an extra rule, that specifically allows that traffic, because allow any  should match all.


  • Netgate

    No that's fine.  You'll have to figure out where the breakdown is then.  Check everything on the inside client (like its default gateway, firewall, etc.)



  • ok the idea is to add a sip trunk and backup sip trunk to have one go over one wan and the other over an other wan

    so basicly i forward the port 5060 to 192.168.1.40 on both pfsense, the gateway for the 192.168.1.40 is 192.168.1.1, however on the pfsense1 box I have a rule that when it matches the target IP it should use the pfsense2 as gateway so this should be fine.

    pfsense1 forward 5060 to 192.168.1.40
    pfsense1 if target IP 91.213.251.136 and dport 5060 make it use gateway 192.168.1.2 otherwise gateway 192.168.1.1
    pfsense2 forward 5060 to 192.168.1.40

    and finaly on both the same nat rules as above. I can reach the traffic up until pfsense2, so if i remove the forward rule and setup nc to listen on 5060 i can connect to it and send and receive data, however for some reason it doesn't want to forward that traffic to the 192.168.1.40


  • Netgate

    I can't decipher that.  Draw a diagram.  See the one in my sig for the type of information you should include.



  • I understand, .. can you recommend a unix/linux program to do so.



  • Sip/rtp will not work with double nat. You will need a sip server or a sip and rtp proxy  on the firewall(or beside) to workaround it.


  • Netgate

    It's hard enough with single NAT/PAT.

    For future reference:

    Free Diagramming Software: https://forum.pfsense.org/index.php?topic=1630.0



  • That's the whole point SIP does work but it uses UDP, I wanted to give clients access to it using TCP cause in some networks UDP can't get passed a proxy, you need TCP but sip does work when it's using UDP it uses the same rules as TCP but for some reason TCP doesn't work.

    And I have a sip server


  • Rebel Alliance Global Moderator

    "You do not need outbound rules with only 1 WAN/default gateway.  pfSense is stateful."

    Agreed.. But from his first few statements..
    "I've got a second cable modem with isp setup the isp for some reason doesn't want to provide bridge"

    But seems he using a different copy of pfsense, which is my bad for not reading it correctly.  But which is why I asked if 2 wan connections.
    "You have 2 wan connections right.."
    " What wan connection will it go out of - the normal wan, or this new double nat wan?"
    "If your traffic comes in in isp 2, and you send out the traffic in answer out isp 1?"



  • double, still working on that network map, have it ready in a bit



  • sorry not really good in these drawings tried to be as clear as possible






  • What ISP are you using, I may be able to help? Why not put your firewall in the DMZ of the ISP NAT device. Most of the time their DMZ will forward all unsolicited traffic to the DMZ IP.



  • The ISP nat is in dmz to 192.168.10.10, .. the target is the router, ..


  • Netgate

    Aren't we just wasting time trying to double-NAT SIP?



  • Who is your ISP?



  • The ISP with the issue is Telenet it's european, .. and no where not wasting time trying to make sip work, sip already works it uses udp. I'm trying to forward a tcp port, with netcat listening on the end, if you want I can make it port 10000, but i don't think this will change the issue, .. why is portforward not working i think it's strange. It shouldn't matter if the traffic before it is NAT it has dmz all is forwarded, .. so why does the traffic stop at the 192.168.10.10 IP

    If I setup netcat on IP 192.168.10.10, then I can connect to it, then if I forward the port, it can't connect, I'm pretty sure it's some configuration issue. But I wouldn't know where to start.

    # sysctl -a | grep forward
    kern.smp.forward_signal_enabled: 1
    net.inet.ip.forwarding: 1
    net.inet.ip.fastforwarding: 0
    net.inet6.ip6.forwarding: 1
    # 
    
    nat on vr0 inet from 127.0.0.0/8 to any port = isakmp -> 192.168.10.10 static-port
    nat on vr0 inet from 127.0.0.0/8 to any -> 192.168.10.10 port 1024:65535
    nat on vr0 inet from 172.16.1.0/24 to any port = isakmp -> 192.168.10.10 static-port
    nat on vr0 inet from 172.16.1.0/24 to any -> 192.168.10.10 port 1024:65535
    nat on vr0 inet from 192.168.1.0/24 to any port = isakmp -> 192.168.10.10 static-port
    nat on vr0 inet from 192.168.1.0/24 to any -> 192.168.10.10 port 1024:65535
    rdr on vr0 inet proto tcp from any to 192.168.10.10 port = 5060 -> 192.168.1.40
    rdr on vr0 inet proto udp from any to 192.168.10.10 port = 5060 -> 192.168.1.40
    rdr on vr1 inet proto tcp from any to 192.168.10.10 port = 5060 tag PFREFLECT -> 127.0.0.1 port 19000
    rdr on vr1 inet proto udp from any to 192.168.10.10 port = 5060 tag PFREFLECT -> 127.0.0.1 port 19000
    

    also i can connect to the ssh pfsense from the outside, however I can't forward it to any other ssh



  • I think I know what the problem is, .. however I'm not sure how to solve this

    from what i gather

    tcp:

    SYN =>
    <= SYN ACK
    ACK =>
    connection established
    

    however, ..

    WAN2 is a secundairy gateway while WAN1 is the default gateway, .. traffic is comming in through WAN2, however since the WAN1 is the default gateway of the client, it responds through WAN1, connection failed.

    Exactly like johnpoz said earlier.

    edit:

    yes, if i change the route for that specific ip i can connect to the tcp, however now my question how can i make a dynamic route so that when I connect to it that traffic goes through the correct gateway. And that's why UDP works. It all makes sense now.

    can this be solved with a routing daemon?