1:1 NAT doesn't reflect



  • Greetings!

    Been fighting with this a few days and I have something that kinda works, but it'll be tough to maintain.

    Requirements:

    • For the WAN I have a small block of static IP addresses. Last octet is 113 (pfSense) through 118 (gateway to Internet).

    • Previous firewall (old + slow) was set to bridging. Worked, but lacks features.

    • I'd like to start using load balancing soon. So bridging may not work.

    • Set up virtual IPs for 114, 115, and 116 (117 left open for debugging for now).

    • I have one service that absolutely positively has to match IP on incoming and outgoing traffic. Acts as both client and server depending on who it's talking to and what they're doing. Only uses DNS for original connection to its server. That is, it sits between a "meta-server" and the actual clients. The meta-server then takes the role of DNS for the actual clients (IP + port).

    • Of course DNS, FTP, web, and mail too.

    • pfSense sits between WAN, DMZ, LAN, and IPMI (management).

    • pfSense is running DNS forwarding on DMZ and LAN only, with strict interface binding set.

    • pfSense is running BIND on WAN only without forwarding or caching, authoritative only, master. Second DNS server as slave lives in DMZ.

    Some things I've tried:

    • Port forwarding NAT, with specific virtual IP + port, to specific DMZ machine.

    • Different settings of System/Advanced/Firewalls & NAT for NAT reflection mode.

    • Despite use of virtual IPs, port forwarding doesn't know that when a particular DMZ server initiates a connection to its meta-server, it needs to go out on the same IP address as what it listens on, not pfSense's IP address.

    Switched to 1:1 NAT:

    • Mostly works. That "weird" service that runs as both client and sever (and did I mention that it uses both TCP and UDP) is happy now.

    • NAT reflection fails for web, mail, and FTP, but works for ICMP (specifically ping) and my weird service.

    • Of course I've tried "pure NAT" and "NAT + Proxy". Neither works.

    • For 1:1 NAT I've only tried IP Alias, not CARP or anything else.

    A temporary solution is to put "Host Overrides" into the DNS forwarding service on pfSense:

    • Works for web 100% of the time.

    • Sort of semi-works for mail some of the time.

    • Doesn't work at all for FPT. It either grabs the wrong address, or when it gets the right address it can't get back a directory listing. This is with either active or passive.

    • I don't like this solution. I had to manually enter 49 host overrides just for this problem (I counted). Not terribly maintainable.

    When I plug my notebook into the WAN hub and give it the spare address, everything works, including FTP using active connections (not passive).

    What I'd love to see;
    NAT reflection working with 1:1 NAT. Or some other solution that will allow me to step up to load balancing in the near future.

    Thanks a million!

    Scotty



  • Update

    1:1 NAT works wonderfully. Because of my requirements, port forwarding NAT doesn't work for everything, but 1:1 NAT does.
    Outbound NAT (to WAN) from DMZ and LAN works wonderfully.
    Inbound rules from WAN to specific addresses and ports in the DMZ works wonderfully. Rule from LAN to everywhere works. And rule from DMZ to everywhere works, and is properly superseded by rule blocking DMZ to LAN.
    Host overrides on the DNS forwarder work for somethings but not others–deleted.
    NAT reflection doesn't work at all.

    Since NAT reflection doesn't work, I've dug around in my junk cabinet and found a old consumer-grade router and set that up to run beside pfSense to provide outbound NAT for LAN. I set a static route so that packets from LAN to DMZ go through the pfSense box instead. Router acts as primary gate for the LAN, and pfSense is secondary gate and DNS forward/cache. All this now works beautifully.

    Although this works it's not ideal. The ideal is for NAT reflection to work. It doesn't. Rather than a bug I assume I've done something wrong, but what?

    Help me please. How do I get NAT reflection to work?

    Thanks.



  • From everything you just explained here it seems that I'm not the only one facing this issue.

    I've been busy for the past few days to get a service/dedicated game server running within my LAN network, it comes with 2 things build into it, A webserver and the game server itself.
    Did some port forwarding, adding entries to the NAT tab under firewall settings etc, configure everything, ports, redirect IP and everything else. Yes everything seems to work fine for outsiders, friends can connect to the game server just fine, access the webserver as well, however, when I want to access both the game server and webserver using my WAN IP I run into some problems, I can access the webserver using the WAN IP but I cannot connect to the gameserver through the WAN IP.

    Yes my NAT Reflection and everything is enabled, I've followed many guides and threads on here, even went as far as going with the whole SPLIT DNS method but this still results in me not being able to connect to the game server using my WAN IP, using the LAN IP to connect works though so there's no trouble within the game or server as far as I know.

    I also tried disabling all of the NAT reflection settings and this resulted in both the webserver and the game server unreachable using my WAN IP, meaning the NAT reflection and 1:1 NAT settings seems to work only half for me.

    Sorry to bump into your thread like this.



  • Let's see if I've got this straight: Two networks, WAN and LAN. One external IP address on the WAN that belongs to your pfSense box. Running your game and web server on the LAN. And port-forwarding to connect clients on the WAN ("net randoms") to your game and web services. All this works.

    Then for you to connect you've tried NAT reflection, which works for the web, correct? But not for the game service. And you've tried split DNS so you can directly connect to your services without sending any packets onto the WAN, but that didn't work for the game service either. Is that about right?



  • That is correct, I've tried changing ports, try other things as well but nothing relevant enough to mention I think.



  • If your web service works but your game server doesn't, it's probably that game servers need to do a little bit of everything so they don't follow a nice clean client/server model of communication.

    I'm running several copies of a VR world server (for customers). It's a bitch of a protocol suite that runs on three levels: Uniserver, world server, and the VR client. Both the client and world server act as clients to the unisever–log in and authenticate, get their rights and partial config (how much land, how many simultaneous visitors). And the Uniserver runs a telegram service for the VR clients. The bitch is that when the world server logs into the uniserver--the uniserver gets the world's IP address and port number. Then it creates a list of running worlds. VR clients (people) click a world on the list and are "teleported" to that world--in other words, its a bit like a dynamic DNS but with a GUI front end. No way no how could anyone find the world server except by going through the same IP and port the uniserver has discovered. Another bitch is the world server handles chat (typed or audio) between VR clients in the world (which is dependent on distance between clients in the world) so there's probably a bit of push as well as pull for that. Objects in the world (models, textures, sounds, etc) are stored on a web server, but the "build data" where stuff is, any mods such as being retextured etc, are stored on the world server and given to the client on demand as UDP packets. So mix of TCP and UDP, just to make things interesting.

    I have a block of public IP addresses, so rather than using port forwarding I'm using 1:1 NAT. In the past I had a SonicWALL firewall setup in bridging mode: same IP addresses on both sides. And with an extra router behind the firewall to translate between a public address and the LAN. Worked great! Except I'm hoping to get into load balancing (not for VR stuff though) so I have high hopes for pfSense and 1:1 NAT.

    Except NAT reflection seems to be a complete fail with 1:1 NAT. Just doesn't happen. Tried split DNS and that mostly worked, but couldn't get FTP to work at all.

    Split DNS is fantastic for some uses--e.g. a corporate setup where employee's see the private version of their company's website. However it's nothing but a pain to me. I'm running authoritative DNS with 17 zones and some zones have many hosts. I have to duplicate all that by hand with DMZ addresses instead of WAN addresses, and I don't see how split DNS would work with load balancing.

    In my current setup I'm back to the cheap router to get packets between LAN and WAN. So no reflection in the pfSense box is needed. Still, it uses a public IP (which costs) and IMO reflection should work on demand. pfSense isn't a cheap consumer-grade product. If it can do redundancy and load balancing and all that, it should also do reflection.

    If you have only one public IP address, then a second router won't work for you.



  • VR world server? wait, are you talking about VRchat by any chance? I've always wondered how worlds are hosted there or what even is happening behind the scenes.

    I could use my cheap TP-LINK router as well and it would all work just fine, however it's limited with the amount of port forward entries (only allows up to a max of 10 port forward entries.) and I have much more than that, even working with the thing besides my pfSense box is not an option sadly, I do however have access to 2 ISP's simultaneously both of them with with dynamic IP's but again, using the cheap router besides the pfSense box is not an option.

    Honestly I didn't expect to encounter such an issue with pfSense at all, if you look at the fact a cheap consumer router has 0 issues with this at all, and then a high-end tier product can't even do something as simple as this(at least I think it's simple because cheap routers can do this). I still have the feeling I'm overlooking something but then again, I'm no network expert either, I just want a end-tier product that offers me more options than I need and just works with a bunch of customizations.



  • ActiveWorlds.

    Perhaps they've gotten better, but in the past cheap routers couldn't reflect at all.

    With 2 ISPs perhaps you should run your servers on one ISP and connect the rest of your LAN on the other. A bit extreme, but it'll work. Or look into getting a block of static IP addresses. It'll cost, but much cheaper than running a 2nd ISP. Generally, with a block of IP addresses the ISP knows and expects you to run servers. So no terms violation. Which ISP has the best uplink speed? And for a game server, which has the lowest latency?

    I agree. It should just work. Or more accurately, it should just be on or off, admin's choice, and then it should just do as the admin tells it.

    Everyone:

    Something I've wondered about: In virtual IPs I'm using plain "IP Alias". Should I be using "CARP" instead? And if so, how do I set that up because the wiki is kinda thin on that page.

    Thanks a million.



  • @ScottyDM:

    Perhaps they've gotten better, but in the past cheap routers couldn't reflect at all.

    With 2 ISPs perhaps you should run your servers on one ISP and connect the rest of your LAN on the other. A bit extreme, but it'll work. Or look into getting a block of static IP addresses. It'll cost, but much cheaper than running a 2nd ISP. Generally, with a block of IP addresses the ISP knows and expects you to run servers. So no terms violation. Which ISP has the best uplink speed? And for a game server, which has the lowest latency?

    The only reason I got 2 ISPs is because someone in our household wants to watch specific tv channels and they only offer that as a whole package with cable internet included, we never or barely use it, besides…. it's downtime is ridiculous, it's only available 73% of the time, upload is terrible, download is a little less than decent, and the latency spikes are all over the place. Anywho I've only hooked it up onto my pfsense machine because I could and in case our primary ISP (Which is Fiber.) ever goes down, which is never.


Log in to reply