[Solved-Sort of] Client -> Server, but server has RFC1918 WAN address!



    • Client (any) cannot connect to the WAN int on a newest pfSense box, which is RFC1918:

    • The Internet connection was:
      1.1.1.1

    • The Internet connection is now (allegedly):
      123.123.123.117
      That is on the outside of a Comcast Cable Modem, which supplies 10.1.10.101/24 as my FW IP. It's GW IP is 10.1.10.1

    • My newly installed pfSense FW gets that IP DHCP from the Comcast modem.

    • The IP address was supposed to have been changed months ago, but when I check my public IP, it is back to the old one. I know that this is just the IP I am seen as going OUT, and that going IN might in fact be the new one. I cannot get clarification so far, but I suspect unnecessary weirdness.

    I cannot tell my cell/workstation clients to use the WAN Interface IP, as it is not route-able. When I set up my OVPN Server for the public IP address, I see it connecting in the log, but the client side says bad/no TLS handshake, and to check network connection. I was going by this guide, but I fear I may have switched a bad option in client export, saving as "default", which I assume is required on the client export page.

    If I setup the Client Exports with the actual IP, it won't work, and there is a mismatch in IP addresses between the .ovpn config (routable) and the fw actual IP (non-routable)...

    Feels like my first time:
    What is weird, is that I have several site-2-site and client-server VPNs up and running for different clients. I don't understand every possible option, but I never created a very complex setup (except one, with Atheros WiFi, etc.)...

    Status:
    I've tried full client installs, just configs, separate certs, and ovpn files on cell and remote workstations, and I'm lost. I'm at that point where you can only whack it all and start over again, over and over...

    Testing:
    I was trying to make sure the new public IP was valid, and so I opened up port 80 on the WAN interface to see if I could get to the GUI, but no go.

    Test:
    From a LAN net on the new FW, I ping the 'new' IP address, and get:

    pat@Beastie:~$ ping 123.123.123.118
    PING 123.123.123.118 (123.123.123.118) 56(84) bytes of data.
    From 10.1.10.1: icmp_seq=2 Redirect Host(New nexthop: 123.123.123.118)
    

    [...]
    ...which I read as my request going out the 10.1.10.101 WAN interface to it's upstream GW 10.1.10.1, and at that point the modem sends it right back. Do y'all see it that way?

    Needed:
    Until I get clear network maps, filters, routing, etc., I'd really appreciate hints on what to do to troubleshoot and figure a way to get my remote clients to be able to connect to a non-routable WAN IP. :)

    TIA!

    P.S. When I get it all working, I will fiddle with the 2 checkboxes for blocking bogon and RFC1918 addresses on the WAN Interface. I'll probably continue to leave those 2 unchecked and do it all in FW Rules...



  • @BogusException

    If your WAN has an RFC 1918 address, there's no way a client can connect to the OpenVPN server. Comcast is moving to carrier grade NAT for IPv4, due the shortage of addresses. They are also providing IPv6. Is IPv6 available for your clients?

    IPv6 vs Carrier-grade NAT



  • Well, it took a bit of testing, but here is what ended up working...
    [note: I a musing 123.123.123... to represent an internet routable address range]

    alt text

    The green ("External") FW is the one of interest.

    123.123.123.117 was being routed to the LAN segment on the WAN side of the FW.
    I simply manually configured the WAN int to pick it up, with the /29 and .118 as it's gw.

    End. Read on for VPN config errata...

    Not part of this, but for anyone doing cascading VPNs, there are VPN servers on both the Ext & Int.
    I chose the simple way out and port-forwarded an off-IP for ovpn (like 1192) as the server port for the Int FW.
    This way, I could leave the Ext FW on it's own port (1194)

    There is an NVR off a bastion LAN on the Ext FW, which is the main rationale for that FW. That LAN is non-routable, but no sweat because the LAN segment I point the VPN to is the one it is on.
    IOW, if by some chance someone was able to get through that VPN-even with the user's creds on their cell phone, the only LAN they can see is the sacrificial/bastion one with the NVR, and that is the only host, and only port 80 is open, passworded.

    I hope the WAN interfaces are obvious.

    Final note: I believe that manually editing the exported OVPN for clients (point to public .117), I could get away with the WAN interface being RFC1918 (10.1.10.0/24)-as long as the upstream router forwarded my public IP to it. Right now, it does not, and I don't manage it, so this is all I can do for now.

    Can't tell if this will be of any use to someone in future.



  • Mods: I tried to fix a typo on line 2 (the 'note:), but was told Akismet thought I was posting spam...

    Not for the post, but an update/correction to the post. I run a lot of blogs, and I never saw it pull that before...



  • @BogusException

    Do you in fact have a routeable address? Or RFC1918? In your original post, you said your WAN IP was 10.1.10.101.



  • @JKnott Thanks for writing. The answer is yes. And this is because I allowed my WAN int to pick up a DHCP address from the ISP's router at 10.1.10.1, serving reservations at .101-199.

    The follow-up post shows that because I cannot configure that router, I made my External FW WAN interface the actual public one (123.123.123.117), even though it was not sitting on that LAN. Those static routeable addresses are turned over to that 10.1.10.0/24 LAN.

    This was done by the ISP when they switched out the public addresses I mentioned at the top of the original post. Everything was fine until then! :)

    Sorry it is confusing to explain, but on the drawing you will see that the WAN interface of the external FW still sits on that network.



  • This is S.O.P. for Comcast Business class. Their equipment always runs a private range, but you can use addresses from your public range. There is generally an option to turn off the firewall for your statics in the modem config.



  • @dotdash Thanks for writing. The 10.1.10.0/24 is a dead giveaway it's Comcast, eh?! :-)

    Although I can get in with the standard cusadmin/highspeed, I told the landlord I wouldn't adjust their stuff. The change of ISPs triggered this wonderful evolution.

    I also hoped that the port forwarding approach would be helpful in future if anyone was contemplating stacking/cascading VPNs! :-)



  • @BogusException said in [Solved-Sort of] Client -> Server, but server has RFC1918 WAN address!:

    The follow-up post shows that because I cannot configure that router, I made my External FW WAN interface the actual public one (123.123.123.117), even though it was not sitting on that LAN. Those static routeable addresses are turned over to that 10.1.10.0/24 LAN.

    That doesn't make sense. Where is the real 123.123.123.117? If it's on the other side of Comcast's NAT, it can't possibly work as you describe. The interface where they assigned it is nowhere near your modem. It will be at their head end and shared by other customers. Just using that same address internally will not cause a VPN to work from elsewhere.



  • @JKnott said in [[Solved-Sort of] Client -> Server, but server has RFC1918 WAN

    That doesn't make sense. Where is the real 123.123.123.117? If it's on the other side of Comcast's NAT, it can't possibly work as you describe. The interface where they assigned it is nowhere near your modem. It will be at their head end and shared by other customers. Just using that same address internally will not cause a VPN to work from elsewhere.

    They only NAT the private range, the public range bypasses NAT, and the firewall, if you check the box.



  • @dotdash

    So they're providing both public and RFC 1918 addresses at the same time??? You must have a very unusual setup. I have never heard of such a configuration and, as the article I linked to mentions, Comcast is moving customers off public IPv4 addresses and onto NAT and IPv6. With IPv6, ISPs may provide both public addresses and unique local (the IPv6 equivaent of RFC 1918), as mine does, if I put the modem into gateway mode.

    Still the best solution is to use IPv6, if your customers have it available.



  • @JKnott
    It is unusual, but it's the standard Comcast setup when you have a business account with static public IPs. For residential, or lower-tier business accounts, you get a dynamic public IP. I'm talking about v4, but they are now providing a static v6 block with the v4, and a residential user gets a dynamic /60.