Complex, working, config now needs to use CGNAT, UGH.
-
@chrisjx said in Complex, working, config now needs to use CGNAT, UGH.:
What I see are tricks that go directly to the server.
How?
Without a public IP, the network is not accessible from outside at all.
There are VPN providers which you can connect to, who forward one or a view ports to you for some fee.
Or if you still have a second location with a public IP you can forward the traffic from there, maybe with HAproxy.
Or you get a VPS with a public IP and set up a VPN server on it. Then you can connect the pfSense to it and forward traffic across the VPN.
-
Thanks for the reply, @viragomann ng around
Without a public IP, the network is not accessible from outside at all.
I get it; CGNAT has no IP address. So what I'm left with is "tricks" to work around that problem. I also mentioned a VPN service like WindScribe for which I'd have to pay and which introduces someone else I have to trust (although I have no indication that they are untrustworthy). I've also used ngrok for quick demos or temporary remote setups; works great for what it does.
What I'm looking for is some kind of connection "trick" that creates the magic tunnel from the outside world, perhaps a vps on digital ocean, that connects/terminates at pfsense not directly to the web server that I'm trying to expose to the internet. In my mind a VPS to web server behind pfsense makes it very messy every time I want to create a new service like that.
Right now, I have a single wildcard letsencrypt cert hooked up to all my haproxy service based (and subdomained) web servers and from pfsense to each webserver its just http. It is so clean and and easy to manage.
I am looking for a systematic approach instead of a patchy, one-off approach. Just asking for some expert advice from the community before I start abandoning a perfectly working config which provides all of the above just because of this tedious CGNAT problem. If there is no elegant approach, oh well.
To give an example of complexity (for me), I have no earthly understanding about how I might run pfsense failover from my primary cgnat connection to my secondary cgnat connection that also redirects the active WAN to one or more tunneled web servers behind my pfsense firewall. Or how to get the equivalent of dynamic dns between the 2 WAN connections. Or how tunneling through cgnat might interfere with my OpenVPN connections from my phone and my laptop into my home network.
One of the projects is a web site I've developed in nodejs express to provide access to family and friends for a photo side with thousands of family images going back to the 1870s. So far it requires over 500 G and climbing. It's just too expensive to host that on a cloud server. I'm currently running this behind a fiber WAN with cable as failover. It feels like a no-brainer compared to the hoops I think will be needed to jump through (or around) CGNAT. As I said... UGH!.
-
@chrisjx said in Complex, working, config now needs to use CGNAT, UGH.:
I am looking for a systematic approach instead of a patchy, one-off approach.
So all your text boils down to this. But Viragomann already named probably all of your options. Now you have to decide which route to take. Or maybe get a business-line without CGNAT...
-
@chrisjx Any reason you cant use Tailscale for this use case?
-
@chrisjx Maybe setup a Site to Site VPN to your main system, and use that for inbound access to the remote location.
User traffic originating within the remote site can still go directly out the two remote WAN's, avoiding the VPN.
-
@michmoor said in Complex, working, config now needs to use CGNAT, UGH.:
Any reason you cant use Tailscale for this use case?
Thanks for the response, @michmoor.
I just looked up tailscale and, after watching this video, it looks very interesting:
https://www.youtube.com/watch?v=P-q-8R67OPYIt looks like a tool that connects from me or a <= 3 team group for free. I am wondering how I host a nodejs/express photo site for friends and family without a need for them to set up tailscale configurations.
I currently have an open VPN connection which I use to securely connect into my network from my laptop of my phone. In fact my current home network with pfsense and 2 fail-over WANs using DDNS works great.
The challenge is for a similar setup in another location (a rural farm, perhaps a year or so out) where the incoming WAN connections will both be CGNAT.
Tailscale seems to be a service that can be connected directly to pfsense (that's my goal) while other alternatives seem to be from internet directly to the each server that I'd like to expose behind the firewall.
-
Having to deal with CGNAT is going to be challenging for sure. You will need traffic initiated from INSIDE to the outside in order to punch a hole. Like many have said, there are options. You could use Cloudflare tunnel (free) to open up the web servers securely for instance, use Tailscale (runs native on pfsense) or Twingate in order to get back into your home network.
As someone else stated, you could run a VPN (wireguard) from your pfsense to a Droplet and present the services from the Droplet aswell.
I have the your problem with 1 of my 2 ISPs for my dual WAN, one is a passthrough IP and the other is CGNET. I simply handle it with both Twingate and Tailscale, if/when I need to expose services to F&F I'll be using Cloudflare.
John
-
@johnwcahill said in Complex, working, config now needs to use CGNAT, UGH.:
when I need to expose services to F&F I'll be using Cloudflare
I tried cloudflare a while back to set up my DDNS and subdomain management but I wasn't smart enough to make it work. I deferred those services to digital ocean and it works great. When I get to the point of setting this up, I will give them another try.
run a VPN (wireguard) from your pfsense to a Droplet and present the services from the Droplet as well.
I get the theory on this approach but what is not clear is the end user experience. If my sister is at her computer browser does she access an https (letsencrypted) web page with my subdomain url and see the same as what I currently have - a subdomained URL, haproxy as endpoint for letsencrypt, redirected to http on the internal webserver? Is the URL directed to the IP of the Droplet? And is the path to the internal webserver more or less transparent to the end user?
Also, where might the failover happen between the 2 CGNAT WAN connections if the entry point is on the droplet?
I have 5 internal servers running as described above and while it took me a while to understand it, I love it's elegance. Most of the work is done within pfSense; the rest in Digital Ocean.
I'm somewhat reluctant to start down a path where I have to configure connections directly to each server from an external tunnel and manage each server's letencrypt updates, etc. My life's too short for too much of that. ;)
But, we do what we must do.
Thanks for your response.
-
@bob-dig said in Complex, working, config now needs to use CGNAT, UGH.:
So all your text boils down to this
Good one. Apologies for the verbosity. And thank you for your concise response. ;)
Sadly in the rural area where I want this home/farm network there are no "lines"; only non-prioritized Starlink, Hughesnet (gag), Viasat, T-Mobile Home Network (prob. the best of the lot), and a few feeble fixed wireless providers. I'm agitating for fiber down my road, which is 3.5 miles away, but not holding my breath.
-
@pwood999 said in Complex, working, config now needs to use CGNAT, UGH.:
Site to Site VPN to your main system, and use that for inbound access to the remote location
Thanks for the response. Is this the VPS solution others have described?
The idea, I think, is to set up wireguard (or openvpn) between a VPS and pfSense. The endpoint for users of my site is the IP/URL of the VPS, right?
I get the idea but it seems to relocate functions like WAN failover to the VPS instead of pfSense. I don't understand the plumbing on how this approach works.
Maybe I'm over thinking it and it's just a different way to do what DDNS does but for a non-ip CGNAT service.
-
@chrisjx Undestand that, CF is a bit of a challenge for me too. Your other option using Tailscale is a new alpha or beta function called Funnel which is suppose to let you expose services to the Internet. I have not looked at it completely yet. Also, Tailscale today announced big changes to their Free plan - it is what I have been using - very powerful set of features.
John
-
Maybe I'm over thinking it and it's just a different way to do what DDNS does but for a non-ip CGNAT service.
You need a so called jump host in the internet, free to reach from else where, that is connected to you home network.
Thats it, at a "Hoster" of your choice for some coin
per month and all is done.
