NAT for multi web servers

gusto · Mar 16, 2021, 8:47 AM

I have a public IP address on pfsense.
I have one web server 192.168.1.101, where NAT is on port 80 and 443. This server (NAT) works great.

Now I have created another web server 192.168.1.104 and I have created another NAT on port 80. It does not work at all and I do not know where the error is.

gusto · Mar 16, 2021, 9:13 AM

I have already come to the root of the problem.
It is not possible to route 2x port 80 to one public IP.
I tried to block the rule on 192.168.1.101 servers. Then the rule worked on 192.168.1.104 servers.
But at the same time, I will not work.

Or would it be possible to work on one public IPv4 address with more than one web server?

Gertjan · Mar 16, 2021, 11:50 AM

@gusto said in NAT for multi web servers:

I have already come to the root of the problem.
It is not possible to route 2x port 80 to one public IP.
I tried to block the rule on 192.168.1.101 servers. Then the rule worked on 192.168.1.104 servers.
But at the same time, I will not work.
Or would it be possible to work on one public IPv4 address with more than one web server?

Basicly, you discovered the short of what NAT can do when using IPv4.
So that's why amongst others, IPv6 is invented.

Look what HAProxy can do for you.

Or, if these web servers do not have a public role, you can use one web server on port 80 - and 443 (as http is pretty dead these days).
Another one on 81 and 444
Et for 82 and 445.

johnpoz · Mar 18, 2021, 2:24 PM

@gusto said in NAT for multi web servers:

would it be possible to work on one public IPv4 address with more than one web server?

Yup - just use haproxy.. And your traffic can go to your different IPs behind pfsense via the sni (fqdn) used to access the site..

Example your public IP 1.2.3.4
public fqdn something.domain.tld points to 1.2.3.4
public fqdn otherthing.domain.tld also points to 1.2.3.4

In haproxy you set
something.domain.tld sends to 192.168.1.101
otherthing.domain.tld sends to 192.168.1.104

Or sure you can use the different ports way of doing it.. But reverse proxy is cleaner and easier for users since they don't have to know the port and can just use https://something.domain.tld or http://something.domain.tld

This also scales nicely and you could have hundreds of different fqdn point to either just one webserver or lots of different webserver IPs on your network. Where the webserver hosts up sites based on the fqdn, or just default page for anything that hits is IP, etc.

Also nice with haproxy setup is you can do ssl offloading if you want, where haproxy handles all the ssl certs. You can use wildcard certs, and also very easy to add ssl to something that doesn't actually support it in the application serving up the service. And on the backend of haproxy you just send to say http://localIP-Name:XXXX where XXXX is the port the service is being served on.

Also nice security feature of using a reverse proxy like this - if the sni (fqdn) isn't there then nothing gets forward. So this can stop bots that are just hitting your public IP on port X from being sent to your webserver behind pfsense. Just setup strict sni..

so if I hit my fqdn from something outside my network, I get served the site.. If I just hit the IP on 443 I just get told to go away ;)

I'm a big fan of the ssl offload feature because stuff serving up is via dockers on my nas, and they don't have ssl support native, and use odd ball ports 5055, 3579 etc. And since they are just dockers its much easier for them to just use the IP of device they are running on 192.168.9.10 in my case.. So the offload allows me to easy just use a acme cert, and then secure that traffic via ssl over the internet, but the service doesn't have to understand certs at all..

And since can control all the ssl stuff right in haproxy - I can make sure only appropriate ciphers are used, etc. So get a nice A when testing ssl for the service. No need to do any setup for ssl on the webservers.

gusto · Mar 18, 2021, 2:24 PM

So far, I'm a very newbie in haproxy. I found this guide, but if I want to test the http site from the wan, then I only see 503 service unavailable.
I also tried to change the pfsense port to 8888, but I always see only 503 service unavailable.

johnpoz · Mar 18, 2021, 2:28 PM

To what service... I recall seeing 503 awhile back because it thought the backend was down.. Changed to no check..

I think this was with ombi.

gusto · Mar 18, 2021, 3:50 PM

@johnpoz
If I make the settings according to that guide. Then I want to access the web server from wan, so I see in web browser

503 Service Unavailable

No server is available to handle this request

johnpoz · Mar 18, 2021, 4:49 PM

Yeah I get it - again change your backend check, validate that haproxy sees that the backend is up!

gusto · Mar 18, 2021, 7:09 PM

@johnpoz
I've tried trying to set it up in

Services / HAProxy / BackendEdit

The section Health check method setp up from http on none.
I don't know much about it, but it works now

johnpoz · Mar 18, 2021, 7:16 PM

Like I said if the health check that its doing doesn't work for whatever reason - it thinks the backend is down, then yeah you get a 503..

I never went into looking any deeper to why say the http check doesn't work for ombi service for example.. Because I only have 1 server, there is little need to actually know if its up or not for loadsharing, etc.