Webserver SSL is not private (How to set)
-
@carrzkiss said in Webserver SSL is not private (How to set):
We have been hosting these servers for several years using a Belkin Router.
How did you have it before? A common practice is to incorporate HAproxy package, especially since you're hosting several.
-
@carrzkiss Another possibility, if you are testing from inside your LAN, did you enable reflection on the NAT rule. Otherwise you'll connect to pfSense's WAN IP. Also for this case, ensure Enable automatic outbound NAT for Reflection is checked.
Hosting multiple servers that have private IPs requires a unique port forwarded per server, unless a proxy is used as noted.
-
Here are some screen shots
LAN
NAT
WAN
-
@carrzkiss
So pretty all your WAN rules show passed packets. So I assume that they arrived at the stated IP in the forwarding.Could the access be blocked on the web server itself?
Possibly the former router did masquerading on incoming traffic, which is a quick dumb solution to get it work.Just out of curiosity, why you need incoming DNS access on WAN?
-
What I have done.
(Testing from Inside and Outside of the Lan.
The outside is using my Cellphone and my carrier's service.)Changed to using a /16 IP Subnet
Changed the Gateway IP From 2.1 to 4.1Changed all Servers to reference the new Gateway and Subnet
All servers can see each other and communicate with the new IP Subnet and Gateway.When I first connected the Switch of the running Web Servers to the Switch with PFSense connected, I jumped onto one of our sites and received the SSL Error.
At that point, I researched how to get SSL To work with PFSense.
I found a video series, which I followed and set up.
Once I created the new Cert with PFSense/LetsEncrypt.
I exported the PFX (P12) file.
Imported and then distributed the working PFX to all Web Servers.The ARR Servers are connected through "Microsoft Load Balancer" and are assigned a single IP Address that takes the Port Forward. (Think of Facebook, YouTube, and others. That is how this is set up. Many servers are connected together, with a single IP Address on a Load Balance.)
This way, I do not have to point to each server, just that single Load Balance IP Address, which, as you can see in the screenshot, is .46.That is all I've done on the software and hardware side.
---To answer Questions---
@SteveITSEnable automatic outbound NAT for Reflection is checked
System/Advance/Firewall & NAT
Checked it.Hosting Servers with multiple IP Addresses.
In this case, as explained above, I am using a Load Balancer with a Single IP Address.
@NollipfSense
Within the Belkin Router, I had all ports forwarded to the IP Address of the Load Balancer .46Will have to check in on the HAProxy.
@viragomann
All information for what has been done is listed above.
All screenshots of the Rules are in the previous post. -
@viragomann
(Just out of curiosity, why you need incoming DNS access on WAN?)We run our own DNS Servers for the Web sites.
-
Just did a check on https://www.yougetsignal.com/tools/open-ports/
And it shows that Port 80, and 443 are both closed.
DNS, SMTP, and POP3 are all open.So, is this a PFSense issue?
I have the records set up, but it seems that PFSense is not allowing traffic in on the two ports. -
@carrzkiss It's showing passed traffic, though:
The port 80 rule is set to log, it is logging those packets? In this situation it's usually the web server firewall blocking the connection, or (rarely) an incorrect gateway on the web server.
-
@steveits
All Web servers have their gateway set to
192.168.4.1Also.
The DNS Server.
The way it is set up is like so.
The domain.com
www - Host(A) - IP Address (Outside IP Address)This is working, as when I set up LetsEncrypt, I did it with DNS Text (txt) Entries. And it worked without fault.
What else would I need to check?
-
looking at the log files.
Firewall --
Passed Mar 2 16:11:19 - HTTP web server interface - source (China IP Address) -- Destination (192.168.2.46) TCP:SI checked on https://www.isitdownrightnow.com/, and it shows all sites are down using HTTP or HTTPS.
Does this mean it is going through PFSense, but being stopped on the ServerSide?
If so, what do I need to check on the Servers? -
Check the state table in Diag > States when you are trying to connect externally.
You should be able to see the open states on WAN and LAN with the WAN states NAT'd.
-
@stephenw10
When I tried to access one of the sites from outside, I refreshed the States page, and there were no new entries. -
Hmm, I would check again. Try filtering by6 the client IP you're testing from. The firewall logs indicate it's passing traffic and NATing it and that will create a state.
The states may get closed almost immediately though if the load-balancer (which I assume is at 192.168.2.46) is refusing them.A pcap on LAN would confirm that.
-
@carrzkiss said in Webserver SSL is not private (How to set):
but being stopped on the ServerSide
Probably.
That said I just spent an absurd amount of time setting up a VLAN on a working-network switch replacement...devices could resolve DNS using pfSense [22.01, need to upgrade] and states were opening out the WAN but no traffic flowed from the VLAN until we restarted pfSense. ٩(͡๏̯͡๏)۶ Haven't seen that before. -
@steveits
I just checked on a video for HAproxy
It seems to me that the HAproxy is used for multiple Servers.
In my case, I do have multiple servers, but only ONE IP Address is used for all Gateways into the Actual Web Servers themselves.
UNLESS the HAproxy is used for Multiple Web Sites???
If that is the case, I will look into setting it up and testing it later this evening.
The SSL Cert is a Wildcard Cert, the same type I've used for 4 years now through LetsEncrypt.
I have 8 Domains, and they are all on the same Cert with my primary web domain as the holder.There are 4 ARR Servers on the load balance using .46
If ARR1 is used, ARR2 is used, and so on.The Port Forwarding would seem to be the most likely one for my setup, but I will let you all tell me otherwise.
-
@carrzkiss Port forward seems like what you want. People do occasionally post here asking to forward port 443 to one server for example.com and another for example.net which doesn't work without something proxying that by hostname.
If you forward directly to one web server IP instead of the load balancer does it work? If so, then it would be a load balancer issue.
Check a packet capture and the states as noted above.
-
If the load-balancer is at .46 and is proxying or forwarding traffic from there to other internal servers it should work. HAProxy would replace any existing load-balancer.
You might be seeing some asymmetric routing depending on how the load-balancer is handling that traffic. I would expect to see some blocked traffic on LAN in the firewall log if that's the case though.
You might also be seeing this: https://docs.netgate.com/pfsense/en/latest/install/upgrade-before-2.2.html#microsoft-load-balancing-open-mesh-traffic
If the load-balancer is using a multicast MAC address you'd need to set that tunable.Steve
-
@steveits
I just checked something.
On one of the ARR Servers, I did a ping.
Google - works
Microsoft - Request Timed Out.So, it seems the servers are not getting into the world.
And if they are not getting out, and nothing is coming in, that would explain why there is no connection.
I also found in System / General
DNS Server Settings.
I am unsure if I did this right or not.
I added in the DNS Server I run (WAN IP Address)
Along with the 3-pointer DNS Servers that point records for us as well.
But none of that seemed to change anything. -
@stephenw10
I am getting someplace now. Thank you.
Adding the
Tunable: net.link.ether.inet.allow_multicast
Value: 1
It was what was needed.I was able to pull the site up.
The SSL is giving an error again. (the connection if not private)
And I have to figure out how to get the SQL Server to work, but at least I am getting something. -
Adding DNS servers there only does anything for queries from pfSense itself unless the resolver is in forwarding mode.
Are those web servers using their own IPs to send pings or is everything going via the load-balancer IP?
Definitely check that multicast MAC issue if it's the latter.Steve
