Call of webserver and/or nextcloud server blocked in LAN/WLAN
-
Hi! Having pfSense 2.5.0 with WAN, LAN1, and LAN2, and a HAProxy package that enabled me to setup at least two Apache-Applications: (1) a webserver and (2) a nextcloud server into LAN1 (in subnet i.e. 192.168.x.0/24). Just like out of the box, (FQDN-)calls of webserver such as https://www.example1.com and/or nextcloud server such as https://cloud.example2.com are working super! Yes, super, from anywhere in the world of the internet. But no, rather bad, from a client that calls (1) and/or (2) within the same subnet (i.e. 192.168.x.0/24) of LAN1 (and ev. also LAN2).
What I stated is that (a) a call/request from the outside world is directed to correct IP (ISP DHCP lease for the WAN interface), and (b) a call/request from the inside of the same LAN1 subnet (i.e. 192.168.x.0/24) is directed to the external IP that is provided by the ISP, but not forwarded to the internal apache server. The browser says: "Website is not reachable. https://www.example1.com is not reachable. ERR_ADDRESS_UNREACHABLE
Who or what can help me to come out of this angrily dilemma?
PS: Since I could find a workaround with the "hosts" on Windows and/or Unix clients, the workaround failed for example with the Android App "Hosts Go". But I think, this should be solved by pfSense's possibilities!
-
Are you forcing clients out a gateway? Say a vpn?
I host a couple of services via haproxy - and have no issues access said fqdn from client on the lan.. That resolves to my public wan IP..
What rules do you have on lan interface? Are you only allowing dns to say lan IP, and blocking to this firewall or your wan?
-
@johnpoz
As I said: Out of the box. I did not change anything as far s I know.
-
Looks like your doing some sort of shaping/queue there.. So not out of the box ;) heheh
Also are these services on 80? You have your webgui using 80.. you show using https.. so just the standard 443 port for these services.
I would disable the use of queue - and see if that fixes your issue.. For your haproxy setup - are you doing https offload - or you just passing it through? I do ssl offload..
Your clients pointing to pfsense for dns - what do they resolve for your fqdn.. Its your current public wan IP?
-
@johnpoz Thanks for the reply. What do you mean with "some sort of shaping/queue there"? How can I stop it? BTW: There is only one port open: 443 (since 1194 is closed at the moment). I'm not doing https offloads.
Yes, as I said in the intro, LAN1 clients are pointing to my current public wan IP, since clients from anywhere are directed to the correct IP (such as ISP's DHCP lease for the WAN interface). I'm still very deep in the tunnel, that is, I can see some light which is not that much for my dilemma. Sorry, to say that.
-
Your using this queue
But I would suggest just sniff and see what is happening.. So for example when I access my stuff doing ssl offload..
While traffic to my client is from my pc rfc1918 IP, to my public wan IP. the traffic from the haproxy towards the server serving is from pfsense lan IP, etc. its a true reverse proxy. In the mode your doing it.. I am not sure how that is done.
We prob have better luck moving this to the proxy section - where someone more versed in proxy operation might be able to chime in..
I'm going to move this thread there.
-
@johnpoz Hello, thanks a lot for your efforts. But please remember that I said in the intro that (a) all requests from the outside world such as https://www.example1.com and/or https://cloud.example2.com are working fine! And (b) for all requests from the inside of LAN1 (192.168.x.0/24) it produces, for example: "Website is not reachable. https://www.example1.com is not reachable. ERR_ADDRESS_UNREACHABLE". For the last (b), I have a "poor" workaround via hosts file (Win 10: in C:\Windows\System32\drivers\etc and Linux: via "sudo nano /etc/hosts"). In the appropriate hosts file I placed the following lines:
192.168.x.y www.example1.com # => results OK via Laptops' or PC's Browser
192.168.x.z cloud.example2.com # => results OK via Nextcloud AppThe workaround for (b) does not work if the client is a smartphone! So, from my point of view, I would not like to integrate a hosts file into every smartphone, I would suggest that pfSense should provide a solution for this kind of inside calls. Or, is this a system immanent bug? (BTW: I had this phenomenon occurred also in a non-pfSense such as Apache Multisite Virtual Hosting environment. There, I had to install DNSmasq !!!)
-
host overrides are never going to work for some device that doesn't use pfsense for dns.. You could try dns redirection to get them to use your dns.. But could be using doh for all we know..
Please stop repeating the same thing over and over again..
What are you wanting to do - have haproxy send the traffic or use nat reflection? Work on 1 thing or the other. I do not use nat reflection, haproxy sends the traffic. clients resolve the fqdn to public IP..
-
@johnpoz Sorry to repeat some issues (old school in Latin: repetitio est mater studiorum).
Just tried host overrides. Smartphones connected now. But Nextcloud "closed" login all the way, cannot connect to Nextcloud anymore, even I deleted host overrides. Have to re-stage Nextcloud server.
I do want that haproxy sends the traffic, but clients (smartphones) still not resolving the fqdn to connect (without host overrides).
What do you mean by: "You could try dns redirection to get them to use your dns.. But could be using doh for all we know"? What is "doh"? Could you give me some more details how to use dns redirection (links)?
-
What is your phone using for dns.. If not resolving the public fqdn your using?
doh - dns over http, you been sleeping in a cave the last couple of years? You hear about the global pandemic? ;)
doh and dot (dns over tls) are the latest craze to get you to send your dns to the big players, while telling you its more secure.. Because that big bad isp of yours won't see your dns queries.. Oh my gawd - they know you looked up amazon.com ;) Even though they still know you went to ip of amazon, and hey your https connection sent and sni that told them you going to amazon.. But oh my goodness - lets hide the dns query from them.. Anyhoo - browsers like to turn it on by default.. Phones for sure do, etc..
So if your phone is doing that it wouldn't be using your local pfsense dns to even see your host overrides. Also phones like to not use your local dns - android big on this.. you know they know better and even though you tell them via dhcp to use pfsense IP for dns, they like to use 8.8.8.8 anyway. If that is the case and not doing doh, you can just redirect the dns query going to 8.8.8.8 to pfsense.
https://docs.netgate.com/pfsense/en/latest/recipes/dns-redirect.html
One way or the other you really need to pick your poison here.. Do you want haproxy to send the traffic.. So your clients use the public IP to try and access. If your doing that you do not use nat reflection.. Nat reflection is for port forwards, not reverse proxies..
Either or - if your using host overrides - devices on your local network using your local dns, would never hit your wan/public IP to either be reflected or proxied.
So your phone is on your wifi - right? And this is not behind some nat router doing your wifi? its on one of your lan1 or lan2 networks? Also why are you hiding rfc1918 addresses? Nobody gives 2 shits if your using 192.168.1 or 192.168.23.. They are all private.. They don't tell anyone where your at, Sure and the hell can not get to your network via that address.. I use 192.168.9/24 on my lan, and my current pc is 192.168.9.100.. Does that tell anything that you could use to do anything to me, or find out where I am, or anything?
I use 192.168.9/24, and 192.168.3/24 for my dmz network - hey I have ntp server open to the public on 192.168.3.32.. There is zero reason to hide or obfuscate rfc1918 space.. My nas is at 192.168.9.10, and I also using 192.168.2 and .4 and .5 and .6 and .7 for other vlans.. And I also have a 192.168.10 network I use as san between my pc and nas that uses 2.5gbps interfaces.. But since I do not have a 2.5gbps switch I have that setup as a san.. Does any of that info really give away anything? Its rfc1918 - everyone on the planet is using it.. It doesn't route over the public internet.
Is your wan of pfsense actually public, ie not a rfc1918 IP? 10/8, 192.168/16, 172.16/12 - pick your poison.. If your using haproxy there is little need for host overrides pointing public fqdn to your rfc1918 IP..
