HA Proxy / Acme Lets encrypt : LAN access problem from inside with external url https

[ducati57]

Hello everyone,

I have come to ask for your help to resolve my problem, you will find below the infra with the different endings.

Info:
OVH domain name
Redirection to public fixed IP of the Bbox Bytel

Pfsense:
Pfsense VM via my ESXI with a 2x SFP slots PCi card. (1 port for WAN vmk1 / 1 port for LAN vmk2)
DNS Bytel and Google
HAProxy 443 (https://dom.nomdedomaine.ovh / https://vms.nomdedomaine.ovh / https://nvr.nomdedomaine.ovh)
ACME Lets encrypt

Miscellaneous :
ESXI LAN management interface vmk0 (Ethernet on server motherboard)
PFsense VM with WAN vmk1 and LAN vmk2
The other home automation VMs, nas, nvr, etc... itself on the vmk2 LAN interface

Problem 1: Access to my local infrastructure, when I am connected to my LAN network via wifi or rj45 I cannot reach the machines via the external url https://____.domainname. ovh
However in 4G therefore in WAN no problem....

After several tests, I noticed that if I temporarily deactivate the option: Block private networks and loopback addresses present in the WAN interface it works!
This will allow traffic from IP addresses reserved for private networks per RFC 1918 (10/8, 172.16/12, 192.168/16). This option should generally be enabled unless this network interface resides in a private address space. (WAN interface behind a Router / Box)

---

Problem 2: My VM NVR (vms) broadcasting on port 37777 for video streaming. This is not a web interface, but a manufacturer application to connect to the VMS.
I want to use a URL type https://nvr.domainname.ovh to integrate it into my application. I set up the necessary in the backend frontend, however no connection possible.
I suspect a problem because of SSL and port 443 which should not pass through the mobile application (Dahua)

Do you have an idea for moving forward on these topics? :)

Thank you in advance for your ideas!

[johnpoz]

@ducati57 said in HA Proxy / Acme Lets encrypt : LAN access problem from inside with external url https:

This option should generally be enabled unless this network interface resides in a private address space. (WAN interface behind a Router / Box)

True - but you are with this setup actually doing that.. When you hit your nat reflection at your actual wan.. Its sending the traffic to your pfsense wan.. What is the source IP then - it would be your rfc1918 address, so yeah pfsense with the block rfc1918 rule would block it.

[ducati57]

The devices in my LAN have a typical IP: 192.168.1.xx

So it is better to leave Block private networks and loopback addresses enabled ? Or dangerous in security to disable it?

[johnpoz]

@ducati57 blocking rfc1918 at the wan has debatable security value.

It prevents clients with rfc1918 from accessing services you have opened on the wan in the first place. Normally the only devices that would be able to talk to your wan services via rfc1918, are either local to your own networks anyway. Or on the same ISP as you..

If you open up service X.. what should it matter that the IP that talks to it is public or rfc1918.

IMHO these rules to block rfc1918 and bogon a of little value normally. But they are common to do - because rfc1918 and bogon should never route over the public internet in the first place. But if your wanting to allow this sort of traffic, then disable them..

In your current configuration - yeah sure looks like the source of the traffic your edge device is reflecting to the pfsense wan would be rfc1918.. If you want to allow that reflection, then you would need to disable the block rfc1918 rule on your wan.

[ducati57]

@johnpoz
Okay thanks for the information.

Is there another option to solve my problem without disabling this "Block private networks and loopback addresses" security?

I'm French, so I use google translate... :)

[johnpoz]

@ducati57 well you could set your local dns to resolve these public fqdn to your pfsense wan IP, vs public IP that is being reflected by your edge device.

That way vs going to your edge device to be reflected back in, they would hit your pfsense wan where haproxy would direct them where to go.. I do the same sort of haproxy ssl offload sort of setup.. But in my case my public IP is on my pfsense wan. So when my clients behind pfsense want to go there, they just hit the pfsense wan IP coming in from the lan side which is allowed.

You could setup a simple override or multiple if you have multiple different fqdns so your local devices using pfsense as their dns would resolve them to your pfsense wan IP..

[ducati57]

@johnpoz

Here are the DNS currently used by Pfsense, the whole thing is attached to the WAN interface 10.1.1.1.

Provider: primary 194.158.122.10 Secondary 194.158.122.15
Google : primary 8.8.8.8 Secondary 8.8.4.4

[johnpoz]

@ducati57 don't matter what pfsense is using - matters that your clients are pointing to pfsense for theirs.

also - you could set those all you want, pfsense out of the box resolves. So unless you went in and enabled forwarding mode, unbound on pfsense just resolves.

[ducati57]

@johnpoz

Is there a tutorial for making the modification ? Because if I understand correctly, it is necessary to make the modification on each client or on Pfsense?

I am not an "expert" in the IT / Firewall part, I work in the telecom (backbone / longhaul) fiber optic sector :)

[johnpoz]

@ducati57 so out of the box if you enable dhcp on pfsense it would hand out its own IP to any dhcp clients. Unless you don't have unbound or dnsmasq running on pfsense. Then it would hand out the dns you have set in general.

To be honest, this is out of the box for most users - nothing really to do.

You would just need to create the host overrides in unbound (resolver).. example here is somee host overrides I have setup

So when my clients ask pfsense for dns, it returns my local IP.. Vs the public IP you would get if you asked public dns

$ dig @8.8.8.8 time.nist.com +short
208.91.197.27

$ dig time.nist.com +short
192.168.3.32

$ dig time-ios.apple.com +short
192.168.3.32

$ dig @8.8.8.8 time-ios.apple.com +short
time.g.aaplimg.com.
17.253.26.123
17.253.6.125
17.253.26.251

So now clients wanting to look up those fqdn, instead of going to the public IP (your edge).. They would just go to your pfsense wan IP and haproxy would proxy them to where you have setup.. In my case they just ask my local ntp server vs the ntp servers on the public internet.

[ducati57]

@johnpoz

Here are the basic DNS on Pfsense.

I will make the modifications as you detailed previously and post a screenshot.

[johnpoz]

@ducati57 yeah - you already stated that.. Not sure what else you want me to say? Out of the box unbound doesn't even use those.. Unless you specifically setup unbound to forward.

And it doesn't matter what pfsense does be it resolve for forward.. If your client, ie some pc on your network is asking pfsense for dns.. And you setup a host override then that is the new iP that would be handed to the client for the fqdn it asked for.

If your pc is using say 8.8.8.8 or any of those you listed for dns directly - then no a host override wouldn't work, because the client is never asking pfsense to resolve anything anyway.

[ducati57]

@johnpoz

So here is the modification to be made, directly on the LAN/DHCP server?

[johnpoz]

@ducati57 You don't need to put anything in there..

Out of the box pfsense will ask itself (unbound) for dns - which resolves..

You only need to put those in - if you want pfsense to use them, or you want to forward to them from unbound, etc. I have zero use for any external dns provider since I just let unbound resolve, as it does by default anyway..

[ducati57]

@johnpoz Ok, thanks for your patience :)

I just modified it as indicated in your comment.=>

[johnpoz]

@ducati57 If that is pfsense IP, why would you set that and not just use loopback? Which is does all on its own..

[ducati57]

@johnpoz

If I do not indicate anything, and I configure Pfsense identical to your screenshot, here is the result
(KO update status because no external access because no DNS?)

[johnpoz]

@ducati57 did you modify unbound settings?

By default its ALL, did you change it and not include localhost?

Is unbound running even?

[ducati57]

@johnpoz

Here is the configuration of the different elements / services do you see an error? (security problem, loop, bug, useless,etc..)

[johnpoz]

@ducati57 well I see one thing that I personally would not do.. If your going to set the specific interfaces that unbound listens on, why would you click on wan.. Do you have devices that would be using the dns via its wan IP?

Your saying with those settings it does not populate 127.0.0.1 in the system tab for dns? If you do not put anything in the dns tab?

Here I just fired up my 2.7 box.. Its pretty much default out of the box.. Other than changing its lan to other than the default 192.168.1.1

I then changed from all, to just lan and got this error

So I selected localhost along with just lan and it worked and system still shows dns as loopback 127.0.0.1