Setting reversed proxy
-
Hello,
I have a box running pfSense with OpenVPN ( port 443 ), pfBlockNG ( DNS blocking) and ACME
Have 3 dyndns host names setuppfsense1337.dyndns.org ( used for OpenVPN )
web1.dyndns.org
web2.dyndns.orgHave now setup 2 web servers on LAN that I want to access with both https(443) and http( port 80 for auto renew CA from Let's Encrypt).
Both have there own CA using Let's Encrypt.I have tried to read the wiki for HAProxy but couldn't grasp how do set that pkg for my.
Anyone have a guide or something that explains the logic how it works?
Second should the CA for each web server should the be on the pfSense boxa as well ?TIA
Daniel -
@varazir
Something like this looks like it would work: https://314es.pl/https-openvpn-and-ssh-on-one-port-thanks-to-haproxyThe "frontend main" would be able to make further switching decisions for different web1 / web2 servers.. Might need to use of a few custom acl's in the webgui to make it the same.
-
@piba Hmm, something like that
I have tried to follow this guide but no luck
https://github.com/PiBa-NL/pfsense-haproxy-package-doc/wiki/pfsense_2_3_haproxy_sni_plus_offloading_backends#frontend-2
-
@varazir
Can you share the haproxy.cfg you configured? Might be able to see what part needs to be adjusted.. Please also explain what part does work and what doesnt if health-checks are passing (servers shown in green) in stats page and if not with what LastChk message. -
@piba Where is it saved in pfsense ?
-
@varazir
its visible at the bottom of the haproxy settings tab. -
@piba Ahh missed that
Here it is
https://gist.github.com/varazir/319b50bef3e4de1284ef52b38aaf5e23 -
@varazir
Okay, and what part does and what doesnt work with that config?First thoughts.:
-disable the 'ssl encryption' on all 3 servers (since you didnt decrypt the traffic on the frontend there is no need to encrypt it again before sending to the backend)
-enable 'ssl health-check' on the 2 webservers, to make it pass checks again
-change healthcheck for openvpn to be 'tcp check' i dont think it would show as 'up' otherwiseCheck that all 3 servers are shown as 'up' on the stats page
After that i think at least the 2 webservers should be reachable.. the openvpn might still not be..
Please share config again and tell what does/doesn't work.
-
disable the 'ssl encryption' on all 3 servers (since you didnt decrypt the traffic on the frontend there is no need to encrypt it again before sending to the backend)
I wan't it to be encrypted traffic.
-
@varazir
Yes the client is sending encrypted traffic, the frontend as you configured it does NOT decrypt the traffic. So you can forward that traffic as-is to the webserver and it will still be encrypted.It shouldn't be encrypted twice, as the webserver will not understand the request then after decrypting it once..
-
I got one of the web servers to work but not the other one, the one I redirect to port 446.
It reports that server is down
Edit: I got Openvpn servar as online changing to Basic Health check.
But I get Transport error when I try to connect with my OpenVPN client.
TCP recv EOF -
ok got both of the web servers to work I had to change the Health checking to basic and enable SSL checks
-
@varazir
Though it works its probably better to change health-check's for the webservers back to http, and see what the LastChk column in stats says. Might be a forbidden method, or a authentication required, or it needs a sni indication in the check request, or a host header, or something. That is the thing to change and or fix to make health-checks pass on a webserver.As for the openvpn connection, i expect it is not actually sending a proper ssl encapsulated request.. As such you might need a 'special' acl where you could put the "!{ req.ssl_hello_type 1 } !{ req.len 0 }" in the acl for that use_backend action that might make it sellect the right backend, and allow it to connect.
Can you share the config as you got it now?
-
I have updated my Git https://gist.github.com/varazir/319b50bef3e4de1284ef52b38aaf5e23
The web2 keeps getting "active or backup DOWN for maintenance (MAINT) "
When I update the setting it works fine for a min or 2.Not sure where to add "!{ req.ssl_hello_type 1 } !{ req.len 0 }"
I guess there
-
I removed the check on web2
-
@varazir
What did stats page LastChk say when it was down? (hover over it with your mouse) -
@varazir
Instead of thisuse_backend Octoprint_ipvANY if Octoprint
make that like:
use_backend Octoprint_ipvANY if !{ req.ssl_hello_type 1 } !{ req.len 0 }
-
That is from the config file trying to find how to set it in the GUI :P
-
@piba When it's working it say Layer7 check passed: OK
-
@varazir
You already wrote 'Octoprint' in that same textfield you can put the other acl text..As for above maint line it looks like the 'zwave' servername might not resolve DNS easily to a IP? That shouldnt depend on the type of check used anyhow.. Perhaps give it a few seconds more to resolve? And the check again? A L7 check that passes would be good..
-
@piba Well I have to look at the openVPN tomorrow, getting late.
Thanks for your help
-
Where should I place in the GUI this again ? Front or back end ?
use_backend Octoprint_ipvANY if !{ req.ssl_hello_type 1 } !{ req.len 0 }
-
@varazir
Where in the gui did you write 'Octoprint'?( For a Frontend - action ? )
--Edit--
Though i think i got your backends mixed up.
Should use the special acl for the 'openvpn' action acl name. -
Word of warning: Make extra sure that you do not expose OctoPrint to the Internet. It is not designed to do that. Keep it locked behind a VPN.
Personally, I run haproxy directly on my OctoPrint Pis (and acme.sh to do the certs). It only takes a very small/simple config. Much more secure that way.
-
@piba said in Setting reversed proxy:
@varazir
Where in the gui did you write 'Octoprint'?( For a Frontend - action ? )
Front end but it's OpenVPN that is the issue. It's on the same port on pfSense.
Octoprint back/frontend works fine
I changed to IP and now the health check works much better. -
@jimp said in Setting reversed proxy:
Word of warning: Make extra sure that you do not expose OctoPrint to the Internet. It is not designed to do that. Keep it locked behind a VPN.
Personally, I run haproxy directly on my OctoPrint Pis (and acme.sh to do the certs). It only takes a very small/simple config. Much more secure that way.
ya I read that.
Octopi comes with HAProxy, I have setup a extra authentication. Using acl / http-request auth not sure if it's enoufe or there are other things that is not designed.How can you open port 80/443 to several hosts that using certbot ?
-
If it's protected by at least some kind of auth it may be OK, but I still would prefer to keep it behind a VPN.
As for certbot, that would probably just get routed based on the requested hostname. Should work with plain http and some ACLs. I use acme.sh and DNS-01 challenges though, I do not want any inbound web requests hitting my Pis.
-
@varazir
Yes sorry i mixed up your backends/acls.. Where i wrote octoprint i ment openvpn at least for the last few posts..use_backend Openvpn_ipvANY if !{ req.ssl_hello_type 1 } !{ req.len 0 }
Anyhow still location to put that is the frontend/action acl-name.
-
@piba said in Setting reversed proxy:
@varazir
Yes sorry i mixed up your backends/acls.. Where i wrote octoprint i ment openvpn at least for the last few posts..use_backend Openvpn_ipvANY if !{ req.ssl_hello_type 1 } !{ req.len 0 }
Anyhow still location to put that is the frontend/action acl-name.
acl name :) thanks found the field thanks.
Now everything works.
-
@jimp said in Setting reversed proxy:
If it's protected by at least some kind of auth it may be OK, but I still would prefer to keep it behind a VPN.
As for certbot, that would probably just get routed based on the requested hostname. Should work with plain http and some ACLs. I use acme.sh and DNS-01 challenges though, I do not want any inbound web requests hitting my Pis.
I'm looking at the script I was using the command line that came with Let's Encrypt and this guide
https://www.digitalocean.com/community/tutorials/how-to-secure-haproxy-with-let-s-encrypt-on-ubuntu-14-04
This all started more or less with getting the Lets encrypt not needing to do a portforward each time.
-
If you are going to run all the haproxy bits on pfSense there is a script you can use to hook into the ACME package to handle all of that locally for any hostname.
https://forum.netgate.com/topic/90643/let-s-encypt-support/31
-
@jimp said in Setting reversed proxy:
If you are going to run all the haproxy bits on pfSense there is a script you can use to hook into the ACME package to handle all of that locally for any hostname.
https://forum.netgate.com/topic/90643/let-s-encypt-support/31
Hmm don't understand what it dose.
So I still run the rewnew/setup on the host on then LAN ?
-
The ACME package on pfSense would handle all of the certs, haproxy on pfSense would offload all SSL tasks from local devices. So for example you'd connect https to the firewall and it would hand off http to the octopi backend.
That may not be exactly what you're after but it would be an easy centralized solution.
-
@jimp Hmm sound much easier. Like the idea.
I guess I need to remake the backends on the current setup right ?I guess that will not work when I use https internal ?
-
@varazir
It is possible to keep the webservers the same as they are, HAProxy would need to be reconfigured though to decrypt traffic on a (second) frontend (but not for openvpn), and re-encrypt traffic send to the webserver. Or reconfigure the webservers to accept connections on :80 while 'understanding' that the actual connection from the client is really made over https (so they dont respond with a redirect to https or a absolute link to a http:// resource).. some web-applications are not easily configured to do so though.. -
-
@varazir
That 'message' is not a error, its just some 'unexpected' output as far as the haproxy package is considered.. You could remove that version output from the acme script if you like..If you want to use certificates on haproxy, then first step is to get the certificate to use into pfSense certificate manager, can use a combination of haproxy with acme package for that, as for the current configuration you need to 'offload' and configure a certificate on the frontend, and then check the ssl encyption checkbox on the backend servers. And configure the proper acl's based on host header instead of sni indication.
And because you also still want to pass along the openvpn traffic, the first frontend cannot do the ssl offloading, as openvpn does not use regular ssl. So need to make a second frontend and loopback the connection from a backend on the first one, to the second local frontend used for the https website connections.Basically back to post #2 : https://314es.pl/https-openvpn-and-ssh-on-one-port-thanks-to-haproxy
-
@piba Hmm okay. I'm trying to convert the page you linked to pfSense GUI options.
-
@jimp could you point me in the right direction how to setup so HAProxy on pfSense handels the certs ( not just getting them )