haproxy - not working
-
Hello, I have just switched from UDM to pfSense. And am now trying to make my vaultwarden-server accessible externally. I thought I could do it with HAproxy, but unfortunately not. Here is some data - please tell me whatever you need to help me...
network
Fritzbox (BridgeMode - 10.1.50.1) I 10.1.50.111 pfSense WAN I pfSense (192.168.1.0/24 LAN - actually no VLAN) I I UnifiSwitch I I Proxmox Server (192.168.1.220:8006) I I Vaultwarden LXC (192.168.1.231:8000)dig (domain edit here)
; <<>> DiG 9.18.19 <<>> xxxxx.home64.de ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7827 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1432 ;; QUESTION SECTION: ;xxxxx.home64.de. IN A ;; ANSWER SECTION: xxxxx.home64.de. 60 IN A 78.24.215.1 ;; Query time: 143 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) (UDP) ;; WHEN: Wed Dec 13 07:31:53 CET 2023 ;; MSG SIZE rcvd: 60HAproxy configuation
# Automaticaly generated, dont edit manually. # Generated on: 2023-12-13 07:31 global maxconn 500 stats socket /tmp/haproxy.socket level admin expose-fd listeners uid 80 gid 80 nbthread 1 hard-stop-after 15m chroot /tmp/haproxy_chroot daemon server-state-file /tmp/haproxy_server_state listen HAProxyLocalStats bind 127.0.0.1:2200 name localstats mode http stats enable stats admin if TRUE stats show-legends stats uri /haproxy/haproxy_stats.php?haproxystats=1 timeout client 5000 timeout connect 5000 timeout server 5000 frontend HTTP-to-HTTPS bind 10.1.50.111:80 name 10.1.50.111:80 bind 192.168.1.1:80 name 192.168.1.1:80 mode http log global option http-keep-alive timeout client 30000 http-request redirect scheme https frontend LocalFrontend bind 192.168.1.1:443 name 192.168.1.1:443 ssl crt-list /var/etc/haproxy/LocalFrontend.crt_list mode http log global option http-keep-alive option forwardfor acl https ssl_fc http-request set-header X-Forwarded-Proto http if !https http-request set-header X-Forwarded-Proto https if https timeout client 30000 http-response set-header X-Xss-Protection 1;\ mode=block http-response set-header X-Robots-Tag "noindex, nofollow" http-response set-header X-Frame-Options SAMEORIGIN http-response set-header Referrer-Policy same-origin acl aclcrt_LocalFrontend var(txn.txnhost) -m reg -i ^xxxxx\.home64\.de(:([0-9]){1,5})?$ acl aclcrt_LocalFrontend var(txn.txnhost) -m reg -i ^([^\.]*)\.yyyyy\.home64\.de(:([0-9]){1,5})?$ acl aclcrt_LocalFrontend var(txn.txnhost) -m reg -i ^([^\.]*)\.xxxxx\.home64\.de(:([0-9]){1,5})?$ acl aclcrt_LocalFrontend var(txn.txnhost) -m reg -i ^yyyyy\.home64\.de(:([0-9]){1,5})?$ http-request set-var(txn.txnhost) hdr(host) frontend vaultwarden-xxxxx-home64de bind 10.1.50.111:443 name 10.1.50.111:443 ssl crt-list /var/etc/haproxy/vaultwarden-xxxxx-home64de.crt_list crt-ignore-err all mode http log global option http-keep-alive option forwardfor acl https ssl_fc http-request set-header X-Forwarded-Proto http if !https http-request set-header X-Forwarded-Proto https if https timeout client 30000 acl xxxxx.home64.de var(txn.txnhost) -m str -i xxxxx.home64.de acl aclcrt_vaultwarden-xxxxx-home64de var(txn.txnhost) -m reg -i ^xxxxx\.home64\.de(:([0-9]){1,5})?$ acl aclcrt_vaultwarden-xxxxx-home64de var(txn.txnhost) -m reg -i ^([^\.]*)\.yyyyy\.home64\.de(:([0-9]){1,5})?$ acl aclcrt_vaultwarden-xxxxx-home64de var(txn.txnhost) -m reg -i ^([^\.]*)\.xxxxx\.home64\.de(:([0-9]){1,5})?$ acl aclcrt_vaultwarden-xxxxx-home64de var(txn.txnhost) -m reg -i ^yyyyy\.home64\.de(:([0-9]){1,5})?$ http-request set-var(txn.txnhost) hdr(host) use_backend vaultwarden-xxxxx_ipvANY if xxxxx.home64.de aclcrt_vaultwarden-xxxxx-home64de use_backend vaultwarden-xxxxx_ipvANY if aclcrt_vaultwarden-xxxxx-home64de backend vaultwarden-xxxxx_ipvANY mode http id 100 log global timeout connect 30000 timeout server 30000 retries 3 load-server-state-from-file global server xxxxx.home64.de 192.168.1.231:8000 id 101 ssl verify none -
Not recommended opening a Password Manager to the public but it’s your funeral.
Haproxy requires a VIP. You then configure DNS Resolver to point the host.fqdn.com to the VIP. Backend points to the IP of the Server. Front end has Port 80 /443 which consists of ACLs. ACL name of backend, ACL Frontend which is the name of the domain name and SSL. SSL can be configured with Acme Package.
Port 80/433 needs to be listening on the IP of the VIP.
Allowing WAN doesn’t require any Port Forwards for Port 80/443 just requires an allow rule on WAN.
Now this all depends on how you have DNS configured, are you using Split Horizon or are you hosting DNS on Premise ?
Note for Publicly exposed services you should be using VLANs not a flat network which you currently have.
-
@VioletDragon
Thank you very much for the feedback. I have a rough understanding of it. However, the pfSense is so new to me that I don't
don't know exactly where I have to perform which points.
Is there a good wiki or video or something similar.
I'm pretty much at a loss and tried all day yesterday. But I never came across the VIP, for example :(You're actually right about the password manager. But how do you do it then ... with the update only in the internal LAN... but Vaultwarden needs a HTTPS or ???
DNS I currently still have the public
1.1.1.1and8.8.4.4entered. I'm not sure what I should use for the local DNSVLANs is the next step I need to take. Unfortunately, my UDM broke down in one fell swoop, so I'm trying to make sure that at least everything is up and running again. I currently only have one VLAN for the guest wifi. Should I put the Proxmox server with all VM/LXC in an extra VLAN or what would make sense here?
What exactly would you need to make my pfSense suitable for everyday use? -
@TMG you can internally wrap SSL without Publicly exposing services, all you would need is a A record that points to WAN IP. A Record example.com -> WAN IP.
In Acme package you could then create a SSL Certificate with .*example.com would then be a wildcard Certificate which would work with all subdomains for each internally services and public facing services too so an all Certificate for all. It would be all the same example you would just not allow a rule under WAN. I would create two VIPs one for Internal and another for External Public Facing services.
VIP (Virtual IP) can be created under Firewall -> Virtual IPs.
All this all depends on how DNS is configured and if you have a Static IP. If you don’t have a Static IP then your only option would be Dynamic DNS.
You could use a Self Signed SSL certificate but this requires you to install the Certificate on each device.
Another option you have, is you could wrap SSL internally and just connect using OpenVPN when outside of your network.
Regards
-
@VioletDragon
Hello, thanks again for the feedback.
I have not received a static external IP from my provider. So far I have solved this in connection with the UDM via nginx-proxymanger and used IPv64.net DynDNS addresses
worked. In npm I have the certificate and forwarded from the DynDNS to the internal server IP.
Exactly, that would also be great with pfSenese.What IP address must/can I assign in the VIP?
I have currently only entered the public ones from cloudflare and google as DNS servers. I actually plan to have my own DNS server. The question is where to put it: on the pfsense or on the proxmox server behind the pfsense?I have created certifacte with ACME. Have I already created some for testing? Do I have to use the same one everywhere?
Wireguard is planned for later, for access to the homelab.
-
# Automaticaly generated, dont edit manually. # Generated on: 2023-12-14 20:12 global maxconn 100031 stats socket /tmp/haproxy.socket level admin expose-fd listeners uid 80 gid 80 nbthread 1 hard-stop-after 15m chroot /tmp/haproxy_chroot daemon server-state-file /tmp/haproxy_server_state listen HAProxyLocalStats bind 127.0.0.1:2200 name localstats mode http stats enable stats refresh 10 stats admin if TRUE stats show-legends stats uri /haproxy/haproxy_stats.php?haproxystats=1 timeout client 5000 timeout connect 5000 timeout server 5000 frontend xxxxxx-Server bind 192.168.1.1:443 name 192.168.1.1:443 ssl crt-list /var/etc/haproxy/xxxxxx-Server.crt_list mode http log global option socket-stats option http-keep-alive option forwardfor acl https ssl_fc http-request set-header X-Forwarded-Proto http if !https http-request set-header X-Forwarded-Proto https if https timeout client 30000 acl vaultwarden var(txn.txnhost) -m beg -i xxxxxx.home64.de acl unifi var(txn.txnhost) -m beg -i xxxxxx.duckdns.org acl aclcrt_xxxxxx-Server var(txn.txnhost) -m reg -i ^xxxxxx\.home64\.de(:([0-9]){1,5})?$ http-request set-var(txn.txnhost) hdr(host) use_backend vaultwarden_ipvANY if vaultwarden aclcrt_xxxxxx-Server use_backend unifi_ipvANY if unifi aclcrt_xxxxxx-Server frontend HAProxy_stats_ssl_frontend bind 192.168.1.1:444 name 192.168.1.1:444 ssl crt-list /var/etc/haproxy/HAProxy_stats_ssl_frontend.crt_list mode http log global timeout client 30000 default_backend HAProxy_stats_ssl_backend_ipvANY backend vaultwarden_ipvANY mode http id 100 log global cookie nocache stats enable stats uri / stats realm . timeout connect 30000 timeout server 30000 retries 3 load-server-state-from-file global server vaultwarden 192.168.1.231:8000 id 101 ssl verify none crt /var/etc/haproxy/server_clientcert_657b0f05e4047.pem backend unifi_ipvANY mode http id 102 log global stats enable stats uri / stats realm . timeout connect 30000 timeout server 30000 retries 3 load-server-state-from-file global server unifi 192.168.1.221:8443 id 101 ssl verify none crt /var/etc/haproxy/server_clientcert_657b0f05e4047.pem backend HAProxy_stats_ssl_backend_ipvANY mode http id 105 log global stats enable stats uri / stats realm . stats refresh 10 timeout connect 30000 timeout server 30000 retries 3 load-server-state-from-file globaltomuser@MacBook-Pro ~ % dig @8.8.8.8 xxxxxx.home64.de ; <<>> DiG 9.10.6 <<>> @8.8.8.8 xxxxxx.home64.de ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 532 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;xxxxxx.home64.de. IN A ;; ANSWER SECTION: xxxxxx.home64.de. 60 IN A 77.23.205.1 ;; Query time: 52 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Thu Dec 14 20:56:55 CET 2023 ;; MSG SIZE rcvd: 60 -
@TMG said in haproxy - not working:
UDM
UDM is a Dream Machine by Ubiquti. I would suggest having pfSense handling DNS as it will add more complexity to the solution.
DynDNS is Dynamic DNS.
VIP can be any IP as it's Virtual,.
For SSL in Haproxy, the Certificate is mapped by the Frontend.
-
Nothing needs to go in here?
I still don't understand the connection between the VIP and the DynDns domain xxxxx.home64.de?
How exactly this plays together.... I have been looking all day
for a video or a configuration guide for my configuration.Do you mean these here, no??
How and where exactly does the VIP come into play?
-
-
@TMG Create a A Record that points to WAN.
Under DNS Resolver -> Host Overrides. Create a Host & Domain Name, Add the IP of the VIP.
Frontend of Haproxy needs two Entries One for Port 80 and another Port 443. Both needs the IP of VIP you created which comes under Listen address for both.
For the SSL Certificate you specify it under Certificate. Make sure that the type is configured as http/https offloading.
-
-
@TMG No. You need two frontends, one for Port 80 & Port 443.
Listen address needs to be set to the VIP.
-
@VioletDragon
Good morning. I can only say thank you again and again for your efforts.
Must here in the dnsresolver/host_overrides also the created VIP purely
-
@TMG Good morning, yes the IP is the VIP.
Depending on your OS, you can test by using Dig in Linux / macOS or nslookup in Windows.
Regards
-
@VioletDragon
It's never happened to me before that I can't solve a problem for days. I can't get it to work?
dig always points to the public IP address.
Can you do me a big favor and take a look at the
pdf with the screenshots to see if you notice anything where I'm
am wrong ... or if something important is missing.
... thanks thanks thanks
I just realized I can't attach a pdf? -
Here is a dropbox link
pfsense_screenshotsIf you don't want that, please let me know. Maybe we can find another way.
-
@TMG Attach Screenshots of DNS Resolver. Screenshots in pdf are small, Also attach screenshots of System -> General Setup & System -> Advanced -> Admin Access.
Regards
-
@VioletDragon
I hope it´s bigger and you can read it
