Passthrough for destinations with offloading frontend
-
I'm trying to link a backend on a frontend to which SSL connections are passed through. I don't want to, as HAProxy as it is works like magic but the backend breaks down if I put HAProxy in front of it.
The stubborn backend happens to be another proxy, Microsoft's Web Application Proxy to be exact; go figure why it has xenophobic issues.
When I try to reach a server from outside it just times out or presents some error with seriffed fonts, Microsoft's new web servers that handle this traffic have these errors that look a lot like Apache's or NGINX's instead of your typical IIS error pages. It makes it very hard to diagnose where it comes from.
When reenable the HTTPS HAProxy frontend which has the same routes as the MSWAP server, it proxies fine the websites.
When I disable the HTTPS HAProxy frontend and let traffic NAT to the MSWAP server, it proxies fine as well the websites.
But, when I enable both, first HAProxy taking the traffic (its firewall rule comes first) then pass it to the MSWAP server that's when it times out or comes back with a non-descriptive error.
I would prefer not to use Microsoft's proxy at all as it has more downsides than benefits but it's necessary for some Azure stuff.
With the details done--here's my question; how can I set it up so I can use both. As I'm typing this it occurred to me I could change the frontend from offloading to HTTPS TCP mode and create an additional where the traffic previously going the the offloading proxy is redirected and then I could set MSWAP server on the HTTPS frontend therefore it would think it's its birthday party and no one else's.
Would that work?? It seems plausible but I don't wan't to ruin my setup just yet as it took me forever to get it this close to perfection. <3 Or, there's this option to setup shared frontends but I couldn't make them work last time using different type of frontend on the same port.
There are several boxes for passthrough options throughout the interface as well but the official documentation isn't meant for pfSense and the one for pfSense is limited at best. :/
Any ideas, criticism or alternatives...anything at at all is welcome. :)
-
Well...
This is embarrassing. As it turns out I was did the config remotely from one Windows Server machine to another...I fired up a RHEL VM and changed the hosts file to force the traffic to go through the WAP server and now it was not working on the intranet either.
I finally logged into the server and I got a message Windows Update had installed the latest fixes and what not, fixes that broke the system; so I guess...breaks?
Anyway, a simple restart got it working. :)
I'm still curious if my idea would've worked, I'mma obsessively make a ton of backups and I'll try it out.
-
I think it would have worked, mostly, maybe not exactly as described but well..
-
https-tcp frontend can do SNI to select some traffic and send it to a TCP backend. Likely the MSWAP would have been happy with that..
Use a second frontend on localhost to pass the other traffic to and do offloading on that one as you currently already were doing. -
Other option might be to re-encrypt the traffic from haproxy-backend-server to the MSWAP? But then it wouldn't receive a client-certificate if that is needed.. Other than that i think it 'should' work as well.. Could 'fake' the SNI send to the wasp with that what the client originally send if needed.. But client-cert, that cant be passed when offloading..
-
-
I didn't know about the client->server cert flow, thanks!! I will definitely do it because I do use certificates for identification.
HAProxy in combination with pfSense and the ACME package unbelievable power though, I refuse to replace it by MSWAP. For instance, just to cover securing my domains I'd have to get 8 wildcard certs, just for starters, then it comes the fact that wildcards aren't that liked so next I'd have to hunt down the problematic sites which I do not remember anymore and issue more certificates. Then there's the load balancing ability that's just so freaking fast. I love it
I have this piece of software to manage Linux servers akin to Webmin or cPanel, but only over SSH and it doesn't make major modifications on systems, it's called goPanel 2, as the others it can request LE certificates but using the connection method instead of the DNS methods--it failed over MSWAP, succeeded over HAProxy...it was when the connection attempts were spotty though.
-
It worked !
Thanks one more time, you advice was insanely helpful. Made it work, and though I was about to give up I slept it up and once refreshed it sort of everything clicked it became so easy and logical, it's just all the looping around server to server that makes it seem more complicated than it is, huh?!
I also went a little further set it up to proxy internally as well to simplify web addresses. And the best yet is that I finally came up with a solution to both use the naked domain for Active Directory and to host a website in it without hosting it on the DCs.
I'm leaving some screenshots, hopefully I can help somebody else too. :)
BACKENDS
LOOPAROUND BACKEND
FRONTENDS
-
-
@zwck
There are always options to do the same things differently. But there should not be a 'need' to do so unless you like that way better. Its not like the package changed significantly. -
i'd really would love to see an updated tutorial from you, since you are by far the most advanced user of the package :D
-
@zwck
What would you like to see updated? Is there something specifically that is missing that should be explained to basic users to get started?. If some pages are 'outdated' i can give it a go at creating a few new screenshots.. but really almost nothing has changed with regard to the basic frontend+acl&action>backend configuration stuff.. from whats already explained on my wiki pages.. -
@Piba, well i am not sure, tbh as i am not to experienced with haproxy it self. I played around with map files, on a standalone system that seemed to work alright too. Not sure if that has an advantage in regards to sni vs offload.
I used haproxy mainly to serve a couple of webpages with subdomains, and i created a virtualip for haproxy 2.1.1.2, maybe you can check if i didnt build in some obvious mistakes or forgot to tick somethings on.
# Automaticaly generated, dont edit manually. # Generated on: 2018-09-25 10:17 global maxconn 10000 stats socket /tmp/haproxy.socket level admin expose-fd listeners gid 80 nbproc 1 nbthread 1 hard-stop-after 15m chroot /tmp/haproxy_chroot daemon tune.ssl.default-dh-param 2048 server-state-file /tmp/haproxy_server_state ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256 ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ssl-default-server-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256 ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets tune.ssl.maxrecord 1370 listen HAProxyLocalStats bind 127.0.0.1:2200 name localstats mode http stats enable stats admin if TRUE stats show-legends stats uri /haproxy/haproxy_stats.php?haproxystats=1 timeout client 5000 timeout connect 5000 timeout server 5000 frontend http-https-redirect bind 2.1.1.2:80 name 2.1.1.2:80 mode http log global option http-server-close timeout client 30000 #redirect scheme https code 301 if !{ ssl_fc } redirect scheme https code 301 frontend https_frontend bind 2.1.1.2:443 name 2.1.1.2:443 ssl crt-list /var/etc/haproxy/https_frontend.crt_list mode http log global option http-server-close option forwardfor acl https ssl_fc http-request set-header X-Forwarded-Proto http if !https http-request set-header X-Forwarded-Proto https if https timeout client 90000 acl chatACL var(txn.txnhost) -m reg -i ^chat.mydomain.io(?::443)?$ acl momACL var(txn.txnhost) -m reg -i ^mom.mydomain.io(?::443)?$ acl homeACL var(txn.txnhost) -m reg -i ^home.mydomain.io(?::443)?$ acl speedACL var(txn.txnhost) -m reg -i ^speed.mydomain.io(?::443)?$ acl restACL var(txn.txnhost) -m reg -i ^rest.mydomain.io(?::443)?$ http-request set-var(txn.txnhost) hdr(host) use_backend backend-chat_ipvANY if chatACL use_backend backend-test-copy_ipvANY if momACL use_backend backend-home_ipvANY if homeACL use_backend backend-speedtest_ipvANY if speedACL use_backend backend-rest_ipvANY if restACL backend backend-chat_ipvANY mode http id 105 log global timeout connect 30000 timeout server 30000 retries 3 server chat 192.168.0.21:3000 id 106 check inter 1000 backend backend-test-copy_ipvANY mode http id 123 log global http-response set-header Strict-Transport-Security max-age=31536000; timeout connect 30000 timeout server 30000 retries 3 http-response set-header Content-Security-Policy "default-src 'none'; font-src 'self'; connect-src 'self' wss: ws: https://mb3admin.com; media-src 'self' blob: data:; manifest-src 'self'; base-uri 'none'; form-action 'self'; frame-ancestors 'self'; object-src 'none'; worker-src 'self' blob:; script-src 'self' https://www.gstatic.com; img-src data: https: http: ; style-src 'unsafe-inline' 'self'" server test_34 192.168.0.112:8096 id 126 backend backend-home_ipvANY mode http id 113 log global timeout connect 30000 timeout server 30000 retries 3 server homepi 69.0.0.150:8123 id 114 check inter 1000 backend backend-speedtest_ipvANY mode http id 129 log global http-response set-header Strict-Transport-Security max-age=31536000; timeout connect 30000 timeout server 30000 retries 3 #http-request set-header X-Client-IP req.hdr_ip([X-Forwarded-For]) server speed 192.168.0.19:2020 id 130 check inter 1000 backend backend-rest_ipvANY mode http id 111 log global timeout connect 30000 timeout server 30000 retries 3 server rest 192.168.0.19:3580 id 112 check inter 1000 -
I have a specific question regarding the HAProxy setup, In your wiki https://github.com/PiBa-NL/pfsense-haproxy-package-doc/wiki/pfsense_2_3_haproxy_sni_plus_offloading_backends you forward your request that arrive on wan side on port 80 and 443 to the address 192.168.0.133, my question here is, what is this ip, is it an ipalias in your network outside of the dhcp range that haproxy is listening on, or a virtual ip on the lan interface that haproxy is listening on. Would you mind guiding me.
-
@zwck
In my case 192.168.0.133 a virtual-ip of type alias on the pfSense WAN interface, of a test VM that runs with its wan on a private network segment.. Usually you would use the wan-ip of pfSense for that listening. Or wherever you want that haproxy frontend to listen.. Some websites i run with haproxy listening on the lan-ip, as they only need to be accessible when connected to the company network.. So its really up to you, where are your clients going to connect to, and do you want haproxy to directly handle that traffic, or are your going to create a portforward (though imho thats kinda ugly.., but then again, ive heard of haproxy failing to listen directly on a pppoe interface iirc..). -
@piba I would like to do it the pretty way, right now i create a virtual ip (ip alias) on the wan interface that is an ip that is outside of my network (lan 192.168.0.0/24) my ip alias is 2.1.1.2/32 then i have a nat portforward from wanadress to 2.1.1.2 of port 80 and 443 and have haproxy listen on that. I guess thats the ugly part :D
-
@zwck
Yup sounds 'ugly'
But well, shouldn't be to difficult to change that and make the frontend listen on the wan-ip right.? add a firewall rule to allow the traffic, and remove the portforward. -
@PiBa Thanks for your time and help ! And for the nat rule you just say allow 80/443 to any (e.g. *** http *** and *** https ***)
-
@zwck
So the firewall-rule it would look like:interface: WAN
protocol: TCP
source-address: any
source-port: any
destination-address: wan-ip
destination-port: 80 (or 443.. or what you want to allow traffic to.) -
thanks !
