Completely stumped as to why HAProxy does not connect me to backend.
-
Hi,
I'm currently having issues connecting to my HAProxy frontend from a remote destination.
When I connect from a remote address, I do not get a response. However, if I enter in my WAN address into my browser on LAN, I get a 503 service unavailable request.
global maxconn 1000 log /var/run/log local0 info stats socket /tmp/haproxy.socket level admin uid 80 gid 80 nbproc 1 chroot /tmp/haproxy_chroot daemon tune.ssl.default-dh-param 2048 #Modern browser compatibility only as mentioned here: # https://wiki.mozilla.org/Security/Server_Side_TLS ssl-default-bind-ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK tune.ssl.default-dh-param 2048# Time-to-first-Byte (TTFB) value needs to be optimized based on # the actual public certificate chain see # https://www.igvita.com/2013/10/24 # /optimizing-tls-record-size-and-buffering-latency/ tune.ssl.maxrecord 1370 listen HAProxyLocalStats bind 127.0.0.1:2200 name localstats mode http stats enable stats admin if TRUE stats uri /haproxy/haproxy_stats.php?haproxystats=1 timeout client 5000 timeout connect 5000 timeout server 5000 frontend front_http bind #Censored#:80 name #Censored#:80 mode http log global option http-keep-alive option forwardfor acl https ssl_fc http-request set-header X-Forwarded-Proto http if !https http-request set-header X-Forwarded-Proto https if https timeout client 30000 default_backend backend_http_ipvANY frontend front_https bind #Censored#:443 name #Censored#:443 ssl crt /var/etc/haproxy/front_https.pem mode http log global option http-keep-alive timeout client 30000 acl aclcrt_front_https hdr_reg(host) -i ^butterantoast\.com(:([0-9]){1,5})?$ acl aclcrt_front_https hdr_reg(host) -i ^www\.butterantoast\.com(:([0-9]){1,5})?$ use_backend backend_http_ipvANY if aclcrt_front_https backend backend_http_ipvANY mode http log global timeout connect 30000 timeout server 30000 retries 3 redirect scheme https if !{ ssl_fc } server owncloud 192.168.1.109:80I have not yet tried connecting using my domain name because I don't think that is the issue here and I have not yet gotten DNS forwarding setup yet(with my Domain provider).
The main Issue that I want to attack is not being able to connect via remote access.
I am completely stumped.
Any ideas where I should start troubleshooting?
I would be grateful for your help,
Thank you in advance,
Michael L. -
Your site looks to be 'working' over :80 but sending wrong content.? And because of that shows a empty page.?
The fixed ip in the frame isnt right..?
As for the :443 frontend it will only respond when one of the two domain names is requested.. Testing with a wan-ip wont work there.. As no backend will be selected due to the certificate acl's..
-
Your site looks to be 'working' over :80 but sending wrong content.? And because of that shows a empty page.?
The fixed ip in the frame isnt right..?
As for the :443 frontend it will only respond when one of the two domain names is requested.. Testing with a wan-ip wont work there.. As no backend will be selected due to the certificate acl's..
Oh, are you talking about the areas where I replaced my IP with #censored#? I did that out of paranoia, just so I wouldn't be posting my public ip. The part in there is a ip address and not a name or domain name.
Hmm, okay I see.
by 'working' are you implying that the server at the endpoint is at fault here?
I apologise, I'm not quite sure what you mean by frame isn't right. I don't see any part of that code you posted shown in my config.
–-----------------
On a different note, I've tried connecting to the http frontend using my free duckdns.org DNS (remotely) and doesn't even give me a response.
But, I suppose because I've added in the line
redirect scheme https if !{ ssl_fc }it automatically redirects me to HTTPS and therefore by default will not work with ip or different domain name.
However, I've tried removing that line and testing again, with the same results. (getting no response)
–---------
I am stumped, Is this happening because pfsense will not even accept incomming connections through port 443 or 80 since it is being used by PFsense GUI?
Diagnostics>Sockets list shows port 80 and 443 being taken up by nginx(which I assume is the pfsense server for the GUI).
root nginx 32282 8 tcp4 *:80 *:* root nginx 32100 6 tcp4 *:443 *:* -
Haproxy should be able to bind on the wanip:443 even while nginx is also listening on *:443 , though imho that would be a security risk if for some reason haproxy isnt running your webgui would be exposed to the internet..
Few things i would propose to change for now.
-enable http checks on the backend, and make the server is shown in 'green' on the stats page.
-disable the checkbox for the certificate acl's .
-make sure your request actually passes through the frontend/backend by checking counters on the stats, or maybe better gather the syslogs it can send.. -
Haproxy should be able to bind on the wanip:443 even while nginx is also listening on *:443 , though imho that would be a security risk if for some reason haproxy isnt running your webgui would be exposed to the internet..
Few things i would propose to change for now.
-enable http checks on the backend, and make the server is shown in 'green' on the stats page.
-disable the checkbox for the certificate acl's .
-make sure your request actually passes through the frontend/backend by checking counters on the stats, or maybe better gather the syslogs it can send..By default, I'm assuming that PFSense does not allow connections to the GUI via WAN interface. (right?) I was a bit confused at first when I saw that nginx in the sockets list listening to port 80 and 443. My install is mostly default (as-is:out of the box)with a few small changes made (port forwarding) and (local DNS resolver).
I'm assuming to allow pfsense GUI to be accessible on WAN, that the admin needs to add a Firewall rule to open :80/:443 on WAN right? And it seems like you're implying the same.
I guess that's my newbie mistake, my appologies.
(I had that though in the back of my mind, but I never thought about it).–----------------------
So, if that were the case,
How could I go about allowing :80/443 traffic to my firewall but prevent PFSense GUI from listening on WAN:80/:443 and only permit HAProxy?
Also, I noticed that the status of the server (in HAProxy stats) is indeed down. I guess that explains half of my problem... T.T
The backend server is an Owncloud server using Nginx. Is there any specific configuration I need to add Nginx to allow HAProxy?
Edit: the error shows as "Layer 7 wrong status: not allowed"
-Man, I feel like a goof for missing all these simple details, my apologies.
-
Normally pfSense will block all traffic from the WAN and as such be secure.. However if you want to run a website you will need to open the 80 and or 443 ports..
Webgui will (sadly) always listen on 0.0.0.0, so only choice there is moving it to a different port.. (to get rid of the :80 nginx you could disable the webgui redirect)
The 'not allowed' might be solvable by choosing a different check method like GET or HEAD instead of OPTIONS for example. Or perhaps sending a domain name in the check. https://github.com/PiBa-NL/pfsense-haproxy-package-doc/wiki/haproxy_troubleshooting
-
I have just changed my webGUI port to 8081 and added a rule for WAN on 80 and 443 also, I've changed the health check to GET rather than OPTION and it works!!
Connecting to my fileserver now works when trying to access HAProxy from the local network (I enter in www.butterantoast.com from my web browser and it proxies me to my server)
Feels great that progress is being made!! :D Thank you!
–----
One last(possibly) issue. I still have a problem connecting to my backend via Remote access. I'm testing this using my cellular network. I can connect to the backend server on the WAN url (remotely) but not when there is a path after it. IE: https://butterantoast.com/owncloud
I locally use the path '<serverip>/owncloud' to connect to the file server.
HAProxy shouldn't have any problems accepting extra paths right?</serverip>
-
The https://www.butterantoast.com/owncloud/ does end up on owncloud but then tells it isnt trusted/known/configured on the owncloud configuration..
The other url https://butterantoast.com/owncloud/ however results in a 302 redirect to a 'http' page:
Request URL:https://butterantoast.com/owncloud/ Request Method:GET Status Code:302 Found Response Headers Location:http://butterantoast.com/owncloud/login Server:nginx/1.10.1If that would have been 'https://butterantoast.com/owncloud/login' then i do get the login page.
It looks like this is something that needs to be configured on the owncloud configuration owncloud documentation mentions "overwriteprotocol"=>"https" could be set, i 'think' that might solve the issue.. https://doc.owncloud.org/server/8.2/admin_manual/configuration_server/reverse_proxy_configuration.htmlp.s. it seems :80 isnt working/listening..
-
This is absolutely perfect! I have added HAProxy as a trusted proxy. It's working flawlessly now! Thanks for your help!
Today I learned… :)