HAProxy noob (SONARR NZBGET RADARR etc…)



  • Hi,

    I'm hoping someone can walk me through setting up HAProxy. I've been at it for well over a week and just can't get it to work for my particular situation.

    I've tried to follow (what feels like every setup i could google) instructions from other sites but I feel like I was always missing a piece of the puzzle.

    The bottom line is that I'm very green when it comes to networking. In addition I only started using pfsense 2 weeks ago.

    I have a QNAP Server that has all my media stored on it. On that server I'm running PLEX, SONARR, RADARR, NZBGET and OMBI - nothing is in a docker or VM. These are snap packages.

    The Server is on my network and not in a DMZ (a project for a later date).

    my home layout is => ISP MODEM (bridged) => pfsense (WAN and NAT ports and 2 others not being used) => ASUS Router (Access Point WIFI and meshed to second ASUS Router) => D-LINK 16 port (smart switch) => internal devices (iMacs) in addition to the QNAP871 Server.

    In case it matters, I have SNORT, pfBlockerNG and HAProxy packages installed.

    For a little while I ran the QNAP 'web server' and was able to do a reverse proxy using Apache (it was quite simple, but mostly because theres so many detailed walkthroughs out there on the GOOGLE). In that configuration I had everything running through port 443 via an SSL wildcard. I was able to type sonarr.myregisteredomain.com and it would take me to 192.168.1.10:6785 (sonarr) etc…

    This is what I'm trying to do:

    QNAP SERVER housing everything = .mydomain.com (SSL Wildcard)

    Have everything come through one SSL port with the reverse proxy; lets say port 5000. And HAProxy feeding the below ports into port 5000.

    192.168.1.10            QNAP
    192.168.1.10:6785    sonarr.mydomain.com
    192.168.1.10:6786    radarr.mydomain.com
    192.168.1.10:6787    ombi.mydomain.com
    192.168.1.10:6788    nzbget.mydomain.com

    I would also like to setup a redirect so that if i type sonarr.mydomain.com in my browser it will automatically redirect to https://sonarr.mydomain.com.

    This is what my apache custom.conf looked like before - if that kind of thing interests you:

    ServerName				localhost
    ServerSignature			Off
    ServerTokens			ProductOnly
    
    UseCanonicalName		On
    TraceEnable				Off
    
    Timeout					10
    MaxRequestWorkers		64
    
    LoadModule alias_module modules/mod_alias.so
    
    LoadModule log_config_module modules/mod_log_config.so
    LoadModule logio_module modules/mod_logio.so
    
    LoadModule proxy_module modules/mod_proxy.so
    LoadModule proxy_http_module modules/mod_proxy_http.so
    
    LoadModule ssl_module modules/mod_ssl.so
    LoadModule headers_module modules/mod_headers.so
    
    ErrorLogFormat          "[%{cu}t] [%-m:%-l] %-a %-L %M"
    LogFormat "%h %{GEOIP_COUNTRY_CODE}e %u [%{%Y-%m-%d %H:%M:%S}t.%{usec_frac}t] \"%r\" %>s %b \
    \"%{Referer}i\" \"%{User-Agent}i\" %v %A %p %R %{BALANCER_WORKER_ROUTE}e %X \"%{cookie}n\" \
    %{UNIQUE_ID}e %{SSL_PROTOCOL}x %{SSL_CIPHER}x %I %O %{ratio}n%% \
    %D %{ModSecTimeIn}e %{ApplicationTime}e %{ModSecTimeOut}e \
    %{ModSecAnomalyScoreIn}e %{ModSecAnomalyScoreOut}e" extended
    
    LogLevel				debug
    ErrorLog				/usr/local/apache/logs/error.log
    CustomLog				/usr/local/apache/logs/access.log extended
    
     <ifmodule headers_module="">Header always set Public-Key-Pins "pin-sha256=\"[PLACE KEY HERE]=\"; max-age=5184000; includeSubDomains"
    Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    Header set X-XSS-Protection "1; mode=block"
    Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
    Header set X-Content-Type-Options nosniff
    #Header set Content-Security-Policy "default-src 'self'; style-src 'self' data:; img-src 'self' data:; script-src 'self'; connect-src 'self';"</ifmodule> 
    
    # === MAIN =========================================================================
    
     <virtualhost *:443="" *:8081="">ServerName [NAME OF YOUR REGISTERED DOMAIN, FOR EXAMPLE SOMETHING.COM]
    ServerAlias [WWW.SOMETHING.COM]
    
    ErrorLog				/usr/local/apache/logs/main-error.log
    
    SSLEngine 				On
    SSLHonorCipherOrder     On
    SSLCompression          off
    
    SSLCertificateKeyFile   /[LOCATION OF YOUR KEY FILE]/private.key
    SSLCertificateFile      /[LOCATION OF YOUR CERTIFICATE]/mydomain.crt
    SSLCertificateChainFile /[LOCATION OF YOUR CHAINFILE, IF NEEDED]/rootca.crt
    SetEnvIf User-Agent		".*MSIE.*" nokeepalive ssl-unclean-shutdown
    
    SSLProtocol 			all -TLSv1.1 -TLSv1 -SSLv2 -SSLv3
    SSLCipherSuite 			ALL:+HIGH:!ADH:!EXP:!SSLv2:!SSLv3:!MEDIUM:!LOW:!NULL:!aNULL</virtualhost> 
    
    # === SONARR =========================================================================
    
     <virtualhost *:443="" *:8081="">ServerName [NAME OF YOUR SUB-DOMAIN, FOR EXAMPLE SONARR.SOMETHING.COM
    
    ErrorLog				/usr/local/apache/logs/sonarr-error.log
    
     <ifmodule headers_module="">Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    Header set X-XSS-Protection "1; mode=block"
    Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
    Header set X-Content-Type-Options nosniff</ifmodule> 
    
    ProxyTimeout			60
    ProxyErrorOverride		On
    
    SSLEngine 				On
    SSLHonorCipherOrder     On
    SSLCompression          off
    
    ProxyPass 				/ http://[QNAP IP]:6787/
    ProxyPassReverse 		/ http://[QNAP IP]:6787/
    
    SSLCertificateKeyFile   /[LOCATION OF YOUR KEY FILE]/private.key
    SSLCertificateFile      /[LOCATION OF YOUR CERTIFICATE]/mydomain.crt
    SSLCertificateChainFile /[LOCATION OF YOUR CHAINFILE, IF NEEDED]/rootca.crt
    SetEnvIf User-Agent		".*MSIE.*" nokeepalive ssl-unclean-shutdown
    
    SSLProtocol 			all -TLSv1.1 -TLSv1 -SSLv2 -SSLv3
    SSLCipherSuite 			ALL:+HIGH:!ADH:!EXP:!SSLv2:!SSLv3:!MEDIUM:!LOW:!NULL:!aNULL</virtualhost> 
    
    # === RADARR ======================================================================
    
     <virtualhost *:443="" *:8081="">ServerName [NAME OF YOUR SUB-DOMAIN, FOR EXAMPLE RADARR.SOMETHING.COM
    
    ErrorLog				/usr/local/apache/logs/radarr-error.log
    
     <ifmodule headers_module="">Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    Header set X-XSS-Protection "1; mode=block"
    Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
    Header set X-Content-Type-Options nosniff</ifmodule> 
    
    ProxyTimeout			60
    ProxyErrorOverride		On
    
    SSLEngine 				On
    SSLHonorCipherOrder     On
    SSLCompression          off
    
    ProxyPass 				/ http://[QNAP IP]:6788/
    ProxyPassReverse 		/ http://[QNAP IP]:6788/
    
    SSLCertificateKeyFile   /[LOCATION OF YOUR KEY FILE]/private.key
    SSLCertificateFile      /[LOCATION OF YOUR CERTIFICATE]/mydomain.crt
    SSLCertificateChainFile /[LOCATION OF YOUR CHAINFILE, IF NEEDED]/rootca.crt
    SetEnvIf User-Agent		".*MSIE.*" nokeepalive ssl-unclean-shutdown
    
    SSLProtocol 			all -TLSv1.1 -TLSv1 -SSLv2 -SSLv3
    SSLCipherSuite 			ALL:+HIGH:!ADH:!EXP:!SSLv2:!SSLv3:!MEDIUM:!LOW:!NULL:!aNULL</virtualhost> 
    
    # === NZBGET =========================================================================
    
     <virtualhost *:443="" *:8081="">ServerName [NAME OF YOUR SUB-DOMAIN, FOR EXAMPLE NZBGET.SOMETHING.COM
    
    ErrorLog				/usr/local/apache/logs/nzbget-error.log
    
     <ifmodule headers_module="">Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    Header set X-XSS-Protection "1; mode=block"
    Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
    Header set X-Content-Type-Options nosniff</ifmodule> 
    
    ProxyTimeout			60
    ProxyErrorOverride		On
    
    SSLEngine 				On
    SSLHonorCipherOrder     On
    SSLCompression          off
    
    ProxyPass 				/ http://[QNAP IP]:6789/
    ProxyPassReverse 		/ http://[QNAP IP]:6789/
    
    SSLCertificateKeyFile   /[LOCATION OF YOUR KEY FILE]/private.key
    SSLCertificateFile      /[LOCATION OF YOUR CERTIFICATE]/mydomain.crt
    SSLCertificateChainFile /[LOCATION OF YOUR CHAINFILE, IF NEEDED]/rootca.crt
    SetEnvIf User-Agent		".*MSIE.*" nokeepalive ssl-unclean-shutdown
    
    SSLProtocol 			all -TLSv1.1 -TLSv1 -SSLv2 -SSLv3
    SSLCipherSuite 			ALL:+HIGH:!ADH:!EXP:!SSLv2:!SSLv3:!MEDIUM:!LOW:!NULL:!aNULL</virtualhost> 
    
    # === PLEX =========================================================================
    
     <virtualhost *:443="" *:8081="">ServerName [NAME OF YOUR SUB-DOMAIN, FOR EXAMPLE PLEX.SOMETHING.COM
    
    ErrorLog				/usr/local/apache/logs/plex-error.log
    
     <ifmodule headers_module="">Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    Header set X-XSS-Protection "1; mode=block"
    Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
    Header set X-Content-Type-Options nosniff</ifmodule> 
    
    ProxyTimeout			60
    ProxyErrorOverride		On
    
    SSLEngine 				On
    SSLHonorCipherOrder     On
    SSLCompression          off
    
    ProxyPass 				/ http://[QNAP IP]:32400/
    ProxyPassReverse 		/ http://[QNAP IP]:32400/
    
    SSLCertificateKeyFile   /[LOCATION OF YOUR KEY FILE]/private.key
    SSLCertificateFile      /[LOCATION OF YOUR CERTIFICATE]/mydomain.crt
    SSLCertificateChainFile /[LOCATION OF YOUR CHAINFILE, IF NEEDED]/rootca.crt
    SetEnvIf User-Agent		".*MSIE.*" nokeepalive ssl-unclean-shutdown
    
    SSLProtocol 			all -TLSv1.1 -TLSv1 -SSLv2 -SSLv3
    SSLCipherSuite 			ALL:+HIGH:!ADH:!EXP:!SSLv2:!SSLv3:!MEDIUM:!LOW:!NULL:!aNULL</virtualhost> 
    
    # === TAUTULLI =========================================================================
    
     <virtualhost *:443="" *:8081="">ServerName [NAME OF YOUR SUB-DOMAIN, FOR EXAMPLE TAUTULLI.SOMETHING.COM
    
    ErrorLog				/usr/local/apache/logs/tautulli-error.log
    
     <ifmodule headers_module="">Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    Header set X-XSS-Protection "1; mode=block"
    Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
    Header set X-Content-Type-Options nosniff</ifmodule> 
    
    ProxyTimeout			60
    ProxyErrorOverride		On
    
    SSLEngine 				On
    SSLHonorCipherOrder     On
    SSLCompression          off
    
    ProxyPass 				/ http://[QNAP IP]:8660/
    ProxyPassReverse 		/ http://[QNAP IP]:8660/
    
    SSLCertificateKeyFile   /[LOCATION OF YOUR KEY FILE]/private.key
    SSLCertificateFile      /[LOCATION OF YOUR CERTIFICATE]/mydomain.crt
    SSLCertificateChainFile /[LOCATION OF YOUR CHAINFILE, IF NEEDED]/rootca.crt
    SetEnvIf User-Agent		".*MSIE.*" nokeepalive ssl-unclean-shutdown
    
    SSLProtocol 			all -TLSv1.1 -TLSv1 -SSLv2 -SSLv3
    SSLCipherSuite 			ALL:+HIGH:!ADH:!EXP:!SSLv2:!SSLv3:!MEDIUM:!LOW:!NULL:!aNULL</virtualhost> 
    
    # === OMBI =========================================================================
    
     <virtualhost *:443="" *:8081="">ServerName [NAME OF YOUR SUB-DOMAIN, FOR EXAMPLE OMBI.SOMETHING.COM
    
    ErrorLog				/usr/local/apache/logs/login-error.log
    
     <ifmodule headers_module="">Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    Header set X-XSS-Protection "1; mode=block"
    Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
    Header set X-Content-Type-Options nosniff</ifmodule> 
    
    ProxyTimeout			60
    ProxyErrorOverride		On
    
    SSLEngine 				On
    SSLHonorCipherOrder     On
    SSLCompression          off
    
    ProxyPass 				/ http://[QNAP IP]:8976/
    ProxyPassReverse 		/ http://[QNAP IP]:8976/
    
    SSLCertificateKeyFile   /[LOCATION OF YOUR KEY FILE]/private.key
    SSLCertificateFile      /[LOCATION OF YOUR CERTIFICATE]/mydomain.crt
    SSLCertificateChainFile /[LOCATION OF YOUR CHAINFILE, IF NEEDED]/rootca.crt
    SetEnvIf User-Agent		".*MSIE.*" nokeepalive ssl-unclean-shutdown
    
    SSLProtocol 			all -TLSv1.1 -TLSv1 -SSLv2 -SSLv3
    SSLCipherSuite 			ALL:+HIGH:!ADH:!EXP:!SSLv2:!SSLv3:!MEDIUM:!LOW:!NULL:!aNULL</virtualhost> 
    
    

    My sanity would really appreciate the help.

    Thanks.



  • So what does your haproxy.conf look like after working on it for a week? Can you share it perhaps there are some small things that need improving.?.



  • Okay beside that haproxy listens on 10.101.101.101 while your ipalias is 10.10.9.1 and that your domain names don't match websever vs webserver vs mydomain vs myregisteredomain.. (i presume typo's while obfuscating..) Is there a reason that your not using 'wan-ip' to listen on?

    What happens.?
    Is your domain radarr.webserver.media is pointing to the ip haproxy is listening on with a public DNS record? Or is it in fact a private ip.? And the dns points to your wan-ip and youve got a nat rule in place that forwards the traffic to haproxy?
    Do you get any error when trying to visit either domain name? Are the servers 'up' on the stats page?
    Is there a timeout or a 503 error? Or does perhaps the page load 'partially' with images or css missing.?.

    Locally the http://10.4.0.18:32402 does works properly? (Its not using ssl right?)



  • Sorry webserver DNS was just an example in my previous post (hence typos).

    This is what I have, and I'll use mydomain.com as an example DNS url.

    # Automaticaly generated, dont edit manually.
    # Generated on: 2018-04-29 20:04
    global
    	maxconn			5
    	stats socket /tmp/haproxy.socket level admin
    	uid			80
    	gid			80
    	nbproc			10
    	chroot			/tmp/haproxy_chroot
    	daemon
    	tune.ssl.default-dh-param	2048
    	server-state-file /tmp/haproxy_server_state
    
    listen HAProxyLocalStats
    	bind 127.0.0.1:2200 name localstats
    	mode http
    	stats enable
    	stats admin if TRUE
    	stats uri /haproxy/haproxy_stats.php?haproxystats=1
    	timeout client 5000
    	timeout connect 5000
    	timeout server 5000
    
    frontend webreverse
    	bind			10.101.101.101:80 name 10.101.101.101:80   
    	bind			10.101.101.101:443 name 10.101.101.101:443 ssl  crt /var/etc/haproxy/webreverse.pem  
    	mode			http
    	log			global
    	option			http-keep-alive
    	maxconn			100
    	timeout client		30000
    	redirect scheme https code 301 if !{ ssl_fc }
    	acl			sonarr	hdr(host) -i sonarr.mydomain.com
    	acl			radarr	hdr(host) -i radarr.mydomain.com
    	acl			aclcrt_webreverse	hdr_reg(host) -i ^([^\.]*)\.mydomain\.com(:([0-9]){1,5})?$
    	acl			aclcrt_webreverse	hdr_reg(host) -i ^mydomain\.com(:([0-9]){1,5})?$
    	use_backend sonarr_http_ipvANY  if  sonarr aclcrt_webreverse
    	use_backend radarr_http_ipvANY  if  radarr aclcrt_webreverse
    
    backend sonarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	server			sonarr 10.4.0.18:32401  
    
    backend radarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	server			radarr 10.4.0.18:32402
    

    So you're saying it's makes more sense to use my wan ip instead of the virtual ip correct? So I should NAT port forward 80 and 443 to 10.101.101.101; is that correct - I tried this and no difference.

    My current public DNS record is pointing to my WAN.

    when i go to radarr.mydomain.com I get this:

    Safari can't find the server
    safari can't open the page "https://radarr.mydomain.com" because it can't find the server "radarr.mydomain.com"

    Also when I enable HAProxy I get this warning:

    [WARNING] 118/194348 (44770) : stats socket will not work as expected in multi-process mode (nbproc > 1), you should force process binding globally using 'stats bind-process' or per socket using the 'process' attribute.
    [WARNING] 118/194348 (44770) : Proxy 'HAProxyLocalStats': in multi-process mode, stats will be limited to process assigned to the current request.
    [WARNING] 118/194348 (44770) : Proxy 'HAProxyLocalStats': stats admin will not work correctly in multi-process mode.

    I am able to access both radarr and sonarr at http://10.4.0.18:32401 or 32402 - both work internally on local address. They are not using SSl that's one of the reasons I wanted to use HAProxy SSL offloading.

    Thanks for your patience.



  • To get rid of the haproxy warnings do this, it'll work better also. On settings tab:
    -Decrease processes from 10 to 1
    -Increase maximum connections from 5 to 50
    Running with multiple processes only makes sense if you need hundreds of ssl connections or (ten-)thousands of http ones.. Which haproxy should be able to.. but i'm not sure if pfSense is then still the correct platform, probably a dedicated haproxy machine would become more convenient. But thats not yet the case here it seems..

    If you let haproxy use the wan-ip then there is no need to nat anything.. You do need to allow the traffic in through the firewall though with a rule.
    Make a 'pass' rule on the wan that allows from source any:any to destinations wan-ip:443  (and also a rule for wan-ip:80)
    Also i like to run the pfSense webgui on a different port than the default 80/443.

    I'm not completely sure what the message of Safari means that it cant find the server. Could be that it didn't find the dns record, but if you can send a ping to the name from the client device and it finds the correct ip that should be okay.. Or it just cant connect, in which case the isp / modem might be blocking traffic, or the firewall isnt allowing it yet.

    b.t.w. can you also change the healthcheck on the backend to do a 'http' healthcheck.? And then check on stats page again that the servers stay UP 'green'.



  • I feel like I'm close to getting this to work, but no cigar yet.

    • I decreased processes to 1 and increased maximum connections to 10 which got rid of the error warning
    • removed the nat and made a pass rule in WAN as you suggested; any to destinations wan-ip:443 and any to destinations wan-ip:80
    • changed health check to http
    • I am able to ping my domain.com and receive a response with the DNS IP

    Still no go…  :(

    I'm attaching images of my HAProxy GUI to see if theres something I'm not checking off that's causing the issue.

    Thanks again.

    PS. I have SNORT and pfBlockerNG running on the pfsense; just in case it makes a difference?



  • "Still no go…" What does that mean?  Error 503 ? or something else?

    It looks like the frontend got at leat one connection.. But i cant tell if thats on :80 or :443 (there is a checkbox to split frontend socket stats..) The :80 would get a redirect to https so never gets to a backend.. But the 443 one should (if the acls match)

    But it seems the acl's did not direct any connection to the backends yet.. That could be because your making a 'wrong request' perhaps while testing trying to visit haproxy by IP, and then the hostname requested doesnt match also the acl's gor the certificate would interfere with such a 'test'.

    b.t.w. you realize that a single browser loading a website can create multiple connections to the sane site right ;) , 10 connections is a really low limit for the global setting..  if you want to limit the number of connections to a server better do that on the backend config or the server config inside the backend.



  • lol. Still no go means:

    When I type in mydomain.com by itself I get 503 "Service unavailable" "No server is available to handle this request." in Firefox.

    When I type in sonarr.mydomain.com in firefox I get:

    Hmm. We’re having trouble finding that site.

    "We can’t connect to the server at server.webserver.media.
    If that address is correct, here are three other things you can try:

    Try again later.
        Check your network connection.
        If you are connected but behind a firewall, check that Firefox has permission to access the Web."


    Should I be using Transparent ClientIP on the backend? I haven't checked this to on.

    Should Http check method be "OPTIONS" or "GET"? I've been using "OPTIONS"?

    My Frontend ACL (sonarr) matches my Backend (sonarr)?

    # Automaticaly generated, dont edit manually.
    # Generated on: 2018-04-30 18:14
    global
    	maxconn			50
    	stats socket /tmp/haproxy.socket level admin
    	uid			80
    	gid			80
    	nbproc			1
    	chroot			/tmp/haproxy_chroot
    	daemon
    	tune.ssl.default-dh-param	2048
    	server-state-file /tmp/haproxy_server_state
    
    listen HAProxyLocalStats
    	bind 127.0.0.1:2200 name localstats
    	mode http
    	stats enable
    	stats admin if TRUE
    	stats uri /haproxy/haproxy_stats.php?haproxystats=1
    	timeout client 5000
    	timeout connect 5000
    	timeout server 5000
    
    frontend webreverse
    	bind			(my external WAN IP):80 name (my external WAN IP):80   
    	bind			(my external WAN IP):443 name (my external WAN IP):443 ssl  crt /var/etc/haproxy/webreverse.pem  
    	mode			http
    	log			global
    	option			socket-stats
    	option			http-keep-alive
    	maxconn			100
    	timeout client		30000
    	acl			sonarr	hdr(host) -i sonarr.mydomain.com
    	acl			radarr	hdr(host) -i radarr.mydomain.com
    	acl			aclcrt_webreverse	hdr_reg(host) -i ^([^\.]*)\.mydomain\.com(:([0-9]){1,5})?$
    	acl			aclcrt_webreverse	hdr_reg(host) -i ^mydomain\.com(:([0-9]){1,5})?$
    	use_backend sonarr_http_ipvANY  if  sonarr aclcrt_webreverse
    	use_backend radarr_http_ipvANY  if  radarr aclcrt_webreverse
    
    backend sonarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			sonarr 10.4.0.18:32401 check inter 1000  
    
    backend radarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			radarr 10.4.0.18:32402 check inter 1000
    

    Just read your BTW.

    I'll change the amount of connections. thanks



  • FYI…

    when I type in mydomain.com I get some action in the stats:

    but still I get 503 "Service unavailable" "No server is available to handle this request.



  • -Healthchecks are good as they are the L7OK message is what you want there.

    -Transparent-client-ip can be nice in the end, but doesn't fix things.. It can break things if enabled though.. Leave it off for now.

    -The 503 error is 'good'

    The domain name mydomain.com itself is not 'matched' by the 2 acl's that check for the complete host headers.

    mydomain.com v.s. webserver.media  is that indeed what happens? (or obfuscation typo?) If that really happens then we need to figure out where that redirect is coming from and fix that.? Could mean there was some response from the server.?.  As haproxy wouldn't just return a different domain.



  • webserver.media is a typo.

    I'm using mydomain.com (I have an actual DNS address) as an example/obfuscation.

    Anytime I refresh firefox using the example mydomain.com the front end bytes increase which means there is a flow of traffic going through correct?



  • BTW…

    This a more than likely  a dumb question but do I need to do a pass though on the LAN as I did with the WAN side?

    By the way this is the site I was initially using as an example:

    https://www.edwork.org/2017/06/27/pfsense-with-haproxy/



  • No extra firewall rules are needed on lan.

    mydomain.com is supposed to give a 503 error as you dont have a acl that checks for that domainname. But firewallrule is okay as it did connect to haproxy.

    If you visit sonarr.mydomain.com does the frontend also count some new connection / bytes transfered?

    The backends as in the last stats screenshot does not show traffic going to the backends yet.. So your request is not matching the defined acl's yet.

    Perhaps try and configure a 'default backend' in the frontend.? That would skip some of the acl's.. Youve got a good domainname in that certificate also.? Perhaps disable the automatically added certificate acl ? (disable checkboxes in cert section)



  • Ok disabled the, "Add ACL for certificate Subject Alternative Names".

    The certificate is wildcard certificate *.mydomain.com

    The only two options under Frontend "default backend" is sonarr or radarr.

    i tried choosing one of them but no bytes/traffic went through in stat. :(

    BTW in the Frontend advance section I have:

    Use "forwardfor" option - checked

    and

    In the Advanced pass thru section I have:

    redirect scheme https code 301 if !{ ssl_fc }

    Not sure if that matters.



  • The :443 frontend does show traffic arriving right? Can you post the current latest haproxy.conf?

    If there are no acls on the default_backend than basically every request should end up on that default backend..
    Unless perhaps if there is a different issue..

    Can you fill in the Logging options on settings tab as follows?:
    Syslog Host: /var/run/log
    Syslog Facility: local0
    Syslog Level: Informational
    Log Hostname: haproxy

    On the frontend enable 'detailed logging'
    Save and apply settings..

    Then try a request from the browser..

    Check under status/packagelogs/haproxy what shows up there.?



  • I removed the default backend as it didn't do anything but here's the status page and log with the default backend I created:

    Apr 30 21:04:08 haproxy haproxy[39695]: Proxy webreverse started.
    Apr 30 21:04:08 haproxy haproxy[39695]: Proxy sonarr_http_ipvANY started.
    Apr 30 21:04:08 haproxy haproxy[39695]: Proxy radarr_http_ipvANY started.
    Apr 30 21:04:08 haproxy haproxy[39695]: Proxy default_backend_http_ipvANY started.
    Apr 30 21:04:08 haproxy haproxy[93951]: Stopping frontend webreverse in 0 ms.
    Apr 30 21:04:08 haproxy haproxy[93951]: Stopping backend sonarr_http_ipvANY in 0 ms.
    Apr 30 21:04:08 haproxy haproxy[93951]: Stopping backend radarr_http_ipvANY in 0 ms.
    Apr 30 21:04:08 haproxy haproxy[93951]: Proxy webreverse stopped (FE: 1 conns, BE: 1 conns).
    Apr 30 21:04:08 haproxy haproxy[93951]: Proxy sonarr_http_ipvANY stopped (FE: 0 conns, BE: 0 conns).
    Apr 30 21:04:08 haproxy haproxy[93951]: Proxy radarr_http_ipvANY stopped (FE: 0 conns, BE: 0 conns).
    Apr 30 21:04:14 haproxy haproxy[40185]: 10.4.0.10:62273 [30/Apr/2018:21:04:14.559] webreverse~ default_backend_http_ipvANY/ <nosrv>0/-1/-1/-1/0 503 212 - - SC-- 0/0/0/0/0 0/0 "GET / HTTP/1.1"</nosrv>
    

    Below is traffic from me going to mydomain.com no traffic from radarr/sonarr without default backend enabled:

    # Automaticaly generated, dont edit manually.
    # Generated on: 2018-04-30 20:38
    global
    	maxconn			50
    	log			/var/run/log	local0	info
    	stats socket /tmp/haproxy.socket level admin
    	uid			80
    	gid			80
    	nbproc			1
    	chroot			/tmp/haproxy_chroot
    	daemon
    	tune.ssl.default-dh-param	2048
    	log-send-hostname		haproxy
    	server-state-file /tmp/haproxy_server_state
    
    listen HAProxyLocalStats
    	bind 127.0.0.1:2200 name localstats
    	mode http
    	stats enable
    	stats admin if TRUE
    	stats uri /haproxy/haproxy_stats.php?haproxystats=1
    	timeout client 5000
    	timeout connect 5000
    	timeout server 5000
    
    frontend webreverse
    	bind			(my external WAN IP):80 name (my external WAN IP):80   
    	bind			(my external WAN IP):443 name (my external WAN IP):443 ssl  crt /var/etc/haproxy/webreverse.pem  
    	mode			http
    	log			global
    	option			socket-stats
    	option			httplog
    	option			http-keep-alive
    	option			forwardfor
    	acl https ssl_fc
    	http-request set-header		X-Forwarded-Proto http if !https
    	http-request set-header		X-Forwarded-Proto https if https
    	maxconn			100
    	timeout client		30000
    	redirect scheme https code 301 if !{ ssl_fc }
    	acl			sonarr	hdr(host) -i sonarr.mydomain.com
    	acl			radarr	hdr(host) -i radarr.mydomain.com
    	use_backend sonarr_http_ipvANY  if  sonarr 
    	use_backend radarr_http_ipvANY  if  radarr 
    
    backend sonarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			sonarr 10.4.0.18:32401 check inter 1000  
    
    backend radarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			radarr 10.4.0.18:32402 check inter 1000
    

    This is the output from the log:

    Apr 30 20:38:44 haproxy haproxy[57627]: Proxy webreverse started.
    Apr 30 20:38:44 haproxy haproxy[57627]: Proxy sonarr_http_ipvANY started.
    Apr 30 20:38:44 haproxy haproxy[57627]: Proxy radarr_http_ipvANY started.
    Apr 30 20:38:44 haproxy haproxy[51916]: Stopping frontend webreverse in 0 ms.
    Apr 30 20:38:44 haproxy haproxy[51916]: Stopping backend sonarr_http_ipvANY in 0 ms.
    Apr 30 20:38:44 haproxy haproxy[51916]: Stopping backend radarr_http_ipvANY in 0 ms.
    Apr 30 20:38:44 haproxy haproxy[51916]: Stopping backend Default_backend_http_ipvANY in 0 ms.
    Apr 30 20:38:44 haproxy haproxy[51916]: Proxy webreverse stopped (FE: 1 conns, BE: 0 conns).
    Apr 30 20:38:44 haproxy haproxy[51916]: Proxy sonarr_http_ipvANY stopped (FE: 0 conns, BE: 0 conns).
    Apr 30 20:38:44 haproxy haproxy[51916]: Proxy radarr_http_ipvANY stopped (FE: 0 conns, BE: 0 conns).
    Apr 30 20:38:44 haproxy haproxy[51916]: Proxy Default_backend_http_ipvANY stopped (FE: 0 conns, BE: 1 conns).
    Apr 30 20:38:54 haproxy haproxy[65002]: Proxy webreverse started.
    Apr 30 20:38:54 haproxy haproxy[65002]: Proxy sonarr_http_ipvANY started.
    Apr 30 20:38:54 haproxy haproxy[65002]: Proxy radarr_http_ipvANY started.
    Apr 30 20:38:54 haproxy haproxy[57878]: Stopping frontend webreverse in 0 ms.
    Apr 30 20:38:54 haproxy haproxy[57878]: Stopping backend sonarr_http_ipvANY in 0 ms.
    Apr 30 20:38:54 haproxy haproxy[57878]: Stopping backend radarr_http_ipvANY in 0 ms.
    Apr 30 20:38:54 haproxy haproxy[57878]: Proxy webreverse stopped (FE: 0 conns, BE: 0 conns).
    Apr 30 20:38:54 haproxy haproxy[57878]: Proxy sonarr_http_ipvANY stopped (FE: 0 conns, BE: 0 conns).
    Apr 30 20:38:54 haproxy haproxy[57878]: Proxy radarr_http_ipvANY stopped (FE: 0 conns, BE: 0 conns).
    Apr 30 20:39:12 haproxy haproxy[65416]: 10.4.0.10:61464 [30/Apr/2018:20:39:12.810] webreverse~ webreverse/ <nosrv>-1/-1/-1/-1/1 503 212 - - SC-- 0/0/0/0/0 0/0 "GET / HTTP/1.1"
    Apr 30 20:39:50 haproxy haproxy[65416]: 185.227.153.226:53328 [30/Apr/2018:20:39:50.737] webreverse webreverse/ <nosrv>-1/-1/-1/-1/0 400 187 - - CR-- 0/0/0/0/0 0/0 "<badreq>"
    Apr 30 20:39:50 haproxy haproxy[65416]: 185.227.153.226:53367 [30/Apr/2018:20:39:50.967] webreverse webreverse/ <nosrv>1/-1/-1/-1/1 301 102 - - LR-- 0/0/0/0/0 0/0 "PROPFIND / HTTP/1.1"
    Apr 30 20:39:51 haproxy haproxy[65416]: 185.227.153.226:53449 [30/Apr/2018:20:39:51.411] webreverse webreverse/ <nosrv>0/-1/-1/-1/0 301 137 - - LR-- 0/0/0/0/0 0/0 "POST /wls-wsat/CoordinatorPortType HTTP/1.1"</nosrv></nosrv></badreq></nosrv></nosrv>
    

    Thanks..hope this helps



  • Currently it still reports 'webreverse/<nosrv>' so traffic was not forwarded to a webserver..

    Can you add this in "Advanced pass thru" of the frontend:

    capture request header Host len 100
    

    And check again what the package logfile shows?</nosrv>



  • Hi,

    This is what i got:

    May 1 15:55:46 haproxy haproxy[8219]: 10.4.0.10:53016 [01/May/2018:15:55:46.399] webreverse~ webreverse/ <nosrv>-1/-1/-1/-1/0 503 212 - - SC-- 1/1/0/0/0 0/0 {mydomain.com} "GET / HTTP/1.1"
    May 1 15:55:46 haproxy haproxy[8219]: 10.4.0.10:53014 [01/May/2018:15:55:46.304] webreverse webreverse/ <nosrv>0/-1/-1/-1/0 301 89 - - LR-- 1/1/0/0/0 0/0 {mydomain.com} "GET / HTTP/1.1"</nosrv></nosrv>
    

    If I type in sonarr.mydomain.com in the browser the log doesn't show anything. If I just type mydomain.com I get the above log event.



  • And your really really sure that mydomain.com and sonarr.mydomain.com point to the same IP ?



  • mydomain.com is definitely pointing to my WAN address. I've triple checked it.

    sonarr is running on my NAS with an ip of 10.4.0.18 and sonarr being on port 32401. Which matches with the info I put on the backend server.

    I must have not configured HAProxy correctly?

    # Automaticaly generated, dont edit manually.
    # Generated on: 2018-05-01 16:08
    global
    	maxconn			50
    	log			/var/run/log	local0	info
    	stats socket /tmp/haproxy.socket level admin
    	uid			80
    	gid			80
    	nbproc			1
    	chroot			/tmp/haproxy_chroot
    	daemon
    	tune.ssl.default-dh-param	2048
    	log-send-hostname		haproxy
    	server-state-file /tmp/haproxy_server_state
    
    listen HAProxyLocalStats
    	bind 127.0.0.1:2200 name localstats
    	mode http
    	stats enable
    	stats admin if TRUE
    	stats uri /haproxy/haproxy_stats.php?haproxystats=1
    	timeout client 5000
    	timeout connect 5000
    	timeout server 5000
    
    frontend webreverse
    	bind			xx.xx.xx.xx:80 name xx.xx.xx.xx:80   
    	bind			xx.xx.xx.xx:443 name xx.xx.xx.xx:443 ssl  crt /var/etc/haproxy/webreverse.pem  
    	mode			http
    	log			global
    	option			socket-stats
    	option			httplog
    	option			http-keep-alive
    	option			forwardfor
    	acl https ssl_fc
    	http-request set-header		X-Forwarded-Proto http if !https
    	http-request set-header		X-Forwarded-Proto https if https
    	maxconn			100
    	timeout client		30000
    	redirect scheme https code 301 if !{ ssl_fc }
    	capture request header Host len 100
    	acl			sonarr	hdr(host) -i sonarr.mydomain.com
    	acl			radarr	hdr(host) -i radarr.mydomain.com
    	use_backend sonarr_http_ipvANY  if  sonarr 
    	use_backend radarr_http_ipvANY  if  radarr 
    
    backend sonarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			sonarr 10.4.0.18:32401 check inter 1000  
    
    backend radarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			radarr 10.4.0.18:32402 check inter 1000
    

    I feel like it would be easier to just configure the haproxy.cfg file. Which, I've tried to do but it just gets reset as soon as I launch HAProxy in the pfsense GUI.



  • you want requests from a browser to sonarr to be handled by haproxy right? Then the dns record for sonarr must be pointing to the ip where haproxy is listening.. so sonarr.domain.com must be pointing to the wan-address as well.?

    basically you would need (imho) would be that there 3 commands:
      ping domain.com
      ping sonarr.domain.com
      ping radarr.domain.com
    All would perform the same ping to the wan-ip.

    If thats not the case, then in as i currently 'think' the desired state is you would need to reconfigure the DNS records for those names to point to the wanip..



  • HOLY MOTHER OF JESUS!! It worked…well at least radarr did. (see below) ;D

    Your last email made me think and I went on my dns and I had not checked off wildcard for mydomain.com.

    So sonarr.mydomain.com was not being recognized. So dumb!!!

    Thank you so much for your time and especially your patience. I would buy you a beer if you were close by.

    Now that I have ports 80 and 443 open…any suggestions on securing them better with pfsense?



  • Well.. these services should be available from 'the internet' right? So the ports must be open.. nothing can be done about that part..

    :80 doesn't need much securing as it is, as all requests are redirected to :443 anyhow.

    You should move the pfSense webgui to a different port, 1443 or something perhaps, and disable the webgui-redirect as that would keep listening on :80 also otherwise.., so that if haproxy for some reason stops running external people wont end up on the webgui if they visit/scan your wan-ip..

    You could try and use pfBlocker to limit the country's that can request the pages.. However geo-location aint a exact science. But maybe these items are only for a very limited set of known people.?. In that case you could add client-certificates to use for authentication on haproxy frontend ssl options if its only for yourself being 'on the road' then noone will be able to pass if they dont have the right client cert..

    Other than that there aint much i can think of a.t.m. .. Basically you need to trust that the security of the website itself and the separated network segment / hardware its hosted from are secure.. Unless someone else has a great idea and is willing to share that :)



  • Thanks buddy.

    I'm almost embarrassed to ask you this seeings you've already helped so much.

    How do I get the SSL certificate to work for "In that case you could add client-certificates to use for authentication on haproxy frontend ssl options if its only for yourself being 'on the road' then noone will be able to pass if they dont have the right client cert.."

    I have a wildcard certificate *.mydomain.com

    Under SSL offloading in my Frontend my certificate shows up and I have it chosen. Underneath are some tick boxes for:

    • Add ACL for certificate CommonName. (host header matches the "CN" of the certificate)
    • Add ACL for certificate Subject Alternative Names.
    • Load certificate ocsp responses for easy certificate validation by the client.

    Do I check off all of theses?



  • To use client certificates you would first create a CA certificate in pfSense System/CertManager. Then also create a UserCert that is signed by that CA (just select the ca while creating the usercert it will sign it automatically)
    (dont try use your real publicly signed cert for this.. it actually make thing less secure..)

    Then in haproxy configure the "Client verification CA certificates" select the created CA. And on your client devices download and import the user-certificate into the certificate manager of the OS your using or the browser certificate store..

    Now when visiting the website it should ask for the certificate and fail for users that don't have it..

    Another thing i just though of that you should do is to specify the ciphers that can be used. use below to generate the cipher settings for haproxy
      https://mozilla.github.io/server-side-tls/ssl-config-generator/
    To keep it 'simple' i would stick with the 2 ssl-default-bind-ciphers and ssl-default-bind-options settings and put them on the global tab. That should allow a A rating on ssllabs test iirc..

    • OCSP can be enabled it usually makes little difference, but checking the box is easy so why not ;)..
    • the acl's for CN and Alternative names, well they dont really add much functionality as your already performing these actions also yourself..


  • Thanks man.

    Followed what you said step by step but keep getting:

    Errors found while starting haproxy
    [ALERT] 122/084430 (51319) : parsing [/var/etc/haproxy_test/haproxy.cfg:29] : 'bind xx.xxx.xxx.xx:443' : unable to load SSL private key from PEM file '/var/etc/haproxy_test/webreverse.pem'. 
    [ALERT] 122/084430 (51319) : Error(s) found in configuration file : /var/etc/haproxy_test/haproxy.cfg 
    [ALERT] 122/084430 (51319) : Fatal errors found in configuration.
    

    no matter how many times i create a certificate and CA



  • Not sure whats going wrong here..

    Perhaps try to give either the CA or the *.domain.com a different name.?.  The /webreverse.pem file itself was working previously.. And there really is no good reason for that to have changed.. Maybe something got broken somewhere.. Try and re-import or perhaps restore a config from before it broke.?. (last 30 configs are under diagnostics/backup/history  in the gui)



  • Thanks for your response.

    Unfortunately neither of those two options worked.

    I created a few different CA's and user certificates to try with different names; also tried rebooting and then creating a CA/certificate - didn't work.

    Reverted to an earlier version of HAProxy, but that didn't work.

    I even tried uninstalling HAPROXY and installing HAProxy DEV but no good.

    You would happen to have any other ideas?

    # Automaticaly generated, dont edit manually.
    # Generated on: 2018-05-03 16:16
    global
    	maxconn			20
    	stats socket /tmp/haproxy.socket level admin  expose-fd listeners
    	uid			80
    	gid			80
    	nbproc			1
    	nbthread			1
    	hard-stop-after		15m
    	chroot				/tmp/haproxy_chroot
    	daemon
    	tune.ssl.default-dh-param	2048
    	server-state-file /tmp/haproxy_server_state
    	# set default parameters to the modern configuration
    	ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
    	ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
    
    listen HAProxyLocalStats
    	bind 127.0.0.1:2200 name localstats
    	mode http
    	stats enable
    	stats admin if TRUE
    	stats uri /haproxy/haproxy_stats.php?haproxystats=1
    	timeout client 5000
    	timeout connect 5000
    	timeout server 5000
    
    frontend webreverse
    	bind			xx.xx.xxx.xx:80 name xx.xx.xxx.xx:80   
    	bind			xx.xx.xxx.xx:443 name xx.xx.xxx.xx:443  ssl no-sslv3 crt /var/etc/haproxy/webreverse.pem crt-list /var/etc/haproxy/webreverse.crt_list  
    	mode			http
    	log			global
    	option			http-keep-alive
    	option			forwardfor
    	acl https ssl_fc
    	http-request set-header		X-Forwarded-Proto http if !https
    	http-request set-header		X-Forwarded-Proto https if https
    	maxconn			100
    	timeout client		7200000
    	# Remove headers that expose security-sensitive information.
    	rspidel ^Server:.*$
    	rspidel ^X-Powered-By:.*$
    	rspidel ^X-AspNet-Version:.*$
    
    	# add some security related headers
    	rspadd Content-Security-Policy:\ default-src\ https:\ data:\ \‘unsafe-inline\\’\ \\'unsafe-eval\'
    	rspadd X-Frame-Options:\ SAMEORIGIN
    	rspadd X-Content-Type-Options:\ nosniff
    	rspadd X-Xss-Protection:\ 1;\ mode=block
    	acl			radarr	var(txn.txnhost) -m str -i radarr.mydomain.com
    	acl			ombi	var(txn.txnhost) -m str -i ombi.mydomain.com
    	acl			sonarr	var(txn.txnhost) -m str -i sonarr.mydomain.com
    	acl			nzbget	var(txn.txnhost) -m beg -i nzbget.mydomain.com
    	acl			tautulli	var(txn.txnhost) -m beg -i tautulli.mydomain.com
    	http-request set-var(txn.txnhost) hdr(host)
    	use_backend sonarr_http_ipvANY  if  sonarr 
    	use_backend radarr_http_ipvANY  if  radarr 
    	use_backend ombi_http_ipvANY  if  ombi 
    	use_backend nzbget_http_ipvANY  if  nzbget 
    	use_backend tautulli_http_ipvANY  if  tautulli 
    
    backend sonarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			sonarr 10.4.0.18:6787 check inter 1000  
    
    backend radarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			radarr 10.4.0.18:32402 check inter 1000  
    
    backend ombi_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			ombi 10.4.0.18:8976 check inter 1000  
    
    backend nzbget_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			nzbget 10.4.0.18:6789 check inter 1000  
    
    backend tautulli_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			tautulli 10.4.0.18:8660 check inter 1000
    

    Thanks in advance…



  • This shouldnt be dependant on the version of haproxy used..

    unable to load SSL private key from PEM file '/var/etc/haproxy_test/webreverse.pem'.

    Can you check the content of that file, does it properly contain both the certificate and the (decrypted) psk ? Of your *.mydomain.com certificate ?

    if you download the certificate in the certmanager, does it contain the correct cert and key there as well?



  • I used pfsense Diagnostic Edit file to go:

    '/var/etc/haproxy_test/webreverse.pem'

    File is empty?

    When I checked the certificate it was empty too?

    This what I'm doing to create a certificate:

    ADD CA certificate
    Method - create internal CA (fill out info)
    common name - *.mydomain.com

    Add certificate
    Method - create internal certificate
    Certificate authority - same as above from pull down list
    fill out info
    common name - *.mydomain.com

    certificate type - user certificate
    add



  • Could it be that the selected "Certificate" on the frontend is not pointing to the 'official' signed cert anymore? Re-select and save that one?

    (its stored by 'refid', so if you delete a cert and re-create it with the same name it wont be the same cert anymore and it might create a empty file because of that.?.)

    Can you check inside the config.xml that the ID's used for the certificate in haproxy really do match those of a certificate?

    For example below im using a certificate with refid '5a4004718858f'..

    	 <cert><refid>5a4004718858f</refid>
    
    		<type>server</type>
    		<caref>57d3118d56766</caref>
    		<crt>My-Cert-DATA DATA DATA....</crt>
    		<prv>My-PSK-DATA DATA DATA....</prv></cert> 
    
    
    	 <installedpackages><haproxy><ha_backends><name>vhost1</name>
    					<status>active</status>
    					<secondary>yes</secondary>
    					<primary_frontend>TEST-SNI</primary_frontend>
    					<type>http</type>
    					<httpclose>http-keep-alive</httpclose>
    					<backend_serverpool>vhost1</backend_serverpool>
    					<ssloffloadcert>5a4004718858f</ssloffloadcert></ha_backends></haproxy></installedpackages>
    


  • this is what it says

     <cert><refid>5aeb86ca0c986</refid>
    
    		<type>user</type>
    		<caref>5aeb8678bcd07</caref></cert> 
    
     <haproxy><configversion>00.32</configversion>
    			 <ha_backends><name>webreverse</name>
    					<desc>site accessible tp public</desc>
    					<status>active</status>
    					<type>http</type>
    					<httpclose>http-keep-alive</httpclose>
    					<max_connections>100</max_connections>
    					<client_timeout>7200000</client_timeout>
    					<ssloffloadcert>5aeb86ca0c986</ssloffloadcert>
    					<dcertadv>no-sslv3</dcertadv></ha_backends></haproxy>
    

    I think the problem is that when I create an Internal certificate from the Internal CA the certificate is empty; I downloaded the certificate and it's zero bytes nothing inside?

    Is the process that I'm following to create the CA and certificate seem right? See below:

    ADD CA certificate
    Method - create internal CA (fill out info)
    common name - *.mydomain.com

    Add certificate
    Method - create internal certificate
    Certificate authority - same as above from pull down list
    fill out info
    common name - *.mydomain.com

    certificate type - user certificate
    Alternative Names - I didn't put anything here
    add


    Should I be creating an intermediate certificate too?



  • Some progress…  :o

    If a create a CA and then take the export CA info and add it to "import an existing Certificate" when creating a certificate, I no longer get the HAProxy error:

    Errors found while starting haproxy
    [ALERT] 122/084430 (51319) : parsing [/var/etc/haproxy_test/haproxy.cfg:29] : 'bind xx.xxx.xxx.xx:443' : unable to load SSL private key from PEM file '/var/etc/haproxy_test/webreverse.pem'. 
    [ALERT] 122/084430 (51319) : Error(s) found in configuration file : /var/etc/haproxy_test/haproxy.cfg 
    [ALERT] 122/084430 (51319) : Fatal errors found in configuration.
    

    When i go to backend sonarr.mydomain.com I get, "safari can't open the page because Safari can't establish a connection to the server" however when I go to one of the other backends ombi.mydomain.com I was prompted to download the certificate. Although, once I downloaded the certificate I got the same safari warning, "safari can't open the page because Safari can't establish a connection to the server"

    hmmm… :(

    UPDATE*

    I thought it was strange that some backends provided certificate while others didn't so I flushed my DNS Cache and now all the backends provided the certificate when I went to page, However after that still same Safari warning "safari can't open the page because Safari can't establish a connection to the server".

    ****** was going to delete the above, but decided not to in case it's useful to someone in the future. However, it's now irrelevant because of the below ******

    UPDATE TWO

    Well, after a very long night of having to setup pfsense from scratch (off the USB), I can now create user certificates that are populated with info from the Internal CA, and no longer blank. Looks like something was corrupting the ability to do that before.

    So this is where I'm at today:

    # Automaticaly generated, dont edit manually.
    # Generated on: 2018-05-04 11:34
    global
    	maxconn			20
    	stats socket /tmp/haproxy.socket level admin
    	uid			80
    	gid			80
    	nbproc			1
    	chroot			/tmp/haproxy_chroot
    	daemon
    	tune.ssl.default-dh-param	2048
    	server-state-file /tmp/haproxy_server_state
    	# set default parameters to the modern configuration
    
    	    ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
    	    ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
    	    ssl-default-server-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
    	    ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
    
    listen HAProxyLocalStats
    	bind 127.0.0.1:2200 name localstats
    	mode http
    	stats enable
    	stats refresh 10
    	stats admin if TRUE
    	stats uri /haproxy/haproxy_stats.php?haproxystats=1
    	timeout client 5000
    	timeout connect 5000
    	timeout server 5000
    
    frontend webreverse
    	bind			xx.xxx.xxx.xxx:80 name xx.xxx.xxx.xxx:80   
    	bind			xx.xxx.xxx.xxx:443 name xx.xxx.xxx.xxx:443 ssl  crt /var/etc/haproxy/webreverse.pem  
    	mode			http
    	log			global
    	option			socket-stats
    	option			http-keep-alive
    	option			forwardfor
    	acl https ssl_fc
    	http-request set-header		X-Forwarded-Proto http if !https
    	http-request set-header		X-Forwarded-Proto https if https
    	timeout client		7200000
    	acl			sonarr	hdr(host) -i sonarr.mydomain.com
    	acl			radarr	hdr(host) -i radarr.mydomain.com
    	acl			ombi	hdr(host) -i ombi.mydomain.com
    	acl			nzbget	hdr(host) -i nzbget.mydomain.com
    	use_backend sonarr_http_ipvANY  if  sonarr 
    	use_backend radarr_http_ipvANY  if  radarr 
    	use_backend ombi_http_ipvANY  if  ombi 
    	use_backend nzbget_http_ipvANY  if  nzbget 
    
    backend sonarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			sonarr 10.4.0.18:6787 check inter 1000  
    
    backend radarr_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			radarr 10.4.0.18:32402 check inter 1000  
    
    backend ombi_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			ombi 10.4.0.18:8976 check inter 1000  
    
    backend nzbget_http_ipvANY
    	mode			http
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	option			httpchk OPTIONS / 
    	server			nzbget 10.4.0.18:6789 check inter 1000
    
    • When I have a 'user certificate' under 'SSL Offloading' and the CA under 'SSL Offloading - client certificates' and the user certificate loaded on my computer and on my iPhone the webpage just reloads constantly asking to confirm certificate.

    • When I have a 'server certificate' under 'SSL Offloading' and nothing in 'SSL Offloading - client certificates' then the webpage loads in https.

    The only things I have altered from default in the HAProxy Frontend are:

    Advanced settings - Client timeout - 7200000
    SSL Offloading - OCSP - checked

    PS. When creating the certificate for both user and server. I add 'Alternative Names' for each of my backends. For example:

    sonarr.mydomain.com
    radarr.mydomain.com

    In case that makes a difference.

    Any thoughts on why the webpage reloads constantly with a user certificate?

    Cheer, and thanks in advance.



    • When I have a 'user certificate' under 'SSL Offloading' and the CA under 'SSL Offloading - client certificates' and the user certificate loaded on my computer and on my iPhone the webpage just reloads constantly asking to confirm certificate.

    The user certificate is only for the user, it should not be configure in haproxy

    • When I have a 'server certificate' under 'SSL Offloading' and nothing in 'SSL Offloading - client certificates' then the webpage loads in https.

    This is a good starting point.. Now configure the CA that is used to generate the usercertificate under the client certificate options..

    When creating the certificate for both user and server. I add 'Alternative Names' for each of my backends. For example:
    sonarr.mydomain.com
    radarr.mydomain.com

    But you have a valid *.mydomain.com certificate right.? So no need to create a servercert yourself..

    As for the user certificate, it doesnt need any alternative-names… just put in 'zanesavage' as the CN for the user-cert.. and download that to import into the client device OS or Browser certificate store..



  • What do you put in the Base URL on Ombi, Sonarr, etc. when using HAProxy?

    EDIT: I think I found the answer. You don't need it unless you want /ombi for example after your domain name. (yourdomain.com/ombi)


Log in to reply