[HAProxy-1_5] SNI ACL don't work

gparisse · Mar 9, 2015, 3:20 PM

Hi all !

After many tests, I desperately turns to you to find a solution to my problem.

The context I has several sites that I want to publish on the Internet, 2 to start. Workspace will respond when I hit workspace.mydom.com and Extranet should respond to other requests.

So I configured a frontend shared with TLS SNI ACL when the URL matches "workspace.mydom.com" for "Workspace" and "Extranet" when the URL is different from "workspace.mydom.com." Below configuration automatically generated:

/var/etc/haproxy.cfg file contents:
global
	maxconn			10
	stats socket /tmp/haproxy.socket level admin
	uid			80
	gid			80
	nbproc			1
	chroot			/tmp/haproxy_chroot
	daemon
	tune.ssl.default-dh-param	1024

frontend Workspace-merged
	bind			88.177.196.184:443 name 88.177.196.184:443   
	mode			tcp
	log			global
	timeout client		30000
	tcp-request inspect-delay	5
	acl			aclusr_ssl_sni_matches_workspace.mydom.com 	req.ssl_sni -i workspace.mydom.com
	tcp-request content accept if { req.ssl_hello_type 1 }

	use_backend		Redmine_https_ipvANY if aclusr_ssl_sni_matches_workspace.mydom.com
	use_backend		Netscaler_https_ipvANY if !aclusr_ssl_sni_matches_workspace.mydom.com

backend Redmine_https_ipvANY
	mode			tcp
	balance			source
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk
	server			WEB001 10.255.255.1:443 check-ssl  weight 10 verify none 

backend Netscaler_https_ipvANY
	mode			tcp
	balance			source
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk
	server			CTX002 10.255.255.102:443 check-ssl  weight 10 verify none

My concern is that the ACL I positioned absolutely do not work.
Either when I type "workspace.mydom.com" I randomly redirected to the backend "extranet" or from version 1_5 pkg v 0.21 as soon as I position an ACL I get an error "ERR_CONNECTION_CLOSED."

Is it a concern of the package? My configuration? A non-supported scenario?

I can not understand this :/
Thank you in advance for your answers :P

G.

PiBa · Mar 9, 2015, 8:23 PM

Basically your config looks good to me..
Could you configure the internal stats port(settings tab) and check that both backends are 'up' in the stats. You might need to change the check mode on the backends to get them to show green which is usually desired..

gparisse · Mar 9, 2015, 10:29 PM

Thanks for your answer. Thank you for your reply. Attached is a copy of the stats that show that all backends are up. I also changed the configuration of the backend but still nothing. I did a test without ACL and everything works. Apparently it is the ACLs that pose problem in my case.

![Screen Shot 2015-03-09 at 23.20.00.png](/public/imported_attachments/1/Screen Shot 2015-03-09 at 23.20.00.png)
![Screen Shot 2015-03-09 at 23.20.00.png_thumb](/public/imported_attachments/1/Screen Shot 2015-03-09 at 23.20.00.png_thumb)

PiBa · Mar 9, 2015, 11:04 PM

Both backends are 'grey', meaning they are "not checked", so its not possible to see if they are up or down from this you have to set the 'Health check method' to 'HTTP' to get the best confirmation that way.. However if removing the acl's and then pointing to one of the backends makes it 'work' that by itself confirms its configured alright for the connection part i suppose..

What kind of client do you use (a normal browser or some custom application)? And does it send the SNI information in the request it sends?
For example older browsers (IE on XP), some mobile devices, also custom applications don't always send the SNI extension information in the SSL connection request..

gparisse · Mar 10, 2015, 2:06 PM

Okay, I do not have a good look then …

To my browser, I work with the latest version of Google Chrome but I also tested with IE 11 and the latest version of Mozilla Firefox. Same result on 3. I ran a test using a site to test if my browser is sending SNI information and apparently so is the case but at pfsense HAProxy and nothing ...

Since the last package I get the message "ERR_CONNECTION_CLOSED" before I was randomly redirected to the backend "default" (with exclusion of other backends)

PiBa · Mar 11, 2015, 12:01 AM

I've just run a little test of my own using Chrome as a client. And the acl's written do seem to work OK..
The domain your trying to match does exactly match the hostname you mention in the acl?

This works for me..:

frontend testSNI-merged
	bind			0.0.0.0:445 name 0.0.0.0:445   
	mode			tcp
	log			global
	timeout client		30000
	tcp-request inspect-delay	5
	acl			aclusr_ssl_sni_matches_vhost2.pfs.local	req.ssl_sni -i vhost2.pfs.local
	tcp-request content accept if { req.ssl_hello_type 1 }

	use_backend		pb3_451_https_ipvANY if !aclusr_ssl_sni_matches_vhost2.pfs.local 
	use_backend		pb3_452_https_ipvANY if aclusr_ssl_sni_matches_vhost2.pfs.local

Do you have a running syslog server somewhere, or can setup one, so its possible to send logging there?
What would be interesting is what 'flags' haproxy shows for the failing requests.

b.t.w. your setting of maxconn 10 might be a bit on the low side.. Can you increase it to lets say 100 ?

gparisse · Mar 11, 2015, 4:52 PM

I check again and typed host name is the same as in the ACL.
I increased the maximum number of connections to 100.
And I sent the "debug" (and I try others) information to a syslog and this is what I get:

Mar 11 17:39:48 localhost haproxy[79752]: Proxy Workspace started.
Mar 11 17:39:59 localhost haproxy[82259]: Connect from X.X.X.X:53597 to X.X.X.X:443 (Workspace/TCP)
Mar 11 17:39:59 localhost haproxy[82259]: Connect from X.X.X.X:53598 to X.X.X.X:443 (Workspace/TCP)
Mar 11 17:39:59 localhost haproxy[82259]: Connect from X.X.X.X:53600 to X.X.X.X:443 (Workspace/TCP)
Mar 11 17:40:04 localhost haproxy[82259]: Connect from X.X.X.X:53607 to X.X.X.X:443 (Workspace/TCP)
Mar 11 17:40:05 localhost haproxy[82259]: Connect from X.X.X.X:53608 to X.X.X.X:443 (Workspace/TCP)

It still does not work and no information in syslog …

PiBa · Mar 11, 2015, 8:09 PM

Ok add the 'Detailed logging' on the frontend then it should show a little more info in the syslog output.

Should show something like this:
192.168.0.40:1822 [11/Mar/2015:20:49:06.494] testSNI-merged pb3_451_https_ipvANY/local451-a 4/1/274 10928 SD 2/2/2/1/0 0/0

http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#8.2.2

In my case the letters SD tells the connection to the server was reset/closed.

Another angle could be to perform a tcpdump on both wan and lan side of haproxy to capture all packets during a connection attempt and check what connections are made and if data is send over it..

gparisse · Mar 15, 2015, 10:59 PM

I was able to move a little on my side on my worries and with your guidance I could identify some of my worries.
Indeed, the error portion "ERR_CONNECTION_CLOSED" only appears on some machine running on the Windows 10 OS in a network. As against any worries with other sites using SNI … Weird.

But I've always another worry, the random redirection of my Workspace site to the backend Netscaler with no apparent reason. This happens when I browse the application (click on links, updating, ...). Below is an excerpt from my syslog logs corresponding to my tests:

Mar 15 23:46:39 localhost haproxy[11597]: X.X.X.X:56838 [15/Mar/2015:23:46:39.612] Workspace-merged Redmine_https_ipvANY/WEB001 6/0/355 168 -- 5/5/2/2/0 0/0
Mar 15 23:46:39 localhost haproxy[11597]: X.X.X.X:56839 [15/Mar/2015:23:46:39.624] Workspace-merged Redmine_https_ipvANY/WEB001 6/0/347 168 -- 3/3/0/0/0 0/0
Mar 15 23:46:39 localhost haproxy[11597]: X.X.X.X:56840 [15/Mar/2015:23:46:39.618] Workspace-merged Redmine_https_ipvANY/WEB001 7/0/350 168 -- 4/4/1/1/0 0/0
Mar 15 23:46:40 localhost haproxy[11597]: X.X.X.X:56836 [15/Mar/2015:23:46:39.597] Workspace-merged Netscaler_https_ipvANY/CTX002 7/3/594 3760 -- 2/2/2/2/0 0/0
Mar 15 23:46:40 localhost haproxy[11597]: X.X.X.X:56837 [15/Mar/2015:23:46:39.599] Workspace-merged Netscaler_https_ipvANY/CTX002 5/3/637 3760 -- 1/1/1/1/0 0/0
Mar 15 23:46:40 localhost haproxy[11597]: X.X.X.X:56841 [15/Mar/2015:23:46:39.864] Workspace-merged Netscaler_https_ipvANY/CTX002 7/2/670 3760 -- 4/4/1/1/0 0/0
Mar 15 23:46:41 localhost haproxy[11597]: X.X.X.X:56843 [15/Mar/2015:23:46:40.318] Workspace-merged Redmine_https_ipvANY/WEB001 5/0/1501 7414 -- 5/5/4/4/0 0/0
Mar 15 23:46:41 localhost haproxy[11597]: X.X.X.X:56845 [15/Mar/2015:23:46:40.465] Workspace-merged Netscaler_https_ipvANY/CTX002 7/2/1014 4512 -- 4/4/1/1/0 0/0

The URL typed into my browser is exclusively that of the workspace (navigation in the application and updating), and it is clear that by the time HAProxy forwarding me the wrong backend …

Any idea?

PiBa · Mar 15, 2015, 11:57 PM

Can you make this change in /usr/local/pkg/haproxy.inc ? i think 5ms second value is to small and it should have been 5 seconds..:


--				fwrite ($fd, "\ttcp-request inspect-delay\t" . $inspectdelay . "\n");
++				fwrite ($fd, "\ttcp-request inspect-delay\t" . $inspectdelay . "s\n");

gparisse · Mar 16, 2015, 12:25 PM

I just made the change you've told me. For now it seems to work.
If ever the problem persists, I will give you the news.

Thank you very much for your help:-)

PiBa · Mar 16, 2015, 6:46 PM

Should be fixed in pkg v0.22, pullrequest send. https://github.com/pfsense/pfsense-packages/pull/834 will probably be committed and be on the package repository in a day or so.