Syslog-ng TLS configuration help (2.1.5)
-
Best practice for shipping logs is to not do it in the clear, so I'm trying to setup TCP TLS syslog on pfsense. I am getting this error when I try to connect:
Feb 25 10:02:20 ab1234-cbas syslog-ng[57932]: SSL error while writing stream; tls_error='SSL routines:SSL23_WRITE:ssl handshake failure'
Has anyone got TLS working for syslog-ng? I searched the forum and didn't find any questions or answers about TLS, so I'm assuming that either it isn't common or i've missed something blindingly simple and most people set it up without a hitch…
I have installed syslog-ng on two different pfsense boxes, one to act as a "client," another to act as a "server." This is a test, mind, I'll have a different server in production when we get down to that.
I have configured the "client" system according to these instructions:
http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/procedure-configuring-mutual-tls-client.htmlI have configured the "server" system according to these instructions:
http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/procedure-configuring-mutual-tls-server.htmlI placed the local host's key and certificate files in /var/syslog-ng/cert.d
I placed the far end CA's cert in /var/syslog-ng/ca.don the "client," I have configured a destination object with these Object Parameters:
{ tcp("64.141.171.111" port(1999) tls( ca-dir("/var/syslog-ng/ca.d") key-file("/var/syslog-ng/cert.d/ab1234.key") cert-file("/var/syslog-ng/cert.d/ab1234_local.crt")) ); };
Also on the "client" I have configured a log object with these Object Parameters:
{ source(_DEFAULT); destination(tls_server_dest); };
On the "server" I have configured a source object with these Object Parameters:
{ tcp(ip(0.0.0.0) port(1999) tls( key-file("/var/syslog-ng/cert.d/vmw-mgmt-fw02.key") cert-file("/var/syslog-ng/cert.d/vmw-mgmt-fw02.pem") ca-dir("/var/syslog-ng/ca.d/ab1234CA.crt")) ); };
Now, I expected that line on the server to create a listener on the system on port 1999 - it doesn't. So I went to the General tab on the server, changed the protocol to TCP and the port to 1999. Maybe this workaround just doesn't work?
Once that was done, I started getting the "ssl handshake failure" message on the client.
The only thing I see on the "server" is a connect followed by an immediate close.I have tried disabling the requirement for peer verification - I get the same error message.
If I try to connect using opensl at the command prompt, I get the same error.Can anyone help with this? I'm not sure which side is failing form the logs that I have, and frankly I'm stumped at the moment.
after the connection, this is what the "client" logs report as the conneciton "breaks"
Feb 25 10:20:36 ab1234-cbas syslog-ng[57932]: I/O error occurred while writing; fd='32', error='Broken pipe (32)' Feb 25 10:20:36 ab1234-cbas syslog-ng[57932]: SSL error while writing stream; tls_error='SSL routines:SSL23_WRITE:ssl handshake failure' Feb 25 10:20:36 ab1234-cbas syslog-ng[57932]: Syslog connection broken; fd='32', server='AF_INET(64.141.171.111:1999)', time_reopen='60'
-
I did not test it yet on pfsense, but on generic FreeBSD this tutorial worked fine: http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.6-guides/en/syslog-ng-tutorial-mutual-auth-tls/html-single/index.html
-
Thanks, those are the same as the instructions I followed, no luck. when I do a netstat -ln on the "server" box, I don't see a listener on the TCP port I configured, but I do see a stream listener pointing to the syslog-ng file. I re-configured the "server" side to have the standard plugin run on UDP 5140, and used the Advanced tab to create the listener on port 1999 TCP. When I attempt to connect from the client, I don't see the server answer with a TLS exchange as one would expect. the logs on the server are no help - when port 1999 is configured as per the linked instructions, when the client attempts to conenct, I get nothing in the servers logs. I've even tried it with mutual auth disabled - no luck.
-
I have vastly simplified this in an effort to figure out what is going wrong, and I THINK I have a connection, just without authentication. Here is what I've done.
I have managed to get an un-authenticated TLS connection going, which is happy news. I've moved to testing on a firewall that uses RFC1918 space on its "wan" connection to avoid additional complications like NAT for the moment.
First, I configured JUST a TCP connection with a destination object configured like this:
{ tcp("192.168.48.101" port(6515) ); };
and the Server end configured to receive on TCP 6514, the server receives logs - so TCP is working.
Next, when I tell the Server where it's CA is stored, as well as it's private key and certificate files, I configure the destination object in syslog-ng like this:
{ tcp("192.168.48.101" port(6514) tls( ca-dir("/var/syslog-ng/ca.d")) ); };
and the far end gives me the error "unknown ca". This led me to try to use openssl to manually connect using the same information. When I issue this command at the command prompt on pfsense, I get connected and the far end can see data I type:
openssl s_client -tls1 -connect 192.168.48.101:6514 -CApath /var/syslog-ng/ca.d
This made me wonder if, unlike the examples I'd been working from, a statement telling it to explicitly ignore the trust of the certificate would work, so I re-configured my destination object on the pfsense client like so:
{ tcp("192.168.48.101" port(6514) tls( ca-dir("/var/syslog-ng/ca.d") peer-verify(optional-untrusted)) ); };
and now I'm seeing an established TCP connection between the firewall and the log server. Sadly, the person who is responsible for the log server had to step out for the afternoon, but it does appear like I was able to get an encrypted, but un-authenticated, tls connection between pfsense and a log receiver. so getting warmer….
-
I got this working, sort of. I can get it to send encrypted data, but somehow my certificate chain is invalid, even though I can manaully verify it with openssl at the command prompt of BOTH client and server.
I stopped using pfsense as the syslog "server," as the actual server I was attempting to connect to was made available to me. I did, however, continue to use pfsense as the CA for both the server (logstash) and the client (pfsense 2.1.5).
One thing omitted fromthe "client" instructions in some cases is the peer-verify option. If a vlaue ofr peer-verify isn;t specified on the client, the default behavior is to set it to required-trusted. I set my client to all four values and these were my results:
optional-untrusted - works
optional-trusted - server error states Invalid CA
required-untrusted - works
required-trusted - server error states Invalid CAI used the same pfsense install as the CA for both server and client, and on each machine I could validate the cert via openssl manually, as I said - just syslog-ng threw an error.
SO, basically I set up syslog-ng to listen on UDP 5140 bound to localhost, then under Status->System Logs->Settings ticked the box for "Send log messages to remote log server," then set the first server to 127.0.0.1:5140. Going to Services->Syslog-ng and looking at the logs, you should see system logs showing up there now.
After that, I set up the objects below:
Here is my "destination" object on the client pfsense install:
name: tls_dest{ tcp("192.168.48.101" port(6514) tls( ca-dir("/var/syslog-ng/ca.d") key-file("/var/syslog-ng/cert.d/testpfsense.key") cert-file("/var/syslog-ng/cert.d/testpfsense.crt") peer-verify(required-untrusted)) ); };
Here is my "log" object on the client pfsense install:
name: tls_log{ source(_DEFAULT); destination(tls_dest); };
That's it. On the far side we specified the cert, key, and CA files needed in logstash. This is the contents of the file that configured the syslog listener on the logstash box:
input { tcp { mode => "server" port => "6514" ssl_enable => true ssl_cacert => "/opt/logstash/testing/192.168.49.69_CA.crt" ssl_cert => "/opt/logstash/testing/log01.crt" ssl_key => "/opt/logstash/testing/log01.key" ssl_verify => false } } output { stdout { codec => rubydebug } }
-
Really surprised this is not a supported feature in the gui!