Patching/Upgrading OpenSSL

[doktornotor]

@gertk:

What about windows OpenVPN client programs? I have to upgrade them too or not?

https://forum.pfsense.org/index.php?topic=74948.0

[Atomicslave]

Just want to say a big Thanks to all the Devs who put time in on this, upgraded my home firewall last night and my work one this morning.

Thanks again guys Awesome Job!

[Ecnerwal]

Happily my "system where VPN runs and security matters" is still on 2.0.3, as I've I've had continuing nags with 2.1 where I did try it that made me not upgrade that one since 2.0.3 is nice and stable and I can put up with complaints from the users of the other system more easily as I try to work the 2.1 issues out there (or wait for them to be worked out, more likely.) Hopefully 2.1.2 will be better than 2.1, and if not, I guess there's always heading back to 2.0.3 on both systems.

[adam65535]

@Jason:

You're using CARP on all interfaces?  If so, the downtime will be basically zero.

Upgrade the backup first.  Once it's back up and running disable CARP on the first box to force a failover.  Once you're sure things are working on the backup, upgrade the primary.  Once the upgrade is done everyone will fail back to the primary automatically.

Unless you are dependent on packages being installed like some routing daemons, squid, etc.  The current disable carp feature on the carp status page does not survive reboots.  The primary will take over immediately after the reboot before those get re-installed fully so there will be some downtime in that case.  If you don't depend on any packages then it should be 0 downtime.

Someone has submitted a feature in 2.2 version that allows you to put a carp member in maintenance mode so that the member you put into maintenance mode will not take over when it boots up unless carp detects that the other member is down.

[NOYB]

See several OpenVPN connections from an unauthorized location in the firewall logs this morning.
Is this someone trying to exploit the Heartbleed bug?

Name:    hn.kd.ny.adsl
Address:  182.118.48.87

pfSense OpenVPN Log

Apr 11 07:01:20 openvpn[9425]: TCP connection established with [AF_INET]182.118.48.87:53956 
Apr 11 07:01:20 openvpn[9425]: 182.118.48.87:53956 WARNING: Bad encapsulated packet length from peer (5635), which must be > 0 and <= 1560 -- please ensure that --tun-mtu or --link-mtu is equal on both peers -- this condition could also indicate a possible active attack on the TCP link -- [Attempting restart...] 
Apr 11 07:01:20 openvpn[9425]: 182.118.48.87:53956 Connection reset, restarting [0] 

[BBcan177]

http://kb.bothunter.net/ipInfo/nowait.php?IP=182.118.48.87
http://www.ipvoid.com/scan/182.118.48.87/
http://www.senderbase.org/lookup/ip/?search_string=182.118.48.87
https://www.iblocklist.com/search.php?string=182.118.48.87

If you use pfBlocker you could block known malicious ip addresses.

[adam65535]

@NOYB:

See several OpenVPN connections from an unauthorized location in the firewall logs this morning.
Is this someone trying to exploit the Heartbleed bug?

Name:    hn.kd.ny.adsl
Address:  182.118.48.87

pfSense OpenVPN Log

Apr 11 07:01:20 openvpn[9425]: TCP connection established with [AF_INET]182.118.48.87:53956 
Apr 11 07:01:20 openvpn[9425]: 182.118.48.87:53956 WARNING: Bad encapsulated packet length from peer (5635), which must be > 0 and <= 1560 -- please ensure that --tun-mtu or --link-mtu is equal on both peers -- this condition could also indicate a possible active attack on the TCP link -- [Attempting restart...] 
Apr 11 07:01:20 openvpn[9425]: 182.118.48.87:53956 Connection reset, restarting [0] 

You get that error any time someone even just connects to the openvpn TCP port with a regular browser.  It is not an indication specifically of a heartbeat attack.  Certainly someone is scanning for SSL ports though for whatever reason.

EDIT: changed openssl to openvpn

[Doktor Jones]

Just to double-check – if I install pfSense 2.1.2, my OpenVPN server is safely patched correct?

What command could I run to check the version of the libraries (e.g. /usr/local/lib/libssl.so) to ensure it's a patched version?

Thanks!

[doktornotor]

Just in case people still think their private keys are safe:

Can You Get Private SSL Keys Using Heartbleed?
The Results of the CloudFlare Challenge

@CloudFlare:

While it takes effort, it is possible to extract private SSL keys. The challenge was solved by Software Engineer Fedor Indutny and Ilkka Mattila at NCSC-FL roughly 9 hours after the challenge was first published. Fedor sent 2.5 million requests over the course of the day and Ilkka sent around 100K requests. Our recommendation based on this finding is that everyone reissue and revoke their private keys. CloudFlare has accelerated this effort on behalf of the customers whose SSL keys we manage.

[stephenw10]

@Doktor:

What command could I run to check the version of the libraries (e.g. /usr/local/lib/libssl.so) to ensure it's a patched version?

To check the OpenSSL version:

[2.1.2-RELEASE][root@pfsense.fire.box]/root(1): /usr/local/bin/openssl version
OpenSSL 1.0.1g 7 Apr 2014

To check the libraries use the strings command and grep for Openssl. E.g.
@https://redmine.pfsense.org/issues/3588:

[2.1.1-RELEASE][xxx@xxx]/root(1): strings /usr/pbi/haproxy-devel-amd64/lib/libssl.so.8 | grep OpenSSL
OpenSSLDie
SSLv2 part of OpenSSL 1.0.1g 7 Apr 2014
SSLv3 part of OpenSSL 1.0.1g 7 Apr 2014
TLSv1 part of OpenSSL 1.0.1g 7 Apr 2014
DTLSv1 part of OpenSSL 1.0.1g 7 Apr 2014
OpenSSL 1.0.1g 7 Apr 2014

However I don't appear to have the strings command in my Nano install here at home.

Steve

Edit: You can add strings as part of binutils using pkg_add but it's quite big.

[2.1.2-RELEASE][root@pfsense.fire.box]/root(11): strings /usr/local/lib/libssl.so | grep OpenSSL
OpenSSLDie
SSLv2 part of OpenSSL 1.0.1g 7 Apr 2014
SSLv3 part of OpenSSL 1.0.1g 7 Apr 2014
TLSv1 part of OpenSSL 1.0.1g 7 Apr 2014
DTLSv1 part of OpenSSL 1.0.1g 7 Apr 2014
OpenSSL 1.0.1g 7 Apr 2014

[doktornotor]

@stephenw10:

However I don't appear to have the strings command in my Nano install here at home.

Pretty damn sure I used to have strings command on both nano and full, right before 2.1.2 upgrade. Now I do not have it anywhere. Perhaps someone could kindly put that back to base OS install and stop removing useful stuff.

[NOYB]

Strings is still available in my 2.1.2 full x86 install.  Though I've not done a clean install for quite some time.  Just upgrades.

[stephenw10]

Whilst it's useful in this situation the inclusion of binutils could be mostly seen as a security risk than anything else. I can understand leaving it out. You can always add it back as I did. Can strings be added independently?

Steve

[Guest]

@dgcom:

I perfectly understand implications of this particular issue, and yes - it is not just a matter of replacing openssl executable… What I am saying is that recompiling everything is not very efficient. But, I guess, you know your product :)

…
@dgcom:

shouldn't build system be smart enough to recompile only if dependencies changed?

the build system is, but the system is packaged in a way that is a forklift upgrade on every update.

We're exploring ways to update differently (such that updates are more like freebsd-update), but even here
there are impacts that you might not imagine.

PBIs are dead after 2.2, btw.

@dgcom:

I, personally, do not run anything, based on recent versions of openssl - except pfSense.

this is probably mistake at this point.