[HowTo Guide] How to force users to install CA root certificate to gain access

sebastiannielsen · May 3, 2016, 5:37 AM

Hello.
First, I want to make it clear:

I do not endorse any malicious actions by using these instructions. Some people might find these instructions disturbing, as these instructions will allow someone to peek into SSL traffic, and allow a network administrator to bypass both HSTS and HPKP.

BUT, as with everything, SSL is a double-edged sword.
SSL is great to use by the good guys (normal users), to hide good data (credit card numbers, passwords, etc) from bad individuals (hackers, phishers and such).
SSL is also a great tool for the bad guys (hackers), to hide bad data (viruses, trojans, worms, exploits) from good individuals (network administrators, virus scanners).

And I think its every network administator's right to scan traffic, if you use someone's network, you use it according to THEIR rules.

In the much same way as you would never get a closed safe through the airport scanner without having to open it and show the contents.
You can refuse to open the safe, but then it not getting onboard the plane.
Same here, you can refuse to install the root certificate, but then theres no internet for you.

As with everything, its everyone's own responsibility to use these instructions only for non-malicious, good purposes!

Have found out that you can actually use HSTS to force end users (for example BYOD users) to install a CA root certificate (to use with squid SSL antivirus scanning) to gain access to network.
This is excellent if you operate a public hotspot, and want to use content filtering or antivirus scanning for hotspot guests and/or BYOD users.

This by using the Captive Portal function.

To get started, you need the following:
1: A real CA signed server certificate.
2: Squid SSL interception set up, with a fake certificate.
3: A server certificate signed by the said Squid certificate.

Now to get started.
First, you need to set up a "fake interface" in pfSense. This can be done by setting up a unused VLAN or whatever, to act as a interface. The point of this is to allow setup of a extra captive portal zone since a captive portal zone MUST have at least one interface assigned.

Now you create 2 captive portal zones:
One tied to the interface you actually want to force CA installation on. (This I call the "real" captive zone)
One tied to the fake interface. (This I call the "fake" zone)

Enable SSL on both. On the normal zone, you tie the real CA signed certificate. This to permit setup of the HSTS policy.
On the fake zone, you set up the fake server certificate tied to the same root as the Squid certificate.

Now, set up vouchers on the normal zone. A good idea here is to make 16 ticket bits but only 1 roll bit, and a bit lower on the checksum, to make the vouchers sufficently short for it to fit 65535 tickets in a file smaller than 1 MB.
Also create roll, 65535 tickets, and 1400 minutes, to ensure that the users get thrown out if they uninstall the CA certificate. (the authentication process will be transparent to the user if they still have the CA installed).

Download the vouchers. Open it in a text editor, and remove all the comments in start.
Then search&replace 2 quotes and a space ("" ) with a comma (,). This should make sure all vouchers appear one and one with a comma in-between.
Remove the last quote, space and newline from the end of the file.

Now you should have a file similiar to this:

voucher1,voucher2,voucher…,voucherN (but with 65535 vouchers)

Rename this file to a unguessable name, like a md5 hash or something.
Create a file called count.txt, containing only the digit "1". This "count.txt" will keep track of which is the next voucher to serve.

Upload both those files to the fake captiveportal zone (in the filemanager)
Take note of the names, and edit these in cpauth.php

Also make sure to check which port numbers both the fake captive portal uses and the real one, which can be checked by pressing the "view" button after having uploaded the PHP files. In cpauth.php, edit the action= line to point to the real captive portal zone, and also the real captive portal zone port number.

Upload cpauth.php as login page to the "fake" zone.

Upload the CA root certificates to the real captive portal zone, in both binary CER format, and also text PEM format.

Edit the cpauth2.php (the iframe) to point to the fake captive portal zone.
Upload this now as the real login page.
Upload it as the error page aswell, to facilitate a automatic "retry" action if the count.txt file gets out of sync.

Now to how the things works "behind the scenes".
The real login page will load, that will insert a HSTS policy in the browser. This will disable the possibility to temporarly accept a page with a certificate error.
The "fake" login page, is a page that contains a autosubmit form that will submit a voucher, but this page, is loaded with a IFRAME using the "fake" squid certificate, and will refuse to load according to the HSTS policy.
The user cannot either manually grab a voucher code either, as HSTS does not allow the user to "click through" a temporary exception.
The only way to get a voucher autosubmitted, (and automatically gain access to the internet), is to install the CA root.

Thus the real login page will attempt to load a form that contains a pre-filled voucher, that is loaded using the squid certificate, and thus, the user will automatically authenticate if they have installed the squid CA root, else they will not be able to authenticate, not even manually.

As the certificate is manually installed, this will also disable HPKP according to the standard (as the standard says HPKP verification SHOULD be disabled for user-installed certificates to permit the use of antivirus proxies), and also allow pages to load over HSTS but still be antivirus scanned and content scanned by Squid.

Hope you find this useful, and as I said, DON'T use it for malicious purposes!!!!
Feel free to edit cpauth.php and cpauth2.php to fit your installation, and possibly make the instructions in cpauth2.php a bit more prettier.

Here is a youtube video of the whole thing in works:
https://www.youtube.com/watch?v=5OXbfnG-hdg
Whitepaper: http://www.sebbe.eu/whitepaper.pdf
Hope you find it useful with a captive portal that requires the installation of a CA root certificate to gain access to the internet.

zoro_2009 · Apr 27, 2016, 8:41 PM

Thanks for the tuto !

But, does this hide the "Red alert" of fake certificate from the users browsers, is this doable all for free, and what about certificate pining ?

sebastiannielsen · May 3, 2016, 5:38 AM

1: Yes, once the user has installed the certificate, all the warnings from the web browser about invalid certificate will be gone (provided you have configured Squid SSL interception correctly).
HOWEVER, the initial certificate error (If you select to redirect HTTPS traffic to the captive portal), you can never get rid of.
It might be possible to fix it by configuring a custom "Access Denied" page in Squid that contains the authentication page, and then configure the captive portal to never redirect users.
This would remove all SSL warnings for users who have your Root cert installed, so only the very first connection by a very first user would get the certificate warning.

2: Yes its doable for free. You can for example use Startcom's free certificate, or WoSign 3-year free certificate, to provision the HSTS policy.

3: Pinning is disabled when a user-imported CA-root is used. Here is a example of pinning getting disabled due to the SSL interception: http://i.imgur.com/neOGCSQ.png

A cool thing I did, was to set up a EXE file with the following VB.NET code, that I also host on my captive portal login page, the EXE doesn't even need administrator rights, and does not present a UAC warning:

Hosting a EXE, CER and PEM, gives a easy way for all platform users to install the CA root.
EXE for windows, PEM for firefox users (regardless of platform, as they only have to check 3 boxes and press OK), and CER for most platforms. The CER installs extremely easy on iOS, as the user is directed through the process after clicking the CER link, and then just press "Install" 3 times, and then its done.

sebastiannielsen · May 3, 2016, 5:38 AM

Want to add this now.
I made the code above more generic, so it works with any root certificate.

Here is the compiled variants. Its signed with my Code signing certificate. Upload cert.exe and cert2.exe to your Captive Portal.

To use it, add the following to your Captive Portal Login page:


[Download setup EXE](captiveportal-cert.exe)
[Download setup EXE (admin)](captiveportal-cert2.exe)

You must of course ensure comes on separate lines.

Here is how it works:
It will fire off a request to http://www.msftncsi.com/ncsi.txt (Which is Microsofts Captive Portal Detection URL).
If it detects a captive portal, it will scrape the Captive Portal and see if theres any HTML comment start tag on a separate line.
If its there, it will check if there comes the string "RootCA" on a single line somewhere. At that point, anything after RootCA until --> will be readed in as certificate data.

After that, it will attempt to parse it, and then it will attempt to import the parsed CA certificate in the Windows system store.

I have also included recommended cpauth2.php and cpauth.php pages, and also the source code (for both .exe files) in .txt format

To make the application (if you want to compile yourself), use Visual Studio, with a .NET target of 2.0 or higher (take the lowest possible target for maximum compatibility), and create a ConsoleApplication.
After that is done, paste the source code in the Module1.vb file.
Then, in project properties, change Application type to Windows Forms application. This will remove the ugly black window that appears.

Derelict · Apr 29, 2016, 7:06 AM

Sigh.

cmb · May 3, 2016, 5:37 AM

This is a really terrible idea. Installing your own root CA on people's machines to access your captive portal is a massive security no-no. You're compromising the security of all your users.

But this goes to another level of awfulness by automatically installing any root certificates downloaded from a HTTP URL. So probably any user on your network can install their own trusted root CA on your users' machines via ARP poisoning or other means.

I edited the code out here because it's extremely dangerous, and should be considered malware.

Derelict · May 3, 2016, 5:46 AM

Because forcing random strangers to install your root certificate is like one of the worst ideas on the internet.

Even presenting users with a certificate error they have to click through (and possibly install as trusted in the process because, well, they're just users) is bad juju. What if their default home page is their bank? Now you're a trusted CA for their bank. And probably for 20 years because when you self-sign a cert you don't do it for 2.

sebastiannielsen · May 3, 2016, 2:21 PM

You probably didn't understand what this is for.

The idea is that it should be possible for the owner of a public hotspot or BYOD AP, to Anti-virus-scan user's SSL traffic for viruses.
And to do that, the user must install the root certificate, else the user will be presented with both HSTS and HPKP warnings when using squid to antivirus-filter the traffic.

Many exploits and drive-by downloads are hiding in SSL traffic today, thats why you need to uncover the SSL traffic to scan it for virus.

Yes, Derelict, if the user installs the CA, the squid server will become the CA of every website on the globe, regardless of which home page the user have set. BUT: That is only temporary while the user is online on the hotspot/network.
When the user disconnects and moves to a another network, normal CA's applies again, even if the CA root is still installed, the CA root is not being used, and no malicious access to encrypted traffic can be done as only the pfSense router that is behind the hotspot/network in question, knows the private key for the certificate.

About malicious use of the instructions (eg, someone putting up my captive portal idea in a network they are not authorized to do so, just to gain access to all SSL traffic for malicious purposes), that was exactly why I wrote the blue disclaimer box in the start post of this thread.

Gertjan · May 3, 2016, 1:55 PM

So, if I get this right:
"How to force users to install CA root certificate to gain access" boils down to a classic MITM !?!

This is actually just fine if you really want to find out what your kids, or any other person you have authority over, are doing on the net.

But doing this with 'people' (actually strangers to you) that are looking for a internet access, this won't work.
I guess, when you explain them what you are doing, your portal adventure stops right there …. by lack of any interested users.
Not telling them, well, in most civilized states and countries, its a one way ticket to jail ;)

If you really want to protect all these BYOD, just forbid https - accept http only, so users will know that the connection isn't 'save', that Uncle Sam (and his entire family) AND YOU are watching ..... ;)

sebastiannielsen · May 3, 2016, 2:27 PM

Gertjan: It do work. Actually, in most countries, but not all, encrypted VS nonencrypted traffic is treated the same, eg if its illegal to scan HTTPS traffic, then its illegal to scan HTTP traffic, and vice versa.
For example, here there is only 1 law regarding scanning of traffic (regardless of encrypted or not), and that law does only apply to licensed ISPs which operate public mobile or wired large networks (a Wifi hotspot does not count to the license ISP requirement here in sweden unless it cover a very large area so the network essentially is no longer "local").

This law applying to licensed ISPs and operators, actually means customers of prepaid mobile broadband, that uses a captive portal to "refill" the prepaid card, actually needs to consent to the captive portal first time they use the SIM card.

In all other cases, its legal to scan the traffic, as you own the equipment that the traffic pass. Logging however, regardless of cleartext or encrypted traffic, (access logs, virus logs etc) can be a legal problem in some cases (PII laws, or "PUL" as they are called here in sweden), but that is solved by having the user to consent to the logging by a agreement page.

And forbidding HTTPS can be a pretty large problem when almost every site is using HTTPS today.

But as I have clearly descirbed, this is not intended to "find out what people are doing on the internet", this what I wrote, is intended for:
**antivirus scanning and threat scan of encrypted traffic to block any threats from coming into the network.
Its designed to make the network more safe for all users to use, even if someone inside clicks on VideoPlayer.exe with UAC off.

And to ensure that safety, you must make tradeoffs. Either let encrypted traffic pass unscanned, and then you have a virus threat, or scan the traffic, but then you need to make a MITM on the user instead.

Theres really no choice.**

2chemlud · May 3, 2016, 2:33 PM

Just out of curiosity: Where is "here"?

Derelict · May 4, 2016, 2:22 AM

He mentions Sweden.

Yes, Derelict, if the user installs the CA, the squid server will become the CA of every website on the globe, regardless of which home page the user have set. BUT: That is only temporary while the user is online on the hotspot/network.

Or if they are coerced to go to a spoof site with a certificate made with your private key. Are you guarding your private key with the same measure of caution and diligence as, say, http://www.nordea.com/ or a CA like Symantec???

Bad all the way around. Your actions actually make it easier for your users to be scammed / infected.

sebastiannielsen · May 4, 2016, 4:32 AM

Derelict:
I understand your concern with leaking private keys.

But, first, the attacker must gain the private key.
Second, the attacker needs to find a victim that has my root cert installed.
Then the attacker needs to coerce that user into going into a spoof site.

Thats NOT a easy thing to do.

And actually, Let's encrypt, have made SSL even worse now. Now ANYONE, including www.n0rdea.com (note the zero) can get a free SSL certificate to their site, so the attackers don't even need to hunt after my CA private key. They just go to let's encrypt, and they could even manually create the certificate as the 3 month expiry would be enough to carry out a successful phishing/scam attack.

That, on top of the problems that SSL creates with encrypted malware, problems that a proxy cannot see the GET URL, which means phishing sites hosted at free webhosting is harder to block (without causing colleteral damage and blocking every website at that webhost), and much more. And not enough with that, now google uses SSL as a ranking indicator, which means sites that really don't need SSL, gets SSL to get a better score in google's algoritm, combined with vulnerable upload scripts, so people could upload malware to for example simple photo hosting sites and such.

Do you remember that time when SSL was created? The netscape times, with the good ol' 33k modems?
Then SSL was something that only banks could get due to strong encryption laws in USA. Driveby downloads was a very rare thing, malware mostly spread by diskettes.

That was the GOOD times. Then all could know that if they was on a site with the yellow little padlock in the bottom corner, then they was on a safe, vetted and good site. And antivirus proxies didn't need to scan these sites because they were known good sites, so antivirus proxies only needed to pass SSL unchanged, there was no risk in that.

After that, they loosened on the laws and allow 56bit encryption to be used by non-banks, why this thing with SGC (Server Gated Crypto) certificates was created, so banks could still use 128 bit encryption, but other sites needed 56bit. That was somewhat worser, but still, a certificate could only be issued to a organization that had a valid track record and vas trusted, so there was still a big roadblock for bad people to create SSL sites, so still, there was no need for SSL antivirus.

And then it went amok. Yes, really amok.
It started with free SSL CA's. So malicious people could go and get SSL certificates. But the good thing with these, was that these was valid for 1 year, and you couldn't get a new while you already had one issued, so if your phishing/malware/spam/hack domain got blocked, then no new certificate for you, you had to wait until that expired.

And today, it has gone totally out of control. Let's encrypt are causing almost 100% SSL deployment.
How should you be able to find viruses in traffic?

Someone needs to put some brakes on that SSL wheel that are spinning wayy to fast now!

Theres a reason most antivirus and threat companyes apply SSL filtering nowadays:
https://blog.avast.com/2015/05/25/explaining-avasts-https-scanning-feature/
https://www.bluecoat.com/security/security-archive/2012-06-18/growing-need-ssl-inspection
https://support.kaspersky.com/6851
http://nod32.helpmax.net/en/antivirus-protection/protocol-filtering/encrypted-ssl-communication/

I think you should reconsider whats really happened with SSL, and why SSL, in some cases can be bad, and why it can become neccessary for a network administrator, to uncover and scan SSL traffic.
Of course, such scanning should not be used for malicious purposes, and as for example with lock picks, that can be used for good purposes (for example if your key become broken), but also for malicious purposes (breaking and entering), its the same thing with SSL proxies.

When I set up network security, I prefer to put everything that is about security, in a central place. Eg, let pfSense be a thick, hard shell around the network, and almost no security inside.
And to accomplish that, you need to use features like SSL scanning, blocklists over malware and phishing and such things, and very tight firewall rules, so nothing slip through unscanned.

So I wonder, why are you getting so upset about scanning SSL-traffic? Its kinda like SSL was some holy grail that nobody should touch.

cmb · May 4, 2016, 5:15 AM

It's not the scanning HTTPS traffic that's the issue. It's that you're having users install something on their machine that fetches a HTTP URL and installs any CA certificate that's returned as a trusted root certificate. So anyone that can intercept traffic from their system can install a trusted root cert and completely compromise everything on their machine that uses PKI. HTTPS, certain VPN types, and more. It's a massive security hole.

If this was a corporate network and you owned all the machines, then installing your own root CA is acceptable if management deems it appropriate. You own the machines. You'd still be insane to install trusted CAs fetched via HTTP, but it's within your right to do whatever you want to machines you own. Doing it to someone else's machine is completely unacceptable even without the automatic fetch and install "feature".

What AV vendors do there isn't comparable to what you're doing. But pointing to them as a reference isn't an indication that it's a good idea, as many of them have made major screw ups in those implementations that leaves systems vulnerable. This is more comparable to how Dell massively screwed up.

I guarantee you aren't controlling your private key adequately to be considered a root CA. What happens when your private key is compromised? Can't revoke it without manually removing that cert from every system you've infected.

How are you supposed to find viruses in HTTPS traffic on a public network? You don't! You're effectively an ISP, your responsibility is to pass packets. Certain security mechanisms to protect systems are appropriate, like client isolation on your APs. But you should never, under any circumstances, change anything on a client system you don't own.

sebastiannielsen · May 4, 2016, 6:14 AM

But why did you then remove the cpauth.php and cpauth2.php from the first post then?
Those 2 files had nothing with automatic installation to do. The cpauth.php and cpauth2.php was made for manual certificate installation, eg where the user manually downloads the .cer/.pem file and install.

You say its unacceptable to fetch and install certificates over HTTP, even if done manually.
but, cpauth.php and cpauth2.php was, as I described in the first post, designed to be loaded over HTTPS. (else they won't function at all since HSTS is required for the whole function)

And as I wrote in the post, if the user is unconfortable with manually installing a certificate on their system, they can simply refuse to use the network. Nobody forces them to use that particular network.
And then its not I that change something on systems that I don't own, then its the user itself that does it.

And also as I described, this was not only for hotspot, but more commonly, BYOD, where a corporation provides a closed network to its employee's, where employee's can itself connect their own devices, and thus compliance can be required, and here it could be good with having the user install a cert to make it possible to scan incoming traffic for threats, but also scan outgoing traffic for Data Loss Prevention.

Derelict · May 4, 2016, 6:38 AM

~~To me, SSL connections between two parties I do not know are sacred. Nor do I have any desire to have a trusted certificate for which I hold the private key installed in random users' systems.

I also think developments like Let's Encrypt are a good thing, not a bad thing.

If I control one endpoint, like a private network used by my employees, then I have the authority to deploy an enterprise root certificate for use by my SSL proxy.

In my personal opinion, you should take a long look at whether you want to be providing Internet access to the general public based on what the Internet is today, crypto and all.~~
That's been sitting unsent in a compose window. Seems it has already been said. :)

Icook · May 4, 2016, 4:25 PM

@cmb:

It's not the scanning HTTPS traffic that's the issue. It's that you're having users install something on their machine that fetches a HTTP URL and installs any CA certificate that's returned as a trusted root certificate. So anyone that can intercept traffic from their system can install a trusted root cert and completely compromise everything on their machine that uses PKI. HTTPS, certain VPN types, and more. It's a massive security hole.

If this was a corporate network and you owned all the machines, then installing your own root CA is acceptable if management deems it appropriate. You own the machines. You'd still be insane to install trusted CAs fetched via HTTP, but it's within your right to do whatever you want to machines you own. Doing it to someone else's machine is completely unacceptable even without the automatic fetch and install "feature".

What AV vendors do there isn't comparable to what you're doing. But pointing to them as a reference isn't an indication that it's a good idea, as many of them have made major screw ups in those implementations that leaves systems vulnerable. This is more comparable to how Dell massively screwed up.

I guarantee you aren't controlling your private key adequately to be considered a root CA. What happens when your private key is compromised? Can't revoke it without manually removing that cert from every system you've infected.

How are you supposed to find viruses in HTTPS traffic on a public network? You don't! You're effectively an ISP, your responsibility is to pass packets. Certain security mechanisms to protect systems are appropriate, like client isolation on your APs. But you should never, under any circumstances, change anything on a client system you don't own.

Well if you have a better way to scan HTTPS traffic for malicious packets cmb then we are waiting to hear it. The fact that encrypted malware traffic can travel through networks to endpoints, and completely escape inspection, is also a massive security hole. It is not for you to make decision for other administrators about what is the greater threat, and you don't know the level of control they maintain over client systems.

In addition, he, nor any network administrator is "doing something to someone else's machine". Users cannot be forced to install trusted CAs; the users are doing the installing in this case if they want to access the corporate network etc.

Again, this method has many more protective uses than trying to find viruses on public networks or acting as an unauthorized snooping tool. And again, the clients are actively changing something on their system as a condition of gaining access to network resources.

You made a lot of good points about security flaws of https inspection in general, but there are going to be flaws, and trade-offs to any method. You may think you know better about a certain admin's network, but you don't, because you don't have access to the infrastructure design or the individual security concerns. Each network has its own caveats and use cases.

Thanks for posting this useful guide sebastiannielsen.

2chemlud · May 4, 2016, 6:24 PM

So you want the postal service to open each letter and each parcel, in some might be bombs. Or drugs. Or or or… I which kind of weird world do you life that you think you have to "protect" adult people from "malicious packages"?

Do you want to buy a beer even after you already had two or three and the barkeeper won't sell it to you? As an adult, what would you think?

Users are adults, they can take responsibility for what we do and NOBODY has to tell them what site to visit and which content to look at (as long as it's not illegal, but that's not YOUR piece of cake, but the police has to take care).

What is so hard to understand about that? Keep your fingers away from the data of your users. End of story. Otherwise move to China, North Korea or Iran and get a nice federal admin job with your kind of attitude. In Western, democratic countries your attitude is only acceptable for NSA, GCHQ and other thought crime organizations.

Icook · May 4, 2016, 8:35 PM

Really… with the straw man arguments?

The postal service already has ways of inspecting parcels for the things you mentioned, which don't include opening each and every letter and parcel, so my opinions on that matter are irrelevant, it's already taking place. That's totally off-topic as well.

Admins have to protect adult people from malicious packages all the time in enterprise environments where millions of dollars of equipment and intellectual property are at stake, and this guide helps lighten the load on administrators.

Do you not understand application layer packet inspection or what? Were you okay with it before the rise of https? Or have you always been against high-layer packet inspection? If that's the case, you haven't had any experience with high-security environments.

This isn't the electronic frontier foundation forum man, the OP provided a useful guide on how to configure pfsense for a special-use case.

2chemlud · May 5, 2016, 6:33 AM

Yeah, in "enterprise" environment you can life your paternalistic wet dreams.

I know some decent countries you go to jail for something like this MITM plot. And I'm proud of the respective legislation there.

Because I know that nerds like you are managing open WLAN I removed all wireless stuff completely from my computers. I don't want to get infected with this kind of malware just by the whims of some misanthrope "admins".

Hope you stay in your China-North Korea corner of the world and get happy there!

It's a shame to have such a thread in a security forum of an opensource firewall AT ALL.