Captive Portal not working on iOS devices only (DHCP 114)

[EDaleH]

@basharsaba said in Captive Portal not working on iOS devices only (DHCP 114):

I also compared every single field in the Certificate Details between my laptop (with no secure connection) and the VM Host (with a secure connection) and they are all identical. I couldn't find anything offside on my laptop.

Your certificate is "afamiahotelresort.com", that is what it should say in the name column. It simply will not work the way it is because it is currently looking for a second level domain as in: pfsense.portal.afamiahotelresort.com which you don't have.

Below is what a typical multi portal Captive Portal setup SAN certificate chain might look like for example:

DNS Name: domain.com
DNS Name: free.domain.com
DNS Name: day.domain.com
DNS Name: week.domain.com
DNS Name: month.domain.com
DNS Name: data.domain.com
DNS Name: somethingelse.domain.com
DNS Name: pfsense.domain.com
DNS Name: security.domain.com

Each would typically point to a different IP unless you use two different URLs to access the same page. domain.com and pfsense.domain.com could both point to the webconfigurator on 192.168.1.1. and security.domain.com could be a Network Video Recorder on 192.168.1.253 with free.domain.com on a Captive portal which is on VLan10 on 192.168.10.1 which is associated with interface OPT1 as a VLan under interfaces, the same for the rest, all on different VLans/IP ranges. Each will need DHCP, a firewall rule and DNS setup to function. If you are using VLans you will need Level 2 switches that support 802.1Q. The assumption in this thread has been that you are doing everything on the LAN, which is assumed to be on 192.168.1.1/24. Gertjan's responses assume you associated a VLan on 192.168.2.1 which is entirely valid but DIFFERENT. Your Captive Portal IP's DHCP Server must match be supplying an address in the same range(subnet) as the the DNS IP specified in the domain registration for subdomain.domain.com (pfsense.afamiahotelresort.com) in your case.

There will be an external Certificate Authority for domain.com (just the domain name, no subdomain, i.e. only one period before .com, no more periods). Under System, Certificate, Authorities you should see something like:

Acmecert: O=Internet Security
Research Group, CN=ISRG Root X1, C=US external

Under System, Certificate, Certificates you will have one entry for just that domain.com, no subdomains, i.e. only one period before .com

Name:
domain.com
CA: No
Server: Yes

Issuer:
Acmecert: O=Let's Encrypt, CN=R11, C=US CN=domain.com
Valid From: Thu, 18 Jul 2024 02:31:26 -0400
Valid Until: Wed, 16 Oct 2024 02:31:25 -0400

In Use:
webConfigurator Captive Portal DNS Resolver
Acme (1)

The only place you enter your subdomain name(s) is in the captive portal and DNS Host overrides, everything else is referring to the primary domain exclusively.

Below is a DHCP screenshot of a setup similar to what I described above:

If you are working with just the LAN, you will only see WAN and LAN on your setup. The remainder are all related to L2 VLans, the first being the interface and the remainder the VLans assigned to that interface. Each of them has their own Captive Portal and DHCP assisgned to them.

So you have only one LAN, one DHCP server on the LAN and thus a Captive Portal running on the LAN subnet 192.168.1.1/24 (pfSense default setup address assumed)

Bottom Line: My suggestion is that you have not registered your certificate correctly. That certificate should be afamiahotelresort.com

When I check your certificate two interesting things show up. It is not Trusted and it appears to have a wildcard SAN *.afamiahotelresort.com indicating you can add any subdomain in front of afamiahotelresort.com? This is the certificate foundation upon which you have built your configuration.

and

The certificate is not signed !! My Mcafee will not permit me to look at www.afamiahotelresort.com, it pops up a "Suspicious" error.

If you are working with an existing domain that serves your website and trying to add in a captive portal, you have many additional variables to deal with. I would suggest a new domain name that supports single level subdomains for free. Cloudflare is one example for $10 or so a year. Well worth the simplicity it adds. Let your web designer and ISP play with your web site, take control of your Captive Portal by using a dedicated domain/ACME setup, likely with a 90 day free let's encrypt certificate. I am confused though that you seem to be maintaining the afamiahotelresort.com certificate yourself (ACME/pfSense), did you by any chance override the original registration? It looks like a wildcard certificate and they are expensive.

[Gertjan]

Added to what @EDaleH said :

I rent a domain domain name, #######.hotel-fumel.net that I solely use for my pfSense LAN networks(s). (LAN, Portal, DMZ, etc.)

I also have the #######.hotel-fumel.fr domain that is used for the hotel's web site. This domain is accessible and usable "on the Internet" for the public.

My #######.hotel-fumel.net - when used on the Internet, points to nothing .....shuuut, don't tell anyone : my IPv4 ISP WAN IP. Which is actually a pfSense network ^^

Right now, I guess your host ? - or you, get a wild card certificate to be used on the public web server.
And locally, with pfSense, the acme.sh package, you also get a certificate using the same domain.
That's ... Ok, I guess.

My acme.sh setup :

which is the 'wild card' setup - the certificate I get back from Letsencrypt :

And now I can use this certificate for all GUI web interfaces on my LAN network for the devices that have a web GUI :
printer1.#######.hotel-fumel.net set with a DHCP MAC Lease
synology1.#######.hotel-fumel.net set with a DHCP MAC Lease
Airco.#######.hotel-fumel.net set with a DHCP MAC Lease
and also
pfSense.#######.hotel-fumel.net = 192.168.1.1 set in the pfSense GUI.
and
portal.#######.hotel-fumel.net = 192.168.2.1 set with a resolver host override.

VLAN : I'll use them as soon as I have some spare time, so I can safely abandon the Keep It Simple principle. It should work, though, when set up correctly (sic).

Extra - totally not related : : http://www.afamiah#telresort.com/welcome/ shouldn't really exist anymore. http is ok, and should do just one thing : redirecting to https:// Typically this is done as early as possible, in the web server config.
For me, this domain name (web site), the https part, had no certificate errors ....

[EDaleH]

@Gertjan
The status of the certificate changes with different ssl checkers. Only sslchecker.com fails it, ssl.com gives it a clean bill of health. It is curious that one checker fails it though.

The certificate is a wild card certificate.

• this is great because you can create any single level domain name you want without any further work. Despite that, read on.

The certificate authority does not exist in pfSense System, Certificates, Authorities?

•  - Is this a serious problem, I am unsure but I think so.  Normally ACME handles that when you create the certificate but your certificate already existed?  The fact you have no Authority listed in pfSense is undefined and potentially a fatal flaw.
  

The certificate name in pfSense Syste, Certificates, Certificates has the subdomain?

•  - This is wrong as now you need to create pfsense.pfsense.afamiahotelresort.com in DNS.  and I am unsure if second level domains are supported in pfSense?
  

• It needs to be afamiahotelresort.com. That will permit the Dynamic DNS subdomain pfsense.afamiahotelresort.com to work with captive portal and DNS Resolver. Again though, read on.

You still need to have DNS resolve for pfsense.afamiahotelresort.com to the subnet you put the Captive Portal on -> LAN?, (assumed to be 192.168.1.1). This appears to be done.

Given all the above, I suggest a clean start that gives you complete control over the Captive Portal:

Advice: Buy a new domain that supports 1st level subdomains for your pfSense and Captive Portal. Be careful, most issuers do not support subdomains or charge a large fee for them. Cloudflare for one does not charge extra, I am sure there are others out there.

• Add DNS subdomains like portal, pfsense, and no subdomain at all.
• point pfsense at the webconfig address 192.168.1.1
• point portal at the Captive Portal DHCP Subnet, likely same 192.168.1.1.
• Setup ACME from scratch for the new certificate starting with Account Key, then configure the ACME certificate including all 1st level subdomain names and domain name without any subdomain.
• Setup DNS Resolver and Captive Portal for the new domain name and subdomains. Don't forget the firewall rule to allow internet access.
• test from the laptop.
• configure RFC8910 DHCP 114 support for the new domain name and test devices that support it.

[Bisho]

Hello @EDaleH and @Gertjan,

I went through all your notes and advice, I have no problem getting a new/dedicated domain from Cloudflare, but I feel I'm very close to achieving what I want using the existing domain. Let Cloudflare be the last resort.

Out of curiosity, I went to a hotel nearby today where my laptop and Android device always get a secure connection and checked their Captive Portal certificate details, then applied what you guys suggested and compared it to this hotel's Captive Portal certificate that uses 2 SANs. One is the domain and one is the wildcard domain. Their URL also resolves to an internal IP. Here is a screenshot.

And here is mine

Anyway, I started from scratch, but when I was doing the Acme Certificate options, I didn't know what to put in the Server, Key, Key Name, and Algorithm. I appreciate your advice.
By the way. I used the LE's Production ACME V2 to register the ACME account key.

Also, I got an error when I tried to issue/renew the certificate. Is there anything I need to do from the domain/registrar's side?

I also used afamiahotelresort.com for the Certificate's Common Name.

[Gertjan]

@basharsaba

You saw the difference ?

and

Btw :

update fail.
If you want more details : see the "more details".

As you can see, the "CA" is also available. and afaik 'acme' would have stored it in the CA list if it is missing.
You know where it is, you could place it in there manually, if needed.

[Bisho]

@Gertjan,

Yes I saw the difference, Their Certification Hierarchy starts with a Certificate Authority which I'm missing.

In my particular situation, where am I supposed to create the Certificate Authority? Under System/Certificate/Authorities? If yes, Internal, intermediate, or import?

Or under ACME Certificates? What about the Key fields?

Here is the acme_issuecert.log.txt It's tooooooooo long and confusing. I couldn't spot the exact issue.

[Gertjan]

@basharsaba

I saw the log.

is wrong.
(stopped looking for more)

The NS server here is the master domain name server.
Not optional or something.
The 'nsupdate' has to have an IP - this one :

as that IP is my 'master domain name server' where the zone of the domain resides.

Your could be

[24.03-RELEASE][root@pfSense.bhf.tld]/root: dig afamiahotelresort.com NS +short
ns203.canadianwebhosting.com.
ns204.canadianwebhosting.com.
[24.03-RELEASE][root@pfSense.bhf.tld]/root: dig ns204.canadianwebhosting.com A +short
104.36.151.152
[24.03-RELEASE][root@pfSense.bhf.tld]/root: dig ns203.canadianwebhosting.com A +short
104.36.151.151

This IP, the secret key and the algorithm type are made available to you be the domain registrar.

edit :

On the domain name server side, during the certificate renewal I showed above, I found this in the 'bind' logs :

07-Aug-2024 07:53:10.411 update: client @0x7ff5325d5070 82.127.26.108#55966/key update: updating zone '#####-hotel-fumel.net/IN': adding an RR at '_acme-challenge.#####-hotel-fumel.net' TXT "JLJQnIUFDoiQKHfN6iyFvHIuMMhF_-h7I5-mWcaZGX4"
07-Aug-2024 07:53:10.719 update-security: client @0x7ff5325d5070 82.127.26.108#65112/key update: signer "update" approved
07-Aug-2024 07:53:10.719 update: client @0x7ff5325d5070 82.127.26.108#65112/key update: updating zone '#####-hotel-fumel.net/IN': adding an RR at '_acme-challenge.#####-hotel-fumel.net' TXT "Rmcf8iJ7-R7joIpa_hEMgfYBm9jpt_YuSP3eyBjSfq4"
07-Aug-2024 07:55:17.432 update-security: client @0x7ff5325d5070 82.127.26.108#57576/key update: signer "update" approved
07-Aug-2024 07:55:17.432 update: client @0x7ff5325d5070 82.127.26.108#57576/key update: updating zone '#####-hotel-fumel.net/IN': deleting rrset at '_acme-challenge.#####-hotel-fumel.net' TXT
07-Aug-2024 07:55:17.596 update-security: client @0x7ff528513520 82.127.26.108#55196/key update: signer "update" approved
07-Aug-2024 07:55:17.596 update: client @0x7ff528513520 82.127.26.108#55196/key update: updating zone '#####-hotel-fumel.net/IN': deleting rrset at '_acme-challenge.#####-hotel-fumel.net' TXT
07-Aug-2024 07:55:38.455 update-security: client @0x7ff532233980 2a01:cb19:907:a600:92ec:77ff:fe29:392a#52512/key update: signer "update" approved

You can see that at 07h53:10 the two TXT fields are added under the sub domain "_acme-challenge".
Then there was the DNSsleep of two minutes.
The the two TXT fields are deleted / cleaned up.

What also happens : as soon as the master zone is updated, the master DNS domain names servers signals the DNS slaves. As shown above, you have a master, and a slave.
These slave(s), when they see fit ( !! ) will contact the master and ask for a zone sync.
Only when is this done, Letsencrypt should start validating your domain, which is nothing more as checking if under _acme-challenge.afamiahotelresort.com exists, if there is a TXT field and if it contains the 'secret random' text string value that Letenscrypt gave to acme when it started.
Because I asked for a wild card cert, there are two TXT fields.
That's all that is needed to proof that YOU 'own' your domain.
As no one else could do this.

I can see this log info as from the other side as I'm doing my own DNS stuff for my own domains ;: I run my own DNS servers (advise : don't do what I do).

[Bisho]

@Gertjan

A few questions:

1. In your SAN Table, you created 2 domain names. One is the main domain and one is the wildcard one. Does the zone need to match the domain name? I mean one zone is the main domain and the other one is the wildcard one?

2. Is the key for the SAN table the same SSL/TLS Certificate's Private Key that I have imported to Certificates, or am I supposed to look for another key?

3. I saw that you entered a Key Name. It's optional, shall I leave it blank or enter the domain name?

4. I couldn't find the Algorithm from the Registrar, so I chose the most common one SHA256

[EDaleH]

@basharsaba & @Gertjan
I am not replying to the questions asked but rather clarifying a few points and making observations:

First, Let's Encrypt certificates through ACME support wildcard subdomains for the SAN list. I was not actually aware of that until now. It does not change anything, it just makes subdomain setup in the DNS easier and somewhat less secure if others have access to the DNS server setup. Specifying subdomains individually works just fine and you have control over which work. Your call when setting up your Let's Encrypt certificate in ACME.

Next, @Gertjan pointed out the certificate authorities associated with ACME, there appear to be 3 ACME related authorities. Of note is that the "external" one is NOT the one used for the certificate managed by the ACME package. The bottom one in Gertjan's screen capture is:

You can see that there is "1" certificate associated with it. This certificate is the one you will use for the Captive Portal https setup. The certificate is associated with the domain name, no subdomain is in play at all in the System, Certificates setup.
Now we look at @basharsaba 's original Certificate.

The two things of note here are

• As stated previously, the Certificate includes the subdomain name which it should NOT include. It simply won't work as this is the security certificate which already includes the ability to handle wildcard subdomains in this case. It will force a second level domain resolution that is not supported.
• The issuer is listed as "external". This is wrong, it should be the Certificate Authority that is listed last as shown in @Gertjan 's image above but the exact syntax, in particular the CN=R?? part may change. In my opinion, this is why the original configuration was not working, the Certificate authority was wrong and thus failing for Captive Portal. This is why the Certificate Hierarchy is not correct, it simply doesn't exist. In the example below, the red should say "afamiahotelresort.com". The yellow will likely be different.
  

Lastly, the DNS setup points to the address of the captive portal, but it matters not how it came to do so, DNS server, Dynamic DNS or DNS Resolver Host Override. It is simply important that it resolve to the correct address pool that matches the DHCP associated with the Captive Portal. It has nothing to do with the certificate other than enabling that field in the SSL/TLS verification process to match on subdomain and Captive Portal to match on IP address.

Once the above configuration is achieved, I believe you will be operational.

[EDaleH]

@Gertjan @basharsaba

Sorry, I forgot one more suggestion. If you go to Services, ACME, General Settings, you can turn on the saving of all of your certificates to /conf/acme, including the authority. That way you can check the authority in System, Certificates, for both Authorities and Certificates:

I used the Authority certificate to create an authority for a second pfSense server on our network (I wanted https for webconfig on the second pfSense unit) and then pasted in the certificate using that authority, associating it with webconfig. I update it manually every time ACME renews the certificate itself which is a pain but of importance here, I re-created the configuration without configuring ACME itself and successfully enabled https. Thus Authority + Certificate is all that is needed to make https work on a subdomain.

[Bisho]

Hello @EDaleH and @Gertjan, I have 50% good news.

So after numerous attempts of playing with the settings and troubleshooting the errors I was never able to get ACME to issue a certificate. I believe this is because we are not using the DNS from NameSilo (Registrar) but instead, we are using the DNS from the Canadian Web Hosting company. There wasn't enough room to play with the DNS settings or let's say the required parameters for the Domain SAN list in pfSense (Key, Token, Account, or Zone ID) were not available.

Long story short, I took your advice and purchased a brand new domain on Cloudflare (www.afamiahotelresort.net). As pfSesne already has DNS-CloudFlare listed, I was able to fill in the required fields and get the certificate issued.

I also enabled the 2 options (Cron Entry and Write Certificates).

The Certificate Authority and the Certificate were created automatically for the Wildcard domain.

You previously said I should have a subdomain.domain.tld that will be used for the Captive Portal. So I went to CloudFlare and created A record for pfsense.afamiahotelresort.net that is resolving to 192.168.1.1

I got an error when the Proxied option was enabled, so I had to disable it and the record was created as DNS only. Is this ok?

I went after that and changed all the other settings according to the new URL (Domain, DNS Server Settings (IP and Hostname), Captive Portal HTTPS Options, DHCP 114, DNS Resolver Host Overrides...etc but unfortunately, the connection to the Captive Portal page was still not secure.

Is it because I used the ACME Staging, not Production? I have to wait about a week to be able to re-issue a new Production Certificate because I have had a lot of failing ones.

Here are the new settings screenshots. Let me know why you think it's still not working.
DHCP 114 String is now "https://pfsense.afamiahotelresort.net:8003/rfc8910.php?zone=cpzone"

[EDaleH]

@basharsaba said in Captive Portal not working on iOS devices only (DHCP 114):

Here are the new settings screenshots. Let me know why you think it's still not working.
DHCP 114 String is now "https://pfsense.afamiahotelresort.net:8003/rfc8910.php?zone=cpzone"

As far as the general settings are concerned, there is no need for them to align with the domain you register. I don't think there is an issue though, it just might be a source of confusion when your pfSense firewall has the same name as your captive portal.

The DHCP114 string looks fine based on your previous setup info.

You need to move on from staging to the production environment for your certificate to work.

On Cloudflare you should have at least two records, afamiahotelresort.net pointed at your pfSense webconfigurator, and pfsense.afamiahotelresort.net pointed at the subnet of your captive portal. Yes, the proxy has to be off. Your SANs suggest you have that. For a "normal" setup using vlans, they would not be on the same subnet but it appears from prior communication from you that your captive portal is also on the default 192.168.1.1 subnet. That means the same DHCP server gives out addresses for your captive portal as for your LAN. It also means that all your visitors can launch the pfsense webconfigurator and only need to guess the user/pwrd to get in but as long as you are OK with that.....

For the DNS Resolver, I would put both SANs in there.

I would enable production at this stage and only then test it. It looks good from here.

[Gertjan]

@basharsaba said in Captive Portal not working on iOS devices only (DHCP 114):

You previously said I should have a subdomain.domain.tld that will be used for the Captive Portal. So I went to CloudFlare and created A record for pfsense.afamiahotelresort.net that is resolving to 192.168.1.1

I got an error when the Proxied option was enabled, so I had to disable it and the record was created as DNS only. Is this ok?

Perfectly ok.
You already asked and got a wild card certificate.
So "pfsense.afamiahotelresort.net" or "portal.afamiahotelresort.net" or "whatever.afamiahotelresort.net" is already accounted for.

Um' not sure what you are doing here :

Where is this ?

On pfSense, there where "192.168.1.1" has a meaning (192.168.1.1 = RFC1918 = only locally and never 'on the Internet').
You should have :

and your good.

@basharsaba said in Captive Portal not working on iOS devices only (DHCP 114):

Is it because I used the ACME Staging, not Production?

"Staging" is for testing. Useful as you can test the DNS API as long as you want.
Maybe it's time to actually read the Letsencrypt manuals, guidelines etc ??

The "production" isn't good for testing, as you can only fail a couple of time, and then you have to wait for days to get a new cert.

[Bisho]

Hello @EDaleH and @Gertjan,

I had to wait all this time until LE finally allowed me to issue a Production Certificate.

I want to update you that all the devices of all kinds are finally working without issues and getting a secure connection with a valid certificate. Having a dedicated domain on Cloudflare was a very wise idea.

I wouldn't be able to achieve that without your advice and assistance. You guys rock.

One last question. I have enabled Cron Entry. From your experience, will it really renew the certificate automatically after 90 days? Or am I still supposed to click the blue Issue/Renew button?

Apparently, the "Certificate renewal after" field doesn't matter at all because regardless of how many days I set, it's still 90 days as per LE.

[EDaleH]

@basharsaba

ACME will renew every 60 days and it works well with the default settings.

Congrats on your success.

[Gertjan]

@basharsaba said in Captive Portal not working on iOS devices only (DHCP 114):

From your experience, will it really renew the certificate automatically after 90 days? Or am I still supposed to click the blue Issue/Renew button?

Check the cron entry :

To see it, install; the pfSense Cron package first.

The need-to-renew-test runs every day, mine at : 03h16.
Now check the system log : to no surprise :

Don't use '90' as a certificate renewal, but something between 30 and 60.
That leaves you with 60 to 30 days to resolve an issue IF something went wrong.
"90" will leave you with no spare time to resolve the issue.

Btw : I'm using acme for years now, and it never failed. if it did, it was because the admin (me) was messing up again.
That said, free services or even when you pay for them : don't auto trust them, think about putting in place automated checks like this (very old munin based monitoring system). You can see that I check 2 domains there, and they are renewed after 30 days, and the other 60 days. Is like clock work.

[Gertjan]

@EDaleH

I've noticed something.

The scenario :

My PORTAL network :

192.168.2.1/24 - DHCP pool 192.168.6 -> 192.168.2.254
192.168.2.2, 3, 4, 5 = 4 APs with a static IP setup, DNS and gateway set to 192.168.2.1

Normally, when I connect with my iPhone, I always get the same IPv4 : 192.168.1.6 because I created a DHCP MAC lease for my iPhone.

Now, when I add my IPv4 192.168.2.6 to the "allowed IP address" list :

I killed all Portal sessions, relaoded the Portal on pfSEnse, and then reconnected my iPhone.
I still got the login page ....

It took me the better half of this morning that .... this is totally 'ok' as my iPhone, as soon as it receives a DHCP lease, it knows right away it has to go to

= "https://portal.xxx.net:8003/rfc8910.php?zone=cpzone1"

Before, when an IP was white listed like this, the portal became transparent for the device using that IP.

So, not really an issue, I was actually looking for a 'problem' that didn't exist.

Just : keep in mind that "rfc8910" method will not allow the bypass "allowed IP address" method.

edit : MAC list : probably same thing, didn't test yet.

[EDaleH]

Did you "forget" the iPhone "My Network"? I have noticed when testing RFC8910, DHCP 114 that it is/can be "sticky". As I switched between Kea and ISC, I had to remember to forget the old network on the iPhone, otherwise it appeared to be working under Kea when it actually had saved the login info from when it connected under ISC and received a 114 option, even though it got a different IP address. You can also switch "auto login" off and see if it has internet connectivity as that will prevent the captive portal page from loading, even if received via DHCP 114.

[Gertjan]

@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):

Did you "forget" the iPhone "My Network"?

Yep, did that, as part of the 'clean slate' approach.

@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):

I have noticed when testing RFC8910, DHCP 114 that it is/can be "sticky"

I have that impression also. Probably some iPhone side coding doing 'its' thing.

@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):

As I switched between Kea and ISC

I saw your discussion with marcos on Github about the upcoming kea implementation.
Progress is looking fine.

@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):

You can also switch "auto login"

I'll experiment with that one

[Gertjan]

For the "rfc8910.php" file :

if (is_array($cpcfg['allowedip'])) {
	foreach ($cpcfg['allowedip'] as $ipent) {
		if ($ipent['ip'] == $clientip) {
			if ($ipent['dir'] != 'to') {
				// Bail out, 'clientip' is part of the 'allowedip' list
				ob_flush();
				return;
			}
		}
	}
}

If there is an array with "allowedips"
For each IP in the array,
If the IP of the portal c;lient matches an IP in the list,
And if the direction in not 'to' (thus 'both' or 'from'),
Then flush and return.

This will take of he devices that are on the "allowed IP" list, they won't get login page anymore.

Take note : this works for 'English' portals. Afaik, the text 'from' 'to' and 'both' might be different for another languages.