Captive portal + DNS redirect

[regexaurus]

I have a simple captive portal (no auth / click through) set up on a guest vlan, that is generally working as expected. I tried setting up DNS redirect (NAT port forward and firewall rules) on the same vlan / interface, but that partially broke the captive portal. I could still manually access http://captive_portal_ip:8002?zone=captiveportalzone and click to "log in", but I couldn't access any web pages. And yes, under captive portal status, I could see I was logged in. Interestingly, test DNS queries (PowerShell: Resolve-DnsName) failed before logging in (connected to captive portal vlan, but not logged in), but succeeded after logging in. I'm not sure why I was still unable to access any web pages...
I found another post/discussion about this scenario (captive portal + DNS redirect) which mentions workarounds involving adding rules in /etc/inc/captiveportal.inc. The last reply to that discussion includes rules for this setup on 2.4.x. I'm on 2.7.2, and it seems captiveportal.inc has changed substantially enough that those rules won't work. Besides, I'm not thrilled with the idea of modifications that need to be documented and possibly reworked in the event of future pfSense upgrades. Is there a way to get captive portal + DNS redirect working in 2.7.2? I'm open to modifying captiveportal.inc if needed, but prefer a web interface approach if possible, and I don't mean Diagnostics > Edit File...
Without DNS redirect in play, captive portal generally works for me. It works fine on an iPhone I tested. It works OK on a Windows 11 laptop I tested, if Firefox or Edge are the default browser. If Chrome is the default browser, when I connect to the guest (captive portal) vlan, Chrome tries to open http://www.msftconnecttest.com/redirect. If I wait several minutes, eventually it opens the captive portal login page (with a redirect URL of http://www.msftconnecttest.com/redirect), and I can log in. It seems this may be related to "active probing". Any suggestions for this?

Thank you!

[Gertjan]

@regexaurus

This DNS redirect : these are the first 4 firewall rules of my captive portal :

The very first rule isn't related, it's a pass rule so my Unif portal APs can contact the controller on my LAN.
Btw : my captive portal uses 192.168.2.1/24

Rule 2 and 3 are the DNS redirect rules. I've created these years (a decade ?) ago, as instructed by the pfSense DNS redirect documentation page.

Rule 4 block all the other DNS requests."

Btw : when you use the captive portal, all bets are off : you're not allowed to break DNS anymore.
The solution is : use the revolver's default (change nothing) settings, and the portal will works just fine.

@regexaurus said in Captive portal + DNS redirect:

I found another post/discussion about this scenario

Yeah, maybe, that was ages ago, when pfSense was still using the ipfw firewall, not the pf firewall.

@regexaurus said in Captive portal + DNS redirect:

Chrome tries to open http://www.msftconnecttest.com/redirect.

Noop.
That http (not https !!) request is executed by the, probably a Microsoft Windows device.
It's executed as soon as the DHCP client did it's work.
This "none browser" request is send by the OS to test if an "Internet connection" is available.
Every OS uses its own test URL, for example, Apple OS devices uses this http test request : http://captive.apple.com/hotspot-detect.html
Click on it,, and see what happens ^^
If the OS got a "page" back that shows :

then it knows that is has a direct working Internet connection, no further actions are needed.
If something else comes back, like ... the pfSense captive portal login page, then the OS knows that the device might be behind a captive portal. The OS will signal the user with a message that user interaction is probably needed, or it will launch a default browser with the same http://captive.apple.com/hotspot-detect.html, which will get, again, intercepted, and the captive login page will be shown.

Because URLs are used, DNS needed to work

A device that uses a captive portal should (must) use DHCP.
So the device will receive 192.168.2.1 as a DNS (my case).
If the device is stupid enough to use something else, like 1.1.1.1 or 8.8.8.8, then yeah, that is going to fail, as these destination are forbidden by the portal.
That's where the DNS redirect trick comes in : it redirect port 53 UDP or TCP connections to the local 127.0.0.1 on pfSense, where the pfSense resolver unbound is also listening.
Don't presume : you've checked that ? :

[25.03-BETA][root@pfSense.bhf.tld]/root: sockstat -4 | grep ':53 '
unbound  unbound    27811 5   udp4   *:53                  *:*
unbound  unbound    27811 6   tcp4   *:53                  *:*

so my unbound listens on 'everything' which included 127.0.0.1 === localhost using UDP and TCP. The portal is IPv4 only.

If the device uses DNS tricks like DoH or DoT, then you can't do anything about it. They, the device's owner has chosen the "max security" so portal usage is excluded. Not only your pfSense captive portal, but any other captive portal available on planet earth. Si be it.

Tip : be aware that the device to connect to the portal remembers teh Wifi (network, whatever) settings. It doesn't only remember that a password is use, but also if a portal is used.
So, when you activate the portal on a pfSense network, remove/reset the SSID wifi config from your device and the reconnect again.

Tip : install 2.8.0 Beta right away.
I'm using the latest 24.03 Beta 2 for a hotel, and have my hotel clients have no issues what so ever. 99 % connects just fine.

Tip : in the portal forum, have a look at the forum threads that mention "rfc8910". That's the more modern way to handle a portal. No file edits are needed, but you have to create a PHP file (examples are given) and you need the latest kea server functionalities so you can add a DHCP server option so your clients (for those who are RFC8910, many are these days) can connect even "faster and better". I'm using this method for nearly a year now, and works flawlessly.

[regexaurus]

Thank you, @Gertjan

@Gertjan said in Captive portal + DNS redirect:

@regexaurus said in Captive portal + DNS redirect:

Chrome tries to open http://www.msftconnecttest.com/redirect.

Noop.
That http (not https !!) request is executed by the, probably a Microsoft Windows device.

You say I say
I understand that it's the OS calling for that URL. On my test (Windows 11) device, I only see that happen when Chrome is the default browser. If Edge or Firefox are the default browser, then I never see that URL in the browser address bar, and the browser quickly shows the captive portal login page.

If the device uses DNS tricks like DoH or DoT, then you can't do anything about it. They, the device's owner has chosen the "max security" so portal usage is excluded. Not only your pfSense captive portal, but any other captive portal available on planet earth. Si be it.

Right. I'm not trying to accommodate those scenarios. I don't think someone so security minded should be trying to connect a potentially-sensitive device to an open wireless network anyway. Even so, I do make at least a minimal effort to protect guests from each other (client/device isolation).

So, when you activate the portal on a pfSense network, remove/reset the SSID wifi config from your device and the reconnect again.

Yes, I frequently "forget" my guest wifi from known networks, as I'm testing and making captive portal configuration changes.

Tip : install 2.8.0 Beta right away.
I'm using the latest 24.03 Beta 2 for a hotel, and have my hotel clients have no issues what so ever. 99 % connects just fine.

Hmmm...unless there is a compelling reason to do so, I'm not inclined to adopt a beta version at the moment.

Tip : in the portal forum, have a look at the forum threads that mention "rfc8910". That's the more modern way to handle a portal. No file edits are needed, but you have to create a PHP file (examples are given)

Thanks for pointing this out. I'm working on setting this up with ISC DHCP on my pfSense install. No luck so far, but I have at least a thing or 2 to test. By the way, is this a good/current rfc8910.php for my set up or can you point me to something more current/appropriate?

[Gertjan]

@regexaurus said in Captive portal + DNS redirect:

I understand that it's the OS calling for that URL. On my test (Windows 11) device, I only see that happen when Chrome is the default browser. If Edge or Firefox are the default browser, then I never see that URL in the browser address bar, and the browser quickly shows the captive portal login page.

My turn.
PC desktop with a Ethernet and Wifi.
For the fun of making my point, I could remove my browser (Firefox) and with some more effort, remove the crappy Microsoft-supplied-browser Edge. So : a system without any "browser' installed.
Normally, the Wifi is de activated.
I activate the Wifi.
I can see the Microsoft http portal detection request in the pfSense portal's web server request logs.
So again : portal detection its a OS facility, not a browser facility. If a portal guest detected, a browser will be needed to handle the user interaction, aka : typing the password on the login page etc.

@regexaurus said in Captive portal + DNS redirect:

with ISC DHCP

You're using 2.7.2, so the good old (30+ years) ISC will do fine.
With pfSense plus, since 24.11, the new kea dhcp works fine, and even better with 25.03 beta.

@regexaurus said in Captive portal + DNS redirect:

By the way, is this a good/current rfc8910.php

Yeah, looks good.
Step 1:
Place the file in the /usr/local/captiveportal/ like this :

Btw : technically, you could use the GUI "file upload" function.
imho : Don't do that.
Activate SSH "as a system admin needs it anyway".
Install and use the free SFTP client :

and the access setup is identical to the SSH access :

and from now on you can browse your pfSense file system with the click of a mouse.

At this moment, the file 'rfd8910.php' file does nothing.
Step 2:

Add a DHCP (ISC) server Option parameter :

As I'm using https = imho, totally mandatory these days, even more when the portal is used in an environment where you don't know who or what the user will be, as they are unknown to you and me.
This means you need to own or rent a domain name. The domain name I use exists on the Internet, but I'm not using it for anything, except for my internal pfSense LAN(s).
Yes, this means from now on, you're good for a 10 € / $ budget every year. That's the price for using https.
Do not rent a domain name 'anywhere', don't use any registrar, check first if they support the acme.sh functionality. acme.sh is a pfSense package that handles the certificate renewal. This certificate will be accepted by any browser out there.
Because your portal is using https, the FQDN is used, not the IPv4.
My portal uses a dedicated pfSense LAN (= OPTx) network, 192.168.2.1/24
Place this host Override on the Resolver settings page :

so from now on, if a portal device asks tho resolve "portal.bhf.tld" it will get 192.168.2.1 as an answer.

I presume you know how to find the portal port number, as it is not "443" or the default https TCP port.

You can see if a portal connected device ask for the rfc8910 DHCP option :

and hit start.

Now, connect your... dono, phone to your wifi.
Right way, you will see 2 packets - these are the DHCP negotiation :

My phone ask for a lease :

09:17:01.246434 e0:92:5c:d9:6c:fe > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: (tos 0x0, ttl 255, id 16641, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from e0:92:5c:d9:6c:fe, length 300, xid 0x6c780232, Flags [none] (0x0000)
	  Client-Ethernet-Address e0:92:xx:d9:6c:fe
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Parameter-Request (55), length 9: 
	      Subnet-Mask (1), Classless-Static-Route (121), Default-Gateway (3), Domain-Name-Server (6)
	      Domain-Name (15), Unknown (108), URL (114), Unknown (119)
	      Unknown (252)
	    MSZ (57), length 2: 1500
	    Client-ID (61), length 7: ether e0:92:xx:d9:6c:fe
	    Lease-Time (51), length 4: 7776000

You saw the "URL (114)" , That's my phone using RFC8910 !
It wants an URL, if it exists on tjhis network - and it does as this is my captive portal !

The DHCP answer :

09:17:01.248463 90:ec:77:29:39:2d > e0:92:5c:d9:6c:fe, ethertype IPv4 (0x0800), length 421: (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 407)
    192.168.2.1.67 > 192.168.2.10.68: [udp sum ok] BOOTP/DHCP, Reply, length 379, xid 0x6c780232, Flags [none] (0x0000)
	  Your-IP 192.168.2.10
	  Client-Ethernet-Address e0:92:5c:d9:6c:fe
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Subnet-Mask (1), length 4: 255.255.255.0
	    Default-Gateway (3), length 4: 192.168.2.1
	    Domain-Name-Server (6), length 4: 192.168.2.1
	    Hostname (12), length 14: "iphone-gertjan"
	    Domain-Name (15), length 20: "bhf.tld"
	    Lease-Time (51), length 4: 86400
	    Server-ID (54), length 4: 192.168.2.1
	    URL (114), length 65: "https://portal.bhf.tld:8003/rfc8910.php?zone=cpzone1"

Noteworthy :
My phone got the IPv4 192.168.2.10 - and DNS 192.168.2.1 (== pfSense, is also the gateway).
And : RFC8910 :

URL (114), length 65: "https://portal.bhf.tld:8003/rfc8910.php?zone=cpzone1"

so now it knows that is has to 'visit' "https://portal.bhf.tld:8003/rfc8910.php?zone=cpzone1" right away.
This URL request will return this :

$rfc8910_url = 'https://' . $_SERVER['HTTP_HOST'] . '/index.php?zone=' . $cpzone;

and that will be, in my case :
"https://portal.bhf.tld:8003/index.php.php?zone=cpzone1"

which is the classic login page URL.
A default browser will get called with this URL.

The advantages are :
It's way faster.
No more http redirects, no more DNS hassle.
If, for any reason, the device doesn't support rfc8910, no problem, the classic portal detection still works just fine. This means that : if it (rfc8910) doesn't work, nothing gets broken.

Disadvantages :
The https access is needed anyway, and isn't rfc8910 related, but needed.
A DHCP server option is needed. pfSense supports that just fine. A set it and forget it item.
You have to place a file.

I'm pretty sure the the Netgate team will, in a nearby future, integrate all (rfc8910) this.
This still means that if your portal is any serious, you still have to deal with the certificate (https access) but it's soooo common these days, that every system admin should know how to deal with it.
https, or TLS, is part of our live these days, we all use it a thousands times every day, so some one should know what it actually is.

[regexaurus]

@Gertjan Thanks again, for the response/support!
Yes, I have ACME set up to request/renew a TLS/SSL cert and added a host (A) record to Resolver, pointing to the captive portal interface, for the CN host on the cert. Under HTTPS Options for the captive portal, I checked/enabled the Enable HTTPS login option, entered the certificate CN hostname for the HTTPS server name, and selected the appropriate certificate from the SSL/TLS Certificate drop-down. After additional testing/tweaks, this seems to be working quite well. Adding the RFC8910 option seems to be a significant improvement. I even re-enabled my DNS redirect firewall/NAT rules which now work as expected, alongside the captive portal. I haven't done extensive testing, but on one system (an iMac running Sequoia 15.4.1 + Google Chrome 136.0.7103.93), with Google Chrome as the default browser, sometimes the system wouldn't automatically load/display the captive portal login. Once when this happened, I manually opened Chrome and simply entered google.com in the address bar. When I did so, this appeared:

When I clicked the Connect button, the captive portal login page appeared. On the same system, I never encountered this when Firefox or Safari were set as the default browser. I'm not sure what to make of this, but prefer to avoid quirks like this where possible.

Now I want to do some Android and Linux testing...

[Gertjan]

@regexaurus said in Captive portal + DNS redirect:

Yes, I have ACME set up to request/renew a TLS/SSL cert and added a host (A) record to Resolver, pointing to the captive portal interface, for the CN host on the cert. Under HTTPS Options for the captive portal, I checked/enabled the Enable HTTPS login option, entered the certificate CN hostname for the HTTPS server name, and selected the appropriate certificate from the SSL/TLS Certificate drop-down. After additional testing/tweaks, this seems to be working quite well

@regexaurus said in Captive portal + DNS redirect:

Adding the RFC8910 option seems to be a significant improvement

Easy check to see what device is using the RFC8910 login method, obtained by the DHCP lease request.
The rfc8910.php file, line 97, remove the comment :
Change

/*	captiveportal_logportalauth("rfc8910", "EMPTY SESSION", $clientip, $cpzone); */

to

	captiveportal_logportalauth("rfc8910", "EMPTY SESSION", $clientip, $cpzone);

and now you'll see all the request made for this rfc8910.php file.
This will somewhat flood your captive portal authentication log.

@regexaurus said in Captive portal + DNS redirect:

but on one system (an iMac running Sequoia 15.4.1 + Google Chrome 136.0.7103.93), with Google Chrome as the default browser, sometimes the system wouldn't automatically load/display the captive portal login. Once when this happened, I manually opened Chrome and simply entered google.com in the address bar. When I did so, this appeared:

Upfront : I use Apple devices like ipads and iphones. My latest Apple computer experiences dated from ... not sure, probably the the Apple II.
Afaik, as soon as the device knows that it is behind a portal, as it will be aware of this as soon as it connects to a wifi or cabled network and the DHCP event will return a the option 41 ID = the URL to a file where it should connect to using a browser.
On ipads and iphones, this is done automatically, as soon as the wifi connection to the portal SSID becomes active and a DHCP lease was acquired.
On iMac OS : behavior could somewhat be different.

Somewhat strange; imho, that you need to type in 'some' URL to force the browser's to show you the login page. The browsers knows their is a captive portal : it showed you the login URL

The Wi-Fi you are using may require you to visit ....

What was the URL you've shown ? Not the host name (I use portal.bhf.tld here on the forum, that isn't my real host name neither). What is the file ? index.html or rfc8910.php ?

That's isn't a may .... the URL shows was obtained by the DHCP request and needs to be visited so a login page shows up, so the user (human) can identify.

Btw : Chrome from Google. That's not -imho- a browser, more a system / user data collector. I'm a FF man, as long as it lasts.
Be careful with commercial browsers, as they tend to not use the system (iMac, PC's) DNS, they go straight to their own DNS server, like 8.8.8.8, most probably using DoH or DoT, so the pfSense Resolver never sees their DNS requests. So pfBlockerng can't work ... DNSSEC can't work .... But, when a portal is used - present in the network -, this won't work.
And because DNS doesn't work for them, they have a hard time dealing with portals.
Rfc8910 should make live easier on us, but if the browser doesn't care, well, then nothing can be done to solve that.
Well, something can be done. Like not using these kind of browsers