Captive Portal not working on iOS devices only (DHCP 114)
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
quote from EDaleH: I would also like to point out that the current RFC8910.php file is no longer compatible with the index.php implementation under 24.11, despite the fact it appears to work ... so far.
Testing has demonstrated what I consider a serious problem:
With Kea reclaiming IPs very quickly and often reassigning them seconds later, the Captive Portal database can be out of date in a couple of hours. This results in confusion when the client returns, gets a different IP that is no longer authorized based on current code (index.php, RFC8910.php), the client logs in and now there are two IPs authorized with the same MAC. Then Kea gives out the original IP to a client that now shows up with a different MAC. index.php thinks they are authorized but the CP code does not so their system just sits there with no internet access or it pops up a logout page. To observe this, simply connect a few devices, let the leases expire. Then connect a a few more or connect in a different order and you should see IP conflicts and duplicate MACs in the CP database of authorized users.
I have suggested that both index.php and a hopefully "built-in" RFC8910 DHCP 114 Kea support should be checking the MAC in the CP database and updating that database to reflect the current IP instead. See: lBuilt-in Captive Portal Support for RFC8910, DHCP option 114 in Kea
One side effect of this approach is that it will make the transition from ISC to KEA simpler as it will automatically reassign the IPs in the database based on an authorized MAC address in the CP database as KEA assigns new IPs to devices as they connect. This will be seamless, they will not have to re-login. They can also be absent for extended periods of time and when they return, if their authorization has not timed out, they will reconnect without tying up an IP in their absence.
-
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
To observe this, simply connect a few devices, let the leases expire.
I'll try this.
Default leases are 120 minutes ? (more ? I'll see, and adapt if needed).
I presume I have to de activate the wifi after logging into the portal, otherwise my device will simply, at 120/2 renew the lease and I'll wait forever.
I also have to raise the soft idle timeout - mine is set to 60 minutes right now. Knowing that the value should be smaller then the DHCP lease (I don't recall the exact logic, but there were discussion about this eons ago.@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
the Captive Portal database can be out of date in a couple of hours
That how many logged in portal users ?? 50K or so ? My personal record is 200.
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
should be checking the MAC in the CP database and updating that database to reflect the current IP instead
Hummm.
So,
IF (MAC-device is found in the portal database)
THEN replace the IP-device with the IP in the database, in the found record.That's no rocket science but the list with side effect is probably not 'zero'.
Next-best : if the MAC was found (so a previous logged instance exists), but the IP changed, then delete the record (do a log out of user). The user will be thrown back to the login page, and from there it's just a 'comfort thing', but the user will know what to do.I've a question based upon pure on "me thinking" : isn't a DHCP server serving IPs out of the pool, and it will chose the one that wasn't used for the longest period ?
After all, if a connection breaks for a moment, which is more then typical with a wifi connection,the same MAC will be presented and the DHCP will know that that device was using that IP.
A device that disconnects for more then a few moments, and the IP that was used during it's initial lease is already occupied by another device ... doesn't that mean that your DHCP pool is way to small ?
I know a device can ask for a "preferred" IP (lease) - but I really thought there were some condition to be met before kea actually grants such a request. Like : everybody want "192.168.2.2", and kea hands one out to the very first who is asking for it. If this lease isn't renewed, another device might get it, if it was asking also for 192.168.2.2.Btw : I start to see the reason why the portal soft idle time out should be, if possible, shorter as the DHCP lease duration. This will take care of things ...
kea knows what it did in the past, even if you restart it : see the 2 (or 4) files here : /var/lib/kea/
Btw : right now, my captive portal KEA DHCP server is using a pool : 192.168.2.10 to 192.168.2.254 or about 244 lease are possible before something nasty happens.
Right now, the risk I see more then 30 users is .... close to zero.edit : If you think I didn't get the issue, say so.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Btw : right now, my captive portal KEA DHCP server is using a pool : 192.168.2.10 to 192.168.2.254 or about 244 lease are possible before something nasty happens.
Right now, the risk I see more then 30 users is .... close to zero.Fully understanding the issue is not necessary, best to focus on the results. With ISC, it maintained a list of expired leases and you can see those through Status, DHCP Leases, Show all configured leases. The snapshot below is of the Production 8 Portal RV site that closed on Oct 15th. You will notice that the configured leases still fill 3 of the pools. It stays that way after a month or so because so many devices connect that do not log in but do get a lease before they move on.
Everything works fine because ISC appears to re-allocate from the oldest lease and it has never allocated an IP that is recent enough to cause any issues. I suspect many Captive Portals work this way. Even though your lease expired, when you return, you get your old IP and can reconnect, even if your CP timeout is a month for example.
Yes, the lease should be longer than the idle timeout on the user, suggesting they will be timed out at/before the lease expires. That is not the real world though. Even though your lease expired, when you return, you get your old IP and can reconnect, even if your CP logout timer is a month for example. With Kea limited to 24 hr leases, that would mean week, month and forever could not exist but they are in common use, particularly if you use freeRadius. With ISC, those CP time limits are matching those durations associated with the account, be it a voucher or freeRadius "forever" account (they have day, week, month built in). So it is all about if you want the user to be logging back in all the time. If you set the time limit to < or = the lease time, they will have to log in every time they leave or are out of wifi range for longer than the DHCP lease. Yes, they get very angry when they pay for a month of access and only get 2 hrs away before they have to log in again, for example, every time they go buy groceries. They want to log in once and automatically be logged out at the end of the time they paid for. They don't wan to miss email just because they did not notice their phone popped up a login screen when they got back and as a result is not connected. It should work like it does at home, just simply reconnect when you get to the driveway. To achieve that, we can't disconnect them from the portal. With ISC, even though we have filled the pool, we are safely within that month or so time before the lease gets reallocated to a different MAC.
The difference is how Kea manages expired leases. If you watch a lease expire in Kea, it disappears from the "Show All configured leases" in seconds. Now it is in the pool and Kea loves to allocate the lower IPs frequently. During testing I was trying to create a duplicate by logging other systems in one after the other once 3 leases had expired. It only took 3 tries and two minutes before the IP was allocated to a different MAC, that IP was XXX.XXX.XXX.002, the first IP in the pool. Now I logged that user in and sure enough, I had two of the same IPs in the CP database with different MACs. When I try to use one or the other, the CP will look up the sessionid for the first IP in the database and one of them can't connect at all. Now you immediately say, but I broke the rules, the CP account should have expired with the lease and that would prevent the duplicate. True based on the CP warning in the GUI, but not real world and more important, very different from the experience with ISC as the DHCP Server.
There is absolutely no reason why the CP account authentication has to expire with the IP lease, as demonstrated in the logic presented in the example code. Unfortunately it appears to loop while the login screen is on the remote client and waiting for entry of login credentials so pfSense programmers need to get involved in making it CPU efficient.
It also really messes up Smart Devices that use DHCP 114 as if the IP/Mac is not in sync, they will just get stuck in the temporary browser screen trying to load either the login or logout page and of course appear to be hung.
Fixing the CP db IP in the index.php file leaves all existing CP code intact and functional. Pretty basic stuff and a permanent solution that works with ISC and KEA and even lets you switch back and forth between them without disconnecting the users as index.php will now sort out the IP conflicts automatically.
I hope this make sense to you. For us, it is critical to retaining functionality on a very successful installation(s). 0 support requests in 2024 vrs 20 complaints in the first day when authentication was tied to lease timeouts (that was in 2021 when it first went into service). pfSense has been running at this site since 2010 or so. I have supported it that entire time. Without this fix, we stick with the exceptionally stable CE 2.7.2 and ISC (running on Netgate XG-1541 maxes because Plus was not as stable and difficult to keep up with, case in point!)
-
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
The difference is how Kea manages expired leases. If you watch a lease expire in Kea, it disappears from the "Show All configured leases" in seconds
A wild shot :
https://kea.readthedocs.io/en/kea-2.0.0/arm/dhcp4-srv.html#why-is-lease-file-cleanup-necessary
Look what lfc-interval can do for you ?
pfSense doesn't add that config setting, so there must be some 'default' value. Dono what it is.
You suggest : within seconds. Looking at my time stamps, you're not far of.I've added this into my kea-dhcp4.conf file (by editi,ng /etc/inc/services.inc) :
with 2592000 is the number of seconds in a month.
It won't get overboard, as I haven't that many lease to offer.
In theory, the leases file will now grow (not cleaned) ? We'll see.edit : be aware : you use 2.7.2, the kea implementation might be somewhat different.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
edit : be aware : you use 2.7.2, the kea implementation might be somewhat different.
No, the production site is on 2.7.2 and has no issues at all, that snapshot was of it today. The lab site is the subject of this post, i.e. the next upgrade, is on 24.11 Stable with Kea patched for RFC8910.
wrt to extending the life of a lease, that is not a good solution because we do have enough traffic to run out of leases, it is far superior to manage the IP assigned to the "authorized" MAC in the Captive Portal Database. That will survive most attempts to lock up the available leases, be it deliberate or a feature of the usage. The Captive Portal would be able to support more users than there are leases, like a fast food place for example where a lot of people come and go but at any one time they are within the DHCP pool size. Log in once and you reconnect automatically every time you go there, much like you can select to auto login to the WiFi itself.
Now is the time to fix this so Kea can go into service and meet it's high availability goals and we can create portals with logins that never expire unless we want them to, a dramatic improvement in functionality.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I've added this into my kea-dhcp4.conf file (by editi,ng /etc/inc/services.inc) :
I continued to focus on living with the IP changing frequently due to agressive reclamation of leases by Kea as that is the default setting and in our favor if the network gets heavily loaded so that leases are available.
I did find some coding typos and a logic error in my earlier index.php and RFC8910.php attempts to achieve this. I have uploaded the most recent files that passed my lab test today. I think they will demonstrate how this could work. If you want to do any testing yourself, the files are attached to Redmines 15854 and 15904. I will keep those proof-of-concept files up to date if further changes are necessary.
I am optimistic that we can not only adopt Kea in place of ISC sooner, but improve the Captive Portal at the same time. If you see this potential as well, let's encourage Netgate to incorporate a Mac authentication based Captive Portal (i.e. IPs can change anytime) and incorporate a default RFC8910 DHCP 114 support into Captive Portal/Kea so that Smart Devices like the iPhone will work with a first time setup without forcing those new to Captive Portals to read through massive user form entries to achieve a basic functionality with all devices.
-
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
I continued to focus on living with the IP changing frequently due to agressive reclamation of leases by Kea
Kea, if not asked for, doe nothing. It doesn't spam leases into a LAN. It only answers to a device that is asking for a lease. [sorry, had to say this ^^]
With any APs and many SSID type devices, like a lot (thousands) I can presume that your Kea is actually handling a lot of requests.
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
but improve the Captive Portal at the same time.
Your kidding ? This will open the door to full IPv6 support also. And IPv6 that will handle your 'pool full' issue ;)
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
If you see this potential as well, let's encourage Netgate to incorporate a Mac authentication based Captive Portal (i.e. IPs can change anytime) and incorporate a default RFC8910 DHCP 114 support into Captive Portal/Kea so that Smart Devices like the iPhone will work with a first time setup without forcing those new to Captive Portals to read through massive user form entries to achieve a basic functionality with all devices.
Well, sure thing.
You mean lobbying ? Like in the states ?
If there is one guy at Netgate that takes a look what RFC8910 does, that its been test driven for the better part of a year now, that it doesn't need any code changes in the actual portal scripts (code) - that it can be activated or de activated with a click (I've made this in the GUI just for the fun) , that, yes, a DHCP Kea option 114 is needed - which tells me Marcos (Netgate, right ?) is already aware of what RFC8910 does.
But : from their point of view : what do they gain ? I never fully understood the real reason of making things more complicate ... euh .. sorry, adding more gadgets every day.
Gadgets have to be supported.
The average pfSense admin knowledge level isn't getting better every day neither (talking about myself of course).
And one last thing : I do see your issue, but I can 'lobby' for a feature that I do not use myself. Only if it works for me, and, yes, I'll repeat again : RFC8910 works even better as the original 'Netgate' portal implementation. So : we should all be winners ?I'm pretty sure 'they' already know about this RFC8910 things, but what or who is able to put the RFC8910 implementation order higher up in the "must this do first" list... I don't have his or her phone number and something says me this person doesn't use the forum (neither redmine).
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
default RFC8910 DHCP 114 support into Captive Portal/Kea so that Smart Devices like the iPhone will work with a first time setup without forcing those new to Captive Portals to read through massive user form entries to achieve a basic functionality with all devices.
Not really an issue for me.
Because my iPhone 'knows' about 'my captive portal', and when I switch from native portal 'pfSense' portal mode to RFC8910 'assisted' mode (or back to 'Netgate' mode), my phone has difficulties to connect as it keeps profiles of each Wifi connection that it has used in the past. I have to zap the profile, and case closed.
I agree, before, I don't recall I had to do this anywhere else. I didn't even kown I could 'delete' the profile. (well, I did, but didn't saw the usage of it)
Other people, like my clients, who bring in their iPhone or iPad (and other devices also) will connect to my SSID (portal) and don't have any issues at all with my captive portal ruunning on RFC8910 mode.
This issue is probably an Apple iOS issue. Let's contact apple devs ? -
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Kea, if not asked for, doe nothing. It doesn't spam leases into a LAN.
We are trying to live with Kea and within DCHP usage guidelines. When an IP changes because the DCHP server issues a new one, even to a MAC that had a different IP that expired less than 3 minutes ago, it is not spam, it is a "feature". That feature of Kea is what triggered this entire conversation. Kea is coming, how do we live with it?
We are throwing serious systems at these installations (For us RV parks and Yacht Clubs mostly) so Kea can work it's heart out rather than us try to optimize "something". The RV Park that is currently being lab tested has buried fiber to every AP (~20 of them) at 2-3 ms (ping) latency to the internet at the AP, 6-10 ms at the client. The Internet service was brought in from Kms (miles) away at great cost to ensure that premium internet service was available. The ISP is providing a business account at many thousands of dollars a month., that even applies in the off season. All that works perfectly, it is the pfSense Server with Captive Portal support that is taking up all of the support effort.@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
yes, a DHCP Kea option 114 is needed - which tells me Marcos (Netgate, right ?) is already aware of what RFC8910 does
Yes, I was talking with Marcos about support for it in late 2021 and we went live with the first (buggy) implementation that every iOS release broke until roughly when you got involved. You made major contributions to the current version and I hope it is helping a lot of sites keep pfSense/captive portal alive with DHCP 114. He is also involved with this new suggestion but I inform him and once he understands, it becomes a black hole so I double up with a Redmine, every one of which is still waiting to get past the "Future" status.
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
This issue is probably an Apple iOS issue. Let's contact apple devs ?
We tell everyone to "Forget" the connection and/or check that the Auto Login is on for iPhones. I can only estimate but I assume we provide internet access to typically 200 and a max of 500 devices (6 portals for them, each with 250+ leases available) on any day and 30-40% of the devices change to new devices weekly, maybe 10% daily. A lot of these devices are iPhones/andriod phones/tablets but they are also TVs, cameras, Alexa devices, a "Hot Spot Router" we rent them for devices that don't support he captive portal login, etc. We have never had an issue with leases not being available despite the saturated pool showing in ISC (see above). We have many "seasonal" guests (as well as "Transitional") who have monthly, auto-reset, Captive Portal authentications. They login in the spring and never login/logout again unless their account hits limits (data mostly). We do the logout at the end of the season, often leaving for a month or more during the season, simply reconnecting automatically when they return. The "transitional" accounts are 24_Hrs, WEEK and MONTH. We also use multi-user accounts that have a time & Data limit shared amongst all users for that account. (Yes, support for that is customized into captiveportal.inc due to current limitations but all code is covered by Redmine)s. Transitional accounts limit out or just expire on time, independent of when the device itself leaves. Purchases are made through the login page, we never direct sell the guests wifi, just give them an info page at check-in. There are lots of foreign guests so language is often a barrier as well. 2024 was an amazing season, 100% up time, no support issues at all, no service complaints (except cost), just compliments. This is all about keeping it that way.
At this time, Redmines 1584 and 15904 have the preliminary code that is going to make this possible. The only remaining decision will come in March when we decide to stick with CE 2.7.2/ISC for the season or make the jump to the current Plus version with Kea enabled. I load it down in the lab and work it for 32 days to ensure all of the automated code does it's job; this new code included. I am not optimistic pfSense will ever support this natively and I am trying to share the effort as we did with RFC8910. This subject is far, far more influential on the future than Apple's DHCP 114 exercise, it is a make or break for captive portal on pfSense in my opinion.
-
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
When an IP changes because the DCHP server issues a new one, even to a MAC that had a different IP that expired less than 3 minutes ago, it is not spam, it is a "feature".
Let me get this right :
A device uses a lease. The lease expires.
This means that the device isn't there anymore for at least the duration of 50 % of the total lease duration, as DHCP clients renew their lease half way. Right ?!
If the DHCP total lease time is 6 hours, this means that this device looses its lease, but this device already left the network 50 % x 6 = 3 hours earlier. If not, it would have renewed it.So, back : the lease expired (as not renewed). In that case, if another device connects to the network, and asks for a lease, any available IP (lease) out the the pool will be selected.
This can be the IP (lease) that expired just 3 minutes ago.
I would have hoped that a last recently used scenario existed - but I'm not sure if kea works that way.I can't stop thinking of one thing : aren't your pools not to small ?
Or is kea really preferring to asign leases out of the pool that are least recently used ? And in that case I fully understand your question/issue.
How many device (owners) with valid active credentials do you have out there ?/usr/local/captiveportal/index.php : line 85 :
$cpsession = captiveportal_isip_logged($clientip);The session is based upon this unique item : the device's IP.
The function captiveportal_isip_logged takes the IP of the device, and return the session ID out the the connected portal users database.
This can be easily changed against the MAC of the device.The MAC of the device is determined a little bit lower in the index.php file :
/* find MAC address for client */ $tmpres = pfSense_ip_to_mac($clientip);and these two lines can be moved before line 85.
Now I have this :
//$cpsession = captiveportal_isip_logged($clientip); /* find MAC address for client */ $tmpres = pfSense_ip_to_mac($clientip); $cpsession = captiveportal_ismac_logged($tmpres); if (!empty($cpsession)) { $sessionid = $cpsession['sessionid']; }And I've added a new function in /etc/inc/captiveportal.inc ( just below captiveportal_isip_logged()) :
function captiveportal_ismac_logged($clientmac) { global $g, $cpzone; /* read in client database */ $query = "WHERE mac = '{$clientmac}'"; $cpdb = captiveportal_read_db($query); foreach ($cpdb as $cpentry) { return $cpentry; } }And I tested : I could login just fine.
I'm pretty sure that there are more edits needed, as you want to use the MAC as the selector to see if a device is authenticated, instead of the IP.
The thing is : I can't really test this.
( when I'm writing PHP, I start with the idea that what I write is wrong, up until proven otherwise. If something works right away, it become even more suspicious )@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
The ISP is providing a business account at many thousands of dollars a month
Aha !
Yacht club in the middle of the amazon rain forest ?edit : Oh wait there is more.
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
At this time, Redmines 1584 and
Basically, what might do the trick : after portal authentication, get the IP and MAC, create a static DHCP lease ... because you want a device, if it come back, even days week later on, to have it the same IP as before.
For a static trusted LAN, this is a normal situation, as you 'known' every devices.
A portal is by nature a non trusted network. There will by "DHCP", the device will get 'some' I and that it. Only a static lease can guarantee a fixed IP.The captive portal was written with temporary connection in mind I guess. Perfect for me, when running a hotel (people don't stay for weeks - and if they do, I MAC white list them, but this is very rare)
Or portal spaghetti code changes.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Or is kea really preferring to asign leases out of the pool that are least recently used ? And in that case I fully understand your question/issue.
Kea appears to favor assigning lower numbered leases quickly, 3 minutes is quite common. Win 11 requests 2 hr leases, connect , login and immediately disconnect. 2 hrs later the lease expires. Start connecting new devices (the will get the next numerically larger IP) and about 3 min later, it will give out the most recently expired lease. For eg: If the machine is powered off and comes back after all the leases had time to expire, it will start at the beginning of the pool every time and allocate them in sequence upwards. ISC appeared to allocate the oldest expired lease, so a power outage had no noticeable impact. Now, with Kea, it kills the entire portal unless the accounts are set to expire with the leases, that means very frequent logins and vouchers are not memoizable.
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I can't stop thinking of one thing : aren't your pools not to small ?
8 Captive portals with most having 250 leases available each (VLan), 4 of those portals (networks) take the heaviest load of users. We have never had a pool full of active leases. In 2024 we had an Annual holiday line up with a very popular Air Show and came as close to saturation as we ever have and I suspect could support. That 2 day weekend I estimated they had 500 devices but did not count them, I was extremely impressed with how well pfSense handled the traffic that was saturating the ISP bandwidth.
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
And I tested : I could login just fine.
I'm pretty sure that there are more edits needed, as you want to use the MAC as the selector to see if a device is authenticated, instead of the IP.
The thing is : I can't really test this.
( when I'm writing PHP, I start with the idea that what I write is wrong, up until proven otherwise. If something works right away, it become even more suspicious )I am trying to keep all of the modifications within index.php and RFC8910.php. The latest RFC8910.php incorporates a dual ip/mac check and should be compatible with everything, even CE 2.7.2.. This should now become the only file out there.
I want to be sure that the MAC is the current mac so I did not use the CP routine to check the MAC, I used the inline code to get the actual MAC to be certain I got the right one.
I am not a programmer! I start with a state diagram, then inline code that is easy to read, get it working and then optimize it. The index.php code is now at the optimize stage. It will become a switch/case structure and not rely on any new captiveportal.inc calls at all.
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Aha !
Yacht club in the middle of the amazon rain forest ?This area is on a huge inland lake system very close to both marine and vehicle major thorough fares. For the RV Park with the expensive fiber Internet, they spent $250K+ on bringing business fiber in on the understanding they would pay for a high end business account (16 static IPs) for 5 years, 2 years to go on that contract). There is no way they will ever cover the cost from sales, it is all about service. The park owner is president of the Country wide RV association. Most of the RVs that stop there are worth between 1 and 2 Million each. They want Zoom to run flawlessly on the lake front with their RV in the background, for their board meeting. Now one YC is similarly located but has a 2 stall head and no potable water, people treat their boats like cottages with access to amazing sailing. locations and racing. They want their email and marine weather forecasts.
I am retired, this is not a revenue generating endeavor it is a hobby. I like to support the "opensource" nature of some projects, (OpenWRT as well for example). I got involved 15 years ago while I was either sailing or travelling in an RV. pfSense systems were set up all over the place as I travelled so I could get good wifi.
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Basically, what might do the trick : after portal authentication, get the IP and MAC, create a static DHCP lease ... because you want a device, if it come back, even days week later on, to have it the same IP as before.
For a static trusted LAN, this is a normal situation, as you 'known' every devices.
A portal is by nature a non trusted network. There will by "DHCP", the device will get 'some' I and that it. Only a static lease can guarantee a fixed IP.The captive portal was written with temporary connection in mind I guess. Perfect for me, when running a hotel (people don't stay for weeks - and if they do, I MAC white list them, but this is very rare)
Or portal spaghetti code changes.
No, I think this approach is perfect, let the IP change, we don't care. Fix it as a self contained (no new calls to captiveportal.inc) enhancement in index.php and from yesterday's testing, it works pretty much perfectly. You can set your cp accounts to expire with the leases or if you prefer set it to what makes sense. If a client is staying a week, give them a week account, a month, a month account, for a 6 month season, a seasonal account that resets time/data quotas daily/weekly or monthly through freeRadius authentication. Simple, independent of Captive Portal ongoing development and they only log a device in once. If I was to print all of the custom code that achieves this in captiveportal,inc and now index.php, it would fit on the front and back of one page at 10pt. Now keeping up with pfSense releases can be tedious, not the code, the testing to make sure it is still OK. Every one of these enhancements should be part of the release, it just makes sense.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
A wild shot :
https://kea.readthedocs.io/en/kea-2.0.0/arm/dhcp4-srv.html#why-is-lease-file-cleanup-necessary
Look what lfc-interval can do for you ?
I should have mentioned that I looked at 11.4 Lease Affinity to dampen the frequency that the IP changes but only to the extent of hours, maybe days if the pool has sufficient redundancy. The Kea manual has an example configuration for 30 minutes of delay before a lease is reclaimed but of course, it is allocated if there are too many requests and the pool needs it. Kea is concerned about the impact this has on performance. I am testing with the default Kea values but will likely do the next stress testing with around 12 hrs of "Lease Affinity" to verify it reduces the reindexing load on new Captive Portal (CP) connections by many of the reconnects occurring within that window. It will never solve the week and month CP accounts, nor will it handle "hybernation" by devices very well as they cease to renew their leases while asleep.
I still feel that the CP should not care about IPs, I am focusing on using just the MAC to verify authentication through the CP. Damn the IP, full speed ahead! To avoid iOS devices hanging on a lease renewal changing IP, I recommend using the RFC8910.php file that checks both MAC and IP and redirects to index.php with a "False" if both don't match.
-
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
Look what lfc-interval can do for you ?
I use that setting for a couple of days now. Purge time : 1 month or 'many seconds'.
The /var/lib/kea/leases files, become bigger every day. 100 K and growing right now.
I've a 'small' setup : LAN with 50 devices, mostly static IPs, and a portal with 20, 30 devices connecting a day, so close to nothing.I'll revert back monday (remove lfc-interval) as see no advantages of using it, except for the fact that it shows a detailed DHCP activity.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I use that setting for a couple of days now.
I think you will find you are using lkea-lfc - Lease File Cleanup process in Kea which is a completely different animal. It handles the time the list of leases are kept in a "log" of sorts. Reclamation still occurs based on it's settings as far as I understand but I do not know if Kea takes the file into account when it allocates new leases, I believe it impacts performance if the file gets too large.
11.4 Lease Affinity is a different "kettle of fish". It is specifically designed to prevent reclamation unless the pool has no new or reclaimed leases available. I got stuck because when I tried to add it I hard crashed 24.11 and had to rebuild. I see rumors that it will be included in the next release of plus. I am hoping it can be applied by interface, i.e. portal.
In the meantime, it doesn't matter, the revised index.php and rfc8910.php files are working as intended but of course without lease Affinity extended beyond the default of 0 seconds, the IP changes frequently when the device fails to renew the lease. I could always increase the 2 hr lease duration default value but the kind of defeats the point in checking that the MAC authorized Captive Portal is working as intended on the lab bench. In fact, I was thinking of decreasing it to work the lab test harder.
Lease Affinity will be essential for current Captive Portal installations to migrate to Kea/Plus 25.01 due to this issue. They may have to set the Affinity value to the duration of the Captive Portal authorization, a 1 day, 1 week, 1 month for example. The problem is: if there are a lot of devices "just looking" and grab an IP, it will tie up a lease from the pool for the duration of the Lease Affinity value. At the end of the day, this mod to change index.php to respect the Mac authorization instead of IP will become essential for a larger number of devices. I have already decided it is time to assume we are going that way. I will still use Lease Affinity to reduce the number of re-indexes of the Captive portal due to IP changes, likely around12-24 hrs so that trips to the store or work or hibernation/sleep, still stay connected. I think we "got this".
(PS: don't forget to use the new RFC8910.php that checks both IP and Mac or the phones will start hanging on the temp browser screen)
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I'll revert back monday (remove lfc-interval) as see no advantages of using it
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I've added this into my kea-dhcp4.conf file (by editi,ng /etc/inc/services.inc) :
I suggest again that you turn on lease affinity in /etc/inc/services.inc instead. It goes just above the "loggers" line in the DHCP4 section of that file, just below where your log file above was placed. I reversed the Kea Doc JSON to the format used in services.inc to save us some time.
'expired-leases-processing' => [
'reclaim-timer-wait-time' => 3,
'hold-reclaimed-time' => 43200,
'flush-reclaimed-timer-wait-time' => 5
],
So far it sets a default for all interfaces but I am hoping we can set it per interface as I will need different "affinity" on each VLAN/CP. Below you see my test set to 12 hrs (43,200 seconds) of lease affinity, i.e. once allocated and expiried (I set mine to 5 minutes) leases are "held" for 12 hrs unless the pool gets full at which time one of them gets re-allocated anyway, I trust it is the oldest one but untested so far. This will insert the JSON in 11.4. Configuring Lease Affinity into /usr/local/etc/kea/kea-dhcp4.conf:
Working as advertised in my lab testing so far. Lease Affinity will permit simple Captive Portal installations to work with Kea without mofifying RFC8910.php or index.php as long as the CP timeout value is shorter than or equal to the 'hold-reclaimed-time". If you have a voucher good for a month for example, you would end up tying up all of the last 30 days worth of leases, even if they did not log into CP. This could depleate the pool but fortunately the documentation suggests it will allocate from the "Affinity Pool" if such a situation happens, hopefully the oldest IP. This will break the login if that IP gets reallocated, there will be two authorized MACS on the same IP in the Captive Portal list. The original RFC8910.php will indicate an "F" based on the IP match, even though it is a different client, and the second login will hang on iPhones and/or show the logout screen. To fix this, run the index.php and RFC8910.php in the two Redmines mentioned above. If you do not want to be playing "whack-a-mole" with IPs in the authorized list of MACS (and the ire-ndexing delays each time an IP changes), use both those test files and the Affinity setting. This looks like the future of CP with Kea, like it or not.
-
Ok, done.
Instead of editing /etc/inc/services.inc, isn't it possible to add :
"expired-leases-processing": [ { "reclaim-timer-wait-time": 3, "hold-reclaimed-time": 43200, "flush-reclaimed-timer-wait-time": 5 } ],to here :
?
Anyway, I've edited services.inc :
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Instead of editing /etc/inc/services.inc, isn't it possible to add :
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
to here :When I looked through the services.inc it looked dicey and I found any error was a hard crash/reinstall. I assume this will be supported in the next plus release so once I got it working on all interfaces, I just needed it working for testing so I stopped playing with it There is no way we go live with 24.11.
BTW, everything expired this morning, I scrambled the connection order and everything sorted itself out. This is now a stable lab test for the next 31 days to confirm the remaining CP logic (multi user/account, time quotas, data quotas, Tunnel ID freeRadius fix, etc are working as advertised in the 24.11 customized captiveportal.inc.)
It is critical that RFC8910.php check both Mac and IP for this to work or iPhones get really confused and all you have is a blank screen when an IP changes and the IP only check thinks it is validated, loads index.php, which thinks it is already validated and tries to load logout or worse yet has a new IP and duplicates it in the CP database on a new login. All avoided with a very simple mod in RFC890.php.
Of course, you can always make the idle/hard timeout in the CPs less than the lease duration in DHCP but the default setup will give a windows machine a 2 hr lease and an iPhone a 24 hr lease for example so either way you are going to have to do a lot to get your CP working properly under Kea. The modified index.php and RFC8910.php make it work out of the box, and most important, treat the CP as a device you get authorized on until you logout, run out of time or run out of data. Then it cleans itself up. Convenient, single login for the life of the authorization.
Being logged out just because the DHCP server is voracious at cleaning up IPs and reallocating them from the bottom up should not matter, but it does with the current plus code.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Or portal spaghetti code changes.
Yes, time to change the CP to at a minimum support a default RFC8910 detecting if captive and loading index.php automatically so that iOS devices at least work as the default. I stumbled upon the Kea docs for RFC8910
and 8.11. Supported DHCP Standards
Version 24H2 of Windows 11 is behaving differently. I think it is respecting DHCP 114. Although it still works with the traditional captive portal URL, it also detects the change in captive status when the IP changes and pops up the Microsoft default url (vendor_url?) once connected and it takes a while before it clears the "action needed" warning. With an iPhone, when the IP changes, index.php loads the logout page in the temp browser. Of course, if the IP doesn't change, both do nothing, connectivity is maintained in either case though. Of course they can only get an updated IP if you are running the proposed (index.php, RFC8910.php) MAC based Captive Portal (CP) authorization suggested in Redmine 15854 and/or Redmine 15904 (Note, cleaned up index.php uploaded for 24.11 Stable to support changing IPs by respecting Captive Portal Mac Authorization as of Dec 17th, 2024)
I will have to experiment further with Windows 24H2 to find out why their page is loaded rather than the logout page but everyone else is embracing RFC8910, it is time for pfSense to do so as well. May as well implement the MAC authorization within index.php/CP as soon as possible to support the fact we can't simply extend the lease times for all CP clients to avoid so many new logins, when the device sleeps or goes out for a coffee, solely to prevent Kea from changing their IP. There is no reason why CP should be a load that compromises Kea's ability to efficiently assign IPs. Outside of the current index.php implementation, CP checks the MAC anyway, index.php should respect that as well and so should RFC 8910, i.e. DHCP 114, prioritize the MAC to determine if captive or not.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I've edited services.inc :
I have noticed that once Affinity is running, not only does it protect the IP/Mac Pair until it meets the affinity duration after disconnecting, it also does not re-assign that IP again. Instead it keeps counting up with ever increasing IP numbers, even if it is the same MAC that had an IP before, and IP that is now available. It appears that it will use up the pool before re-assigning any of the IPs that were issued while affinity was turned on. Simply turning on Affinity of any duration appears to enable this behavior. This is very beneficial to Captive Portal when running the modified index.php/RFC8910.php files in Redmine 15854 and/or Redmine 15904. The default index.php and original release of RFC8910.php will pop up a login screen and once logged in, create a new authorized entry in the CP DB with the new IP, resulting in two authorized IPs for the same Mac unless the idle/hard timeout is less than the lease duration. iOs devices using the original RFC8910.php may show a blank temporary browser screen.
Without Affinity enabled, the default Kea assignment is to reuse the IPs from the lowest number upwards within seconds to a couple of minutes of the IP expiring. In this case the original index.php and RFC8910.php files will try to use the CP DB entry of that IP and fail if it was previously used & authorized (but not yet timed out), thinking it is already logged in, showing either a blank screen or a copy of the logout page. No internet access is possible until you manually delete the IP from the CP DB before connecting a device with the same IP of one already authorized.
Using the original RFC8910.php, with the recent release of Windows 11 24H2, it is popping up the RFC8910 JSON file with no login screen. The JSON is modified to go to a Microsoft URL rather than the one specified in the Captive Portal configuration. Clearly Microsoft is experimenting with DHCP 114. The modified RFC8910.php mentioned above in the Redmines does not fix this unless you are also running the suggested index.php which will update the CP IP address and thus maintain the connection. There is an issue though with Windows because it loads the connection test screen (msn) instead of the logout screen. I expect Microsoft will fix this though as it works fine with IOS, the folks who created DHCP 114 in the first place.
To make this post complete: Even though the Affinity period has expired, Kea keeps allocating IPs with ever increasing numbers from the DHCP pool until the pool is full. It then starts over at the lowest number and works upwards, respecting existing leases and any within the affinity protected time period that may not have expired yet. This behavior suggests that Captive Portals will be more reliable with Affinity enabled, independent of the duration.
With further testing, Affinity settings do not survive a power outage and Kea will reassign leases from the lowest number upwards for all expired leases, affinity durations are not respected unless the lease is still active when the power resumes. Unless DHCP leases are extended to very large durations (and idle timeouts in CP set to less than that value), this will necessitate the use of the modified index.php and RFC8910.php files for all captive portals so that reassigned IP/MAC pairs are managed within index.php to respect he MAC as the "authorization" under CP. The issue with long duration leases is the possibility of depleting the DHCP pool for that interface, despite the fact many devices are no longer connected. By authorizing based on MAC, the list of authorized CP users can even exceed the size of the address pool.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Anyway, I've edited services.inc :
I assume you are now aware of the fact Kea's Affinity memfile does not survive a reboot and will loose expired leases that still have affinity "protection". Netgate has raised Redmine #15934 to attempt to address this and other lease expiry concerns but so has the Kea development team. See the link in the Redmine.
The Kea development team have scheduled this for possible correction in V 3.0 which is slated for an April 2025 release. There is no certainty that they will include it.
Until Kea supports Affinity surviving a reboot, using Kea with Captive Portal is very risky as a reboot will likely scramble the IP/MAC assignments unless the devices reconnect in the exact same order. We will have to use ISC until then, religiously have idle timeouts less than lease duration (and the frequent re-logins that implies), or incorporate a MAC Captive Portal authorization scheme like that proposed in Redmines 15854 or 15904
This suggests that we are unlikely to have a built in solution for Captive Portal ISC equivalent support under Kea at the next plus release (25.03?) or until Netgate incorporates Kea 3.0 into the pfSense plus and CE releases.
