Captive Portal not working on iOS devices only (DHCP 114)
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Or is kea really preferring to asign leases out of the pool that are least recently used ? And in that case I fully understand your question/issue.
Kea appears to favor assigning lower numbered leases quickly, 3 minutes is quite common. Win 11 requests 2 hr leases, connect , login and immediately disconnect. 2 hrs later the lease expires. Start connecting new devices (the will get the next numerically larger IP) and about 3 min later, it will give out the most recently expired lease. For eg: If the machine is powered off and comes back after all the leases had time to expire, it will start at the beginning of the pool every time and allocate them in sequence upwards. ISC appeared to allocate the oldest expired lease, so a power outage had no noticeable impact. Now, with Kea, it kills the entire portal unless the accounts are set to expire with the leases, that means very frequent logins and vouchers are not memoizable.
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I can't stop thinking of one thing : aren't your pools not to small ?
8 Captive portals with most having 250 leases available each (VLan), 4 of those portals (networks) take the heaviest load of users. We have never had a pool full of active leases. In 2024 we had an Annual holiday line up with a very popular Air Show and came as close to saturation as we ever have and I suspect could support. That 2 day weekend I estimated they had 500 devices but did not count them, I was extremely impressed with how well pfSense handled the traffic that was saturating the ISP bandwidth.
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
And I tested : I could login just fine.
I'm pretty sure that there are more edits needed, as you want to use the MAC as the selector to see if a device is authenticated, instead of the IP.
The thing is : I can't really test this.
( when I'm writing PHP, I start with the idea that what I write is wrong, up until proven otherwise. If something works right away, it become even more suspicious )I am trying to keep all of the modifications within index.php and RFC8910.php. The latest RFC8910.php incorporates a dual ip/mac check and should be compatible with everything, even CE 2.7.2.. This should now become the only file out there.
I want to be sure that the MAC is the current mac so I did not use the CP routine to check the MAC, I used the inline code to get the actual MAC to be certain I got the right one.
I am not a programmer! I start with a state diagram, then inline code that is easy to read, get it working and then optimize it. The index.php code is now at the optimize stage. It will become a switch/case structure and not rely on any new captiveportal.inc calls at all.
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Aha !
Yacht club in the middle of the amazon rain forest ?This area is on a huge inland lake system very close to both marine and vehicle major thorough fares. For the RV Park with the expensive fiber Internet, they spent $250K+ on bringing business fiber in on the understanding they would pay for a high end business account (16 static IPs) for 5 years, 2 years to go on that contract). There is no way they will ever cover the cost from sales, it is all about service. The park owner is president of the Country wide RV association. Most of the RVs that stop there are worth between 1 and 2 Million each. They want Zoom to run flawlessly on the lake front with their RV in the background, for their board meeting. Now one YC is similarly located but has a 2 stall head and no potable water, people treat their boats like cottages with access to amazing sailing. locations and racing. They want their email and marine weather forecasts.
I am retired, this is not a revenue generating endeavor it is a hobby. I like to support the "opensource" nature of some projects, (OpenWRT as well for example). I got involved 15 years ago while I was either sailing or travelling in an RV. pfSense systems were set up all over the place as I travelled so I could get good wifi.
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Basically, what might do the trick : after portal authentication, get the IP and MAC, create a static DHCP lease ... because you want a device, if it come back, even days week later on, to have it the same IP as before.
For a static trusted LAN, this is a normal situation, as you 'known' every devices.
A portal is by nature a non trusted network. There will by "DHCP", the device will get 'some' I and that it. Only a static lease can guarantee a fixed IP.The captive portal was written with temporary connection in mind I guess. Perfect for me, when running a hotel (people don't stay for weeks - and if they do, I MAC white list them, but this is very rare)
Or portal spaghetti code changes.
No, I think this approach is perfect, let the IP change, we don't care. Fix it as a self contained (no new calls to captiveportal.inc) enhancement in index.php and from yesterday's testing, it works pretty much perfectly. You can set your cp accounts to expire with the leases or if you prefer set it to what makes sense. If a client is staying a week, give them a week account, a month, a month account, for a 6 month season, a seasonal account that resets time/data quotas daily/weekly or monthly through freeRadius authentication. Simple, independent of Captive Portal ongoing development and they only log a device in once. If I was to print all of the custom code that achieves this in captiveportal,inc and now index.php, it would fit on the front and back of one page at 10pt. Now keeping up with pfSense releases can be tedious, not the code, the testing to make sure it is still OK. Every one of these enhancements should be part of the release, it just makes sense.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
A wild shot :
https://kea.readthedocs.io/en/kea-2.0.0/arm/dhcp4-srv.html#why-is-lease-file-cleanup-necessary
Look what lfc-interval can do for you ?
I should have mentioned that I looked at 11.4 Lease Affinity to dampen the frequency that the IP changes but only to the extent of hours, maybe days if the pool has sufficient redundancy. The Kea manual has an example configuration for 30 minutes of delay before a lease is reclaimed but of course, it is allocated if there are too many requests and the pool needs it. Kea is concerned about the impact this has on performance. I am testing with the default Kea values but will likely do the next stress testing with around 12 hrs of "Lease Affinity" to verify it reduces the reindexing load on new Captive Portal (CP) connections by many of the reconnects occurring within that window. It will never solve the week and month CP accounts, nor will it handle "hybernation" by devices very well as they cease to renew their leases while asleep.
I still feel that the CP should not care about IPs, I am focusing on using just the MAC to verify authentication through the CP. Damn the IP, full speed ahead! To avoid iOS devices hanging on a lease renewal changing IP, I recommend using the RFC8910.php file that checks both MAC and IP and redirects to index.php with a "False" if both don't match.
-
@EDaleH said in Captive Portal not working on iOS devices only (DHCP 114):
Look what lfc-interval can do for you ?
I use that setting for a couple of days now. Purge time : 1 month or 'many seconds'.
The /var/lib/kea/leases files, become bigger every day. 100 K and growing right now.
I've a 'small' setup : LAN with 50 devices, mostly static IPs, and a portal with 20, 30 devices connecting a day, so close to nothing.I'll revert back monday (remove lfc-interval) as see no advantages of using it, except for the fact that it shows a detailed DHCP activity.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I use that setting for a couple of days now.
I think you will find you are using lkea-lfc - Lease File Cleanup process in Kea which is a completely different animal. It handles the time the list of leases are kept in a "log" of sorts. Reclamation still occurs based on it's settings as far as I understand but I do not know if Kea takes the file into account when it allocates new leases, I believe it impacts performance if the file gets too large.
11.4 Lease Affinity is a different "kettle of fish". It is specifically designed to prevent reclamation unless the pool has no new or reclaimed leases available. I got stuck because when I tried to add it I hard crashed 24.11 and had to rebuild. I see rumors that it will be included in the next release of plus. I am hoping it can be applied by interface, i.e. portal.
In the meantime, it doesn't matter, the revised index.php and rfc8910.php files are working as intended but of course without lease Affinity extended beyond the default of 0 seconds, the IP changes frequently when the device fails to renew the lease. I could always increase the 2 hr lease duration default value but the kind of defeats the point in checking that the MAC authorized Captive Portal is working as intended on the lab bench. In fact, I was thinking of decreasing it to work the lab test harder.
Lease Affinity will be essential for current Captive Portal installations to migrate to Kea/Plus 25.01 due to this issue. They may have to set the Affinity value to the duration of the Captive Portal authorization, a 1 day, 1 week, 1 month for example. The problem is: if there are a lot of devices "just looking" and grab an IP, it will tie up a lease from the pool for the duration of the Lease Affinity value. At the end of the day, this mod to change index.php to respect the Mac authorization instead of IP will become essential for a larger number of devices. I have already decided it is time to assume we are going that way. I will still use Lease Affinity to reduce the number of re-indexes of the Captive portal due to IP changes, likely around12-24 hrs so that trips to the store or work or hibernation/sleep, still stay connected. I think we "got this".
(PS: don't forget to use the new RFC8910.php that checks both IP and Mac or the phones will start hanging on the temp browser screen)
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I'll revert back monday (remove lfc-interval) as see no advantages of using it
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I've added this into my kea-dhcp4.conf file (by editi,ng /etc/inc/services.inc) :
I suggest again that you turn on lease affinity in /etc/inc/services.inc instead. It goes just above the "loggers" line in the DHCP4 section of that file, just below where your log file above was placed. I reversed the Kea Doc JSON to the format used in services.inc to save us some time.
'expired-leases-processing' => [
'reclaim-timer-wait-time' => 3,
'hold-reclaimed-time' => 43200,
'flush-reclaimed-timer-wait-time' => 5
],
So far it sets a default for all interfaces but I am hoping we can set it per interface as I will need different "affinity" on each VLAN/CP. Below you see my test set to 12 hrs (43,200 seconds) of lease affinity, i.e. once allocated and expiried (I set mine to 5 minutes) leases are "held" for 12 hrs unless the pool gets full at which time one of them gets re-allocated anyway, I trust it is the oldest one but untested so far. This will insert the JSON in 11.4. Configuring Lease Affinity into /usr/local/etc/kea/kea-dhcp4.conf:
Working as advertised in my lab testing so far. Lease Affinity will permit simple Captive Portal installations to work with Kea without mofifying RFC8910.php or index.php as long as the CP timeout value is shorter than or equal to the 'hold-reclaimed-time". If you have a voucher good for a month for example, you would end up tying up all of the last 30 days worth of leases, even if they did not log into CP. This could depleate the pool but fortunately the documentation suggests it will allocate from the "Affinity Pool" if such a situation happens, hopefully the oldest IP. This will break the login if that IP gets reallocated, there will be two authorized MACS on the same IP in the Captive Portal list. The original RFC8910.php will indicate an "F" based on the IP match, even though it is a different client, and the second login will hang on iPhones and/or show the logout screen. To fix this, run the index.php and RFC8910.php in the two Redmines mentioned above. If you do not want to be playing "whack-a-mole" with IPs in the authorized list of MACS (and the ire-ndexing delays each time an IP changes), use both those test files and the Affinity setting. This looks like the future of CP with Kea, like it or not.
-
Ok, done.
Instead of editing /etc/inc/services.inc, isn't it possible to add :
"expired-leases-processing": [ { "reclaim-timer-wait-time": 3, "hold-reclaimed-time": 43200, "flush-reclaimed-timer-wait-time": 5 } ],to here :
?
Anyway, I've edited services.inc :
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Instead of editing /etc/inc/services.inc, isn't it possible to add :
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
to here :When I looked through the services.inc it looked dicey and I found any error was a hard crash/reinstall. I assume this will be supported in the next plus release so once I got it working on all interfaces, I just needed it working for testing so I stopped playing with it There is no way we go live with 24.11.
BTW, everything expired this morning, I scrambled the connection order and everything sorted itself out. This is now a stable lab test for the next 31 days to confirm the remaining CP logic (multi user/account, time quotas, data quotas, Tunnel ID freeRadius fix, etc are working as advertised in the 24.11 customized captiveportal.inc.)
It is critical that RFC8910.php check both Mac and IP for this to work or iPhones get really confused and all you have is a blank screen when an IP changes and the IP only check thinks it is validated, loads index.php, which thinks it is already validated and tries to load logout or worse yet has a new IP and duplicates it in the CP database on a new login. All avoided with a very simple mod in RFC890.php.
Of course, you can always make the idle/hard timeout in the CPs less than the lease duration in DHCP but the default setup will give a windows machine a 2 hr lease and an iPhone a 24 hr lease for example so either way you are going to have to do a lot to get your CP working properly under Kea. The modified index.php and RFC8910.php make it work out of the box, and most important, treat the CP as a device you get authorized on until you logout, run out of time or run out of data. Then it cleans itself up. Convenient, single login for the life of the authorization.
Being logged out just because the DHCP server is voracious at cleaning up IPs and reallocating them from the bottom up should not matter, but it does with the current plus code.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Or portal spaghetti code changes.
Yes, time to change the CP to at a minimum support a default RFC8910 detecting if captive and loading index.php automatically so that iOS devices at least work as the default. I stumbled upon the Kea docs for RFC8910
and 8.11. Supported DHCP Standards
Version 24H2 of Windows 11 is behaving differently. I think it is respecting DHCP 114. Although it still works with the traditional captive portal URL, it also detects the change in captive status when the IP changes and pops up the Microsoft default url (vendor_url?) once connected and it takes a while before it clears the "action needed" warning. With an iPhone, when the IP changes, index.php loads the logout page in the temp browser. Of course, if the IP doesn't change, both do nothing, connectivity is maintained in either case though. Of course they can only get an updated IP if you are running the proposed (index.php, RFC8910.php) MAC based Captive Portal (CP) authorization suggested in Redmine 15854 and/or Redmine 15904 (Note, cleaned up index.php uploaded for 24.11 Stable to support changing IPs by respecting Captive Portal Mac Authorization as of Dec 17th, 2024)
I will have to experiment further with Windows 24H2 to find out why their page is loaded rather than the logout page but everyone else is embracing RFC8910, it is time for pfSense to do so as well. May as well implement the MAC authorization within index.php/CP as soon as possible to support the fact we can't simply extend the lease times for all CP clients to avoid so many new logins, when the device sleeps or goes out for a coffee, solely to prevent Kea from changing their IP. There is no reason why CP should be a load that compromises Kea's ability to efficiently assign IPs. Outside of the current index.php implementation, CP checks the MAC anyway, index.php should respect that as well and so should RFC 8910, i.e. DHCP 114, prioritize the MAC to determine if captive or not.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
I've edited services.inc :
I have noticed that once Affinity is running, not only does it protect the IP/Mac Pair until it meets the affinity duration after disconnecting, it also does not re-assign that IP again. Instead it keeps counting up with ever increasing IP numbers, even if it is the same MAC that had an IP before, and IP that is now available. It appears that it will use up the pool before re-assigning any of the IPs that were issued while affinity was turned on. Simply turning on Affinity of any duration appears to enable this behavior. This is very beneficial to Captive Portal when running the modified index.php/RFC8910.php files in Redmine 15854 and/or Redmine 15904. The default index.php and original release of RFC8910.php will pop up a login screen and once logged in, create a new authorized entry in the CP DB with the new IP, resulting in two authorized IPs for the same Mac unless the idle/hard timeout is less than the lease duration. iOs devices using the original RFC8910.php may show a blank temporary browser screen.
Without Affinity enabled, the default Kea assignment is to reuse the IPs from the lowest number upwards within seconds to a couple of minutes of the IP expiring. In this case the original index.php and RFC8910.php files will try to use the CP DB entry of that IP and fail if it was previously used & authorized (but not yet timed out), thinking it is already logged in, showing either a blank screen or a copy of the logout page. No internet access is possible until you manually delete the IP from the CP DB before connecting a device with the same IP of one already authorized.
Using the original RFC8910.php, with the recent release of Windows 11 24H2, it is popping up the RFC8910 JSON file with no login screen. The JSON is modified to go to a Microsoft URL rather than the one specified in the Captive Portal configuration. Clearly Microsoft is experimenting with DHCP 114. The modified RFC8910.php mentioned above in the Redmines does not fix this unless you are also running the suggested index.php which will update the CP IP address and thus maintain the connection. There is an issue though with Windows because it loads the connection test screen (msn) instead of the logout screen. I expect Microsoft will fix this though as it works fine with IOS, the folks who created DHCP 114 in the first place.
To make this post complete: Even though the Affinity period has expired, Kea keeps allocating IPs with ever increasing numbers from the DHCP pool until the pool is full. It then starts over at the lowest number and works upwards, respecting existing leases and any within the affinity protected time period that may not have expired yet. This behavior suggests that Captive Portals will be more reliable with Affinity enabled, independent of the duration.
With further testing, Affinity settings do not survive a power outage and Kea will reassign leases from the lowest number upwards for all expired leases, affinity durations are not respected unless the lease is still active when the power resumes. Unless DHCP leases are extended to very large durations (and idle timeouts in CP set to less than that value), this will necessitate the use of the modified index.php and RFC8910.php files for all captive portals so that reassigned IP/MAC pairs are managed within index.php to respect he MAC as the "authorization" under CP. The issue with long duration leases is the possibility of depleting the DHCP pool for that interface, despite the fact many devices are no longer connected. By authorizing based on MAC, the list of authorized CP users can even exceed the size of the address pool.
-
@Gertjan said in Captive Portal not working on iOS devices only (DHCP 114):
Anyway, I've edited services.inc :
I assume you are now aware of the fact Kea's Affinity memfile does not survive a reboot and will loose expired leases that still have affinity "protection". Netgate has raised Redmine #15934 to attempt to address this and other lease expiry concerns but so has the Kea development team. See the link in the Redmine.
The Kea development team have scheduled this for possible correction in V 3.0 which is slated for an April 2025 release. There is no certainty that they will include it.
Until Kea supports Affinity surviving a reboot, using Kea with Captive Portal is very risky as a reboot will likely scramble the IP/MAC assignments unless the devices reconnect in the exact same order. We will have to use ISC until then, religiously have idle timeouts less than lease duration (and the frequent re-logins that implies), or incorporate a MAC Captive Portal authorization scheme like that proposed in Redmines 15854 or 15904
This suggests that we are unlikely to have a built in solution for Captive Portal ISC equivalent support under Kea at the next plus release (25.03?) or until Netgate incorporates Kea 3.0 into the pfSense plus and CE releases.
