New 4096 MB traffic limit on freeRadius User is not Warranted.

[EDaleH]

All the setting of an “Amount of Download and Upload Traffic” limit when creating/editing a freeRadius user appears to do is to create a max-octets-user file in /var/log/radacct/datacounter/forever (or daily, weekly, monthly) and write the equivalent number of octets to it (1048576000/GB). From that point on, freeRadius tracks data usage based on interim vrs stop/start accounting settings.

I believe the decision to limit “Amount of Download and Upload Traffic” to 4096 GB is applying flawed logic related to the 4096 bytes limit on the packet itself. The Amount of Download and Upload Traffic" uses only two X 32 bits of the accounting packet, one for upload and one for download amounts in that accounting interval (or per minute for stop/start).

Yes, each of these 32 bit download/upload items is itself limited to 4096 MB, but only for the accounting period itself, not the entire life of the “Amount of Download and Upload Traffic” setting.

I calculate that it is impossible to achieve this throughput in a 10 minute accounting period unless the bandwidth is above 60 Mbps for each of upload and download. If you want to limit the “Amount of Download and Upload Traffic”, you need to calculate that limit based on the length of the accounting packet, the bandwidth setting and duration of the user account running at that speed. I don't believe that is realistically possible so I suggest you change the absolute restriction on “Amount of Download and Upload Traffic” to a warning triggered by a bandwidth setting above 60 Mbps or blank, as it works fine. I have bandwidth set to 50 Mbps and 500 GB, working perfectly under 23.01RC. I simply change the value in max-octets for each user.

I hope you reconsider this limitation, it is very restrictive and unnecessary.

Thank You,
Dale

[stephenw10]

You may want to add comments here: https://redmine.pfsense.org/issues/13843

[EDaleH]

@stephenw10 said in New 4096 MB traffic limit on freeRadius User is not Warranted.:

You may want to add comments here: https://redmine.pfsense.org/issues/13843

I looked at 13483 before sending the above in this forum. It appears to me that pfsense-Max-Total-Octets may sent by freeRadius to pfSense in the accounting packet but it is not pfSense that calculates the total Traffic value, it is freeRadius so all that matters is what the capacity of the return parameter is when pfSense reports the traffic use since the last accounting update. That is what I am referring to above, the capacity of the data increment returned to freeRadius, most likely in a Stop/Start, once per minute accounting update.

It may very well be true that the pfsense-Max_Total-Octets is overflowed but outside of memory contamination or whatever, it doesn't appear to be used for anything related to the “Amount of Download and Upload Traffic” limit outside of seeding the initial freeRadius max-octets file..

freeRadius cumulates accounting packets by user into either one used-octets-user-uniqueUserID file per user under interim updates, summing them to used-octets-user every time one user logs out, or if Stop/Start is selected, all accounting updates are immediately summed to used-octets-user, the preferred method to keep track of all data used by all users at a time. That is why it exists I believe. I understand that under the interim captive portal setting, the user won't be logged out simply because the test is against the used-octets-user file relative to max-octets-user file. As the cumulation of interim updates is in a separate used-octets-user-uniqueUserID file, freeRadius won't disconnect the account until that account logs out or times out.

I am only assuming that the 32 bit limit is also present for the used traffic values for down/up in the pfSense->freeRadius accounting packet and thus have the same 4 GB limit, my most likely explanation for the change to the max “Amount of Download and Upload Traffic” 4GB limit now applied to the freeRadius user manager under 23.01 RC. As 13483 is still open, I see no connection between the two as I would have expected a comment there about the "temporary" 4 GB max enforced in the freeRadius user manager so I am assuming they are unrelated.

In way of an update, I have now processed a 100 GB file limit for one user under 23.01RC without incident and with excellent accuracy based on the NIC traffic properties dialog. I created that user with a 4095 MB limit and then changed the value in freeRadius /var/log/radacct/datacounter/forever/max-octets-test to represent 100 GB, 104857600000. used-octets-test correctly incremented until it reached that value and then the user was disconnected, I assume by freeRadius when the used-octets-user file overflowed the max_octets-user file, a fact confirmed by comparing the contents after the forced logout.

The simple fact is that a 4096 GB limit does NOT exist within the window of operation described above.

[EDaleH]

@edaleh For those of you wrestling with this business killing 4 GB down/up traffic limit, it is important to further note that as of 23.01, if you edit a freeRadius user file, you will be forced to set this limit to 4095. That limit is then written to the max-octets-user file but it is also written to the package xml file and will be reloaded on the next reboot or freeRadius service restart, resetting the user to 4095 MB limit, for most users, immediately logging them out.

In order to maintain a down/up traffic limit above the arbitrary 4 GB limit, you must download the package XML file through Backup/Restore, edit it with a simple text editor like notepad and change the 4095 value in the XML to your new value, 100 GB is 100000 in the XML file and in the max-octets-user file it is (1048576000/GB). for eg. 104857600000 is a100 GB. Then use Restore package in Backup/Restore to install it with your updates. The XML file will update the max-octets-user file for you and as long as you don't edit the user in the freeRadius user manager, it will stay that way. The freeRadius user manager will also correctly show the changes you made but will revert on an edit. Basically we are back to using XML to create users, the user manager is now unusable. It is preferred with this release to set the Captive Portal to Stop/Start freeRadius. Yes, a 100 GB limit under Stop/Start is still working fine and logs out the user when the 100 GB limit is reached! There is no "in practice" 4095 MB functional limit.

Further testing on the Interim Captive Portal setting demonstrats it is not reliable unless you stick with the freeRadius managed cumulation of use on an actual logout. My tests post 23.01 for the interim setting are now showing a "suspended animation" of the login when I manually move the used octets from the interim file to the used-octets-user file. I have not seen this before and have been updating this way for years. The captive portal shows the user logged out, I simply refresh our custom dashboard which checks the sessionid, returning a valid session, despite the fact captive portal shows it logged out, after that the session continues and captive portal shows the user logged in again with the next accounting packet, despite the fact that from the user's perspective all that happened was their internet was interrupted and suddenly started working again. The collateral damage impact of this 4 GB traffic limit on Captive Portals that use freeRadius authentication with Data Quotas is very substantial and destructive to the point it is pretty close to unusable.

[EDaleH]

@edaleh I believe I can now explain most of this behavior.

redmine 13418 fixed an issue with freeRadius where freeRadius did not track data at all, it always returned 0's. This issue showed up in release 22.05. The fix to that implemented in late December 2022 was for Captive Portal to only track data used between interim packets, especially in the stop/start freeRadius setting. It was in implementing that fix that this 32 bit integer problem became redmine13843. Because the fix to 13418 resulted in only tracking the data consumed between reporting packets, it ignores the max-octets-user value in the packets and leaves it up to freeRadius to compare the cumulation of data consumed as represented in used-octest-user. Thus it works fine under Stop/Start freeRadius, as freeRadius likely does not use a 32 bit unsigned integer to do so, it compares the actual value in octets in those two respective files.

Now, once either interim updates, or re-authenticate every minute is selected, a different logic is applied to assessing total data consumed. pfSense/Captive Portal is/are actively managing the total data consumed and comparing it to the max-octets-total value in the packet, a value that overflows to a random, but consistent value when represented as a 32 bit unsigned integer. This is demonstrated by the fact 100 GB always terminates at 1.8 GB of total throughput. Now freeRadius still has that user authenticated as it just checks the relative max to used values in the above and the used value has not exceeded the max value yet so it has not sent a false authenticate request in the handshake packet yet. It is the captive Portal that initiated the logout in these cases and thus the freeRadius interim packet value for max-total-octets is used to disconnect/logout the user. That is why I can simply refresh my logout tab and the connection continues (releases the suspended animation above) as Captive Portal has reset its counters on the logout. It will work for another 1.8 GB and repeat until such time as the cumulative totals in used-octets-user has exceeded max-octets-user and freeRadius sends a "false" back in the interim packet for authentication and captive portal logs them out based on the freeRadius authentication status. At that point, refreshing my logout tab results in Captive Portal loading the login screen because freeRadius is the authenication authority.

The fix to this is for freeRadius authentication to dominate, do NOT use the max-octets-user value sent in the interim update packer in pfSense/Captive Portal , instead, do the same as you do for the stop/start freeRadius, send only the data consumed since the last accounting update packet. Let freeRadius cumulate that value in it's used-octets-user file and let it disconnect when the max-octet-user value is reached in freeRadius. As long as pfSense/captive Portal sticks to tracking from packet to packet and leaves the job of tracking the total data Quota to freeRadius, it will work perfectly.

This logic also fixes the interim update option, even for multiple simultaneous users on one user account. You will still have the issue of the users having to log out before the data is counted (i.e sumed into used-octets-user) but that has been there for years and is by design. My workaround of having a cron job do interim sums to used-octets-user was working fine for that in the past.

Please, Please, Please let freeRadius do it's job and get out of its way, this is not an issue with the unsigned integer, it is an issue because you take over tracking total data usage and make the decision to logout yourself while ignoring the fact freeRadius is still validating that user as authenticated. Your diagnosis of the 32 bit overflow is correct but your treatment is killing the patient.

Lastly, for the record, all updates to used-octets-user are done once a minute, independent of the freeRadius interim update setting (default 10 minutes). This is not an issue unless it creates excessive traffic on the interface. It is done once a minute independent of the reauthenticate every minute setting as well.

[EDaleH]

@stephenw10
Please see my prior descriptions made in this post of what I believe is the cause of this serious restriction in usability. Also see https://redmine.pfsense.org/issues/13418 (raised by me) where most of this started. In fact this problem is indirectly caused by the solution to it. This was working "better" prior to 22.05.

lines 708 thru 711 in captiveportal.inc demonstrates the one second delay limitation on functional total number of simultaneous users:

if ($cpcfg['reauthenticateacct'] == "stopstartfreeradius") {
/* Need to pause here or the FreeRADIUS server gets confused about packet ordering. */
sleep(1);
}

As Stop/Start freeRadius is run every minute, this programmatic delay effectively creates a max simultaneous user connections to the freeRadius authenticated portal(s) of 60 users. Many have also complained that each user, checked/reauthenticated each minute results in 2 lines in the system log for each logged in user. This too must be expanded upon to enable support of more authenticated users. As this is the only remaining pfSense to freeRadius setting that can exceed 4 GB per user, it is now a serious limitation.

This logging feature must be available in the gui to enable or disable it as it can contribute to a syslog overflow in seconds.

Most important, the frequency of stop/start freeRadius must be accessible from the gui, I suggest every 600 seconds would be more functional as the default. Currently it is 60 seconds, or once per minute.

Reauthenticate every minute would benefit from this consideration as well. The default should be 600 seconds in my option and user configurable.

[EDaleH]

@stephenw10
I believe I can finally put this project in perspective for all of us.

1. The reconciliation of the 32 bit unsigned int overflow so that pfSense knows the actual value of max-octets-total in freeRadius will enable you to track a "single session" against the correct pfSense-Max-Total-Octets in pfSense/Captive Portal. From that perspective this is a valid improvement.

2. The above achievement is flawed in that total traffic in pfSense/Captive Portal is only maintained for the active session or duration of pfSense powered up state. This means it tracks the traffic data on a single user session between session start and either logout or in this case a traffic quota based on a valid freeRadius supplied pfSense-Max-Total-Octets. A reboot or power loss will result in the pfSense Captive Portal session surviving if "Preserve connected users across reboot" is checked, i.e. True, but the traffic quota cumulated to date in pfSense will be lost on a reboot or power loss and restart from 0. This means pfSense is not capable of tracking a single user session against the freeRAdius max-octets-user value because the traffic to date value is not persistent. But... it is persistent to the nearest accounting interval on freeRadius because it uses a physical storage on a hard drive file, which survives both a login/out session, and reboot, be it deliberate or power related. freeRadius typically autosums all open user sessions to the main used-octets-user file on a restart. pfSense Captive Portal will force the creation of new used-octets-user-uniqueID files on freeRadius after the restart at the appropriate interim accounting interval.

3. If the captive portal is set to "Multiple" under Concurrent user logins, there will be multiple active sessions for the same user account under pfSense. Each separate session is checking that session's traffic against the pfSense-Max-Total-Octets value so it will not logout either user until that individual user reaches the pfSense-Max-Total-Octets value which is not the intent at freeRadius. On the freeRadius side, it will cumulate all users together into used-octets-user based either on stop/start or if set to interim, based on the interim interval value set on freeRadius, it will create a used-octets-user-uniqueID value for each user which are only summed to the main used-octets-user file when each individual user logs out. Although this is by design on the freeRadius side, it could be considered a bug because a logout will not be initiated by freeRadius until the used-octets-user exceeds the max-octets-user setting. Thus, neither pfSense, nor freeRadius will log all users out unless something intervenes and forces summation of all the used-octets-user-uniqueID files into the used-octets-user file that freeRadius uses to determine if the quota was breached.

4. Yes, this now accurate pfSense-Max-Total-Octets value tracked against a single session user for an uninterrupted session will force a captive portal traffic quota related logout, which in turn will force freeRadius to sum all the uniqueID files into the main used-octets-user file but it will not do so in captive portal: "multiple" Concurrent logins mode as it is tracking each individual user against pfSense-Max-Total-Octets instead of summing all the authenticated users on that user account together first. Thus "multiple" is not supported.

5. freeRadius has all it needs to do both multi users per account and session interruption tracking if pfSense was to force the summation of the uniqueID files into the used-octets-user file on a regular basis. This is why stop/start freeRadius solves this problem and works perfectly. Unfortunately that solution is not elegant and has serious scaling issues.

6. As we already have a working solution in stop/start all we need to do to get interim to use freeRadius to accurately invoke pfSense-Max-Total-Octets is to regularly fire one stop/start accounting packet session. I suggest when you select interim, you add a gui option to set a "cumulate every ??? seconds" option and point out it should be at least as large as the freeRadius interim setting (default 600 seconds). Once this is done, pfSense Captive portal in cooperation with freeRadius through the interim accounting period set on freeRadius will enforce the pfSense-Max-Total-Octets in all scenarios.

7. As I said in the outset, the correct determination of the pfSense-Max-Total-Octets is irrelevant other than if the above is implemented, it will fix the failure of either freeRadius or pfSense to accurately track pfSense-Max-Total-Octets when interim sessions are still active within this "powered session".

8. in the above scenario. reauthenticate every minute does not appear to have purpose other than to force freeRadius to check max-octets-user against used-octets-user once a minute. As it does NOT force the cumulation of the multiple used-octets-user-uniqueID files into the used-octets-user file, it is not a tool to address the multiple user under interim setting issue. If it did, it would also be a solution to the tracking of multiuser traffic.

The above assumptions have been lab confirmed on 23.01RC 08 Feb 06:00 version, in both single and multiuser mode.
See #13843 & #13418 for historical information