Captiveportal db not updating?

MortO

The update seems to fix the problem, though now it seems the captiveportal is no longer re-authenticating against the RADIUS server.
Apart from the initial logons there are no further logs in the RADIUS server about any given user, where before they'd get logged once every minute.

I'm not sure where the entry to run the prune_old() function once a minute lies, it seems to be in the php.core file according to grep, but I'm assuming there is a task somewhere perhaps in the scheduler supposed to run this. I'm not sure, but I believe I saw a scheduler entry about the pruning earlier, but can't remember where I found them.

Closest I've found is under Firewall there's a schedules menu, but this list is empty at present.

Is i safe for me to manually make a PHP file which includes the captiveportal.inc file and call the prune function manually? Or would any errors or such get logged somewhere? I've checked /var/log but can't seem to find anything relating to pruning or such?

Doing a cat /var/run/cp_pruneold.pid and using that pid number in ps -aux | grep <pid>shows it's running with minicron. When trying to run the /etc/rc.prunecaptiveportal manually I get

Fatal error: Error converting Address in /etc/inc/radius.inc on line 210

Corresponding line is

return radius_put_addr($this->res, $attrib, $value);

On a bit closer poking I think the problem lies in captiveportal.inc
using $no_users = count($cpdb); was fine when you used a regular indexed array, but the change to the read_db has changed it so where before you were doing $cpdb[$i] you'd use a number from 0 to x, now it's indexed with the keys, and not the number, thus calling $cpdb[0] (first element) gets you a reference to a non existant element.

Need to read through the read and write code to figure out how the keys are handled so I don't break anything in an attempted fix. A quick and dirty fix would be

 $indexes = array();
foreach($cpbd as $cpKey => $cpVal) {
  $indexes[] = $cpKey;
}
$no_users = count($indexes);

and then replace $cpdb[$i] with $cpdb[$indexes[$i]];

–-

Further pondering
Quick, not nescesarily correct fix would be


$keyArr = array();
foreach($cpdb as $key => $value) {
  $keyArr[] = $key;
}
$no_users = count($keyArr);
//for ($i = 0; $i < $no_users; $i++) { 
for($j = 0; $j < $no_users; $j++) {
  $i = $keyArr[$j];

This should remove the need for any further alterations in the prune function as $i will now contain the key to the data again.
I am not certain if things will work correctly with the unsetindexes.

Also looking over the altered write_db function I'm assuming the prune_old function doesn't alter any database rows? If it does the data won't get written as it seems to favor the data in the read_db done during the write_db call. Is this a completely wrong assumption?</pid>

MortO

I've modified the captiveportal.inc file fixing up the code like I suggested, and implemented a sort of optimization to avoid multiple dictionary lookups for each reference to the current row.

Since the forum doesn't allow .inc uploads I've renamed it to .txt

I've put it on our server and it now seems to prune automatically again without problems.

captiveportal.txt

eri--

It should not do that so please check again.
I put this fix in for the referenced entries https://rcs.pfsense.org/projects/pfsense/repos/mainline/commits/3e5c0ab797c65ac4833dfb75049a3e5dd396db74

MortO

Will try the updated snapshot this afternoon then.
My temporary implemented fix as was attached did get reauthentication running again, but I'm seeing entries in the firewall which aren't in the .db file again. Will report back after testing if the new snapshot fixes the problem.

MortO

Right, tried to do an update with the autoupdater, which seemed to go well, except the captiveportal.inc file is now back to before that last edit again (e.g. using ints for trying to access the data).

I guess I'm a bit stupid, but haven't been able to find a download link for that version of the captiveportal.inc file, so I quickly mirrored my kludge back together.

Also with a bit of a lookover I think I might've found why we're still experiencing users getting dropped out of the database.

It seems captiveportal_write_db only re-reads the .db file (and thus any changes other processes might've done) if unsetindexes isn't an empty array. If the pruning function doesn't remove anyone (which is kinda likely considering it passes once every minute) it seems to behave the same way as before with overwriting any changes made with the memory cached version.

To combat this I've made a small change to the prune function on our server again to do thus:


if (!empty($cpdb)) {
  captiveportal_write_db($cpdb, false, $unsetindexes);
}

Still not entirely sure nothing is altered during the prune function other than removing, but the way write_db works seems to discard any changes if there's any deleted indexes, so skipping the write entirely if there's no deleted rows shouldn't prove that much more of a problem.

I'll report back how things go with this approach.

Any pointers on how to retrieve the captiveportal.inc file with your changes would be nice to have too, since any small hacks I make in an attempt to narrow down the root of the problem get overwritten when autoupdate is invoked.

eri--

From console you can use pfSense shell or just go to rcs.pfsense.org and browse the repo.

MortO

The if condition I mentioned, wrapped around the captiveportal_write_db call seems to work. Halfway through a normal day we've got 1402 users online without anyone having dropped out of the database. Good difference from yesterday with 1200 users online and 130 missing from the database file.

The edit was done on the same captiveportal.inc file I uploaded before, as I get an error message when trying to download captiveportal.inc from the tree.

eri--

Well it is now on latest snapshots so you can upgrade or just download an update file.
Extract it and go get the file uploaded manually to pfSense install.

Good to hear it works as it should now.

MortO

Updated to the snapshot last night and reauthentication seems to be running again with the snapshot code.

Unfortunately it's also started dropping logins out of the database again.

Can I once again suggest you alter captiveportal_prune_old() from ending with

captiveportal_write_db($cpdb, false, $unsetindexes);

to ending with


//Write database again if we have deleted anything, as an empty $unsetindexes will cause the database to rollback to when this function was called
if(!empty($unsetindexes)) {
  captiveportal_write_db($cpdb, false, $unsetindexes);
}

eri--

I actually put another solution in.
Please test that.

MortO

I shall update monday and report how it works out. Thank you very much.

MortO

yup, that definately seems to have fixed it. No more spilling out of the database.

Thank you very much for all your help :-)