Vouchers getting Expired before remaining Time
-
Hi!
I tried to investigate this a little deeper. Here is what I suspect: If voucher rolls are created running 2.4.x the upgrade to 2.5.x deletes the db-files in /var/db. On the first login of an existing voucher, the respective db-files will be re-created, but with wrong activation time stamps. (No evidence. just a thought!)
What helps:
Delete all voucher rolls. Upgrade to 2.5.x. Create all voucher rolls again. Check if all corresponding db-files are present in /var/db.
However, there is a BIG disadvantage: All voucher history gets lost, especially vouchers which were expired will be valid again unless you manually expire them!!!
BR,
Volker
-
You might be right
I can't / didn't test an upgrade, as I'm on 2.5.1 already. I'm not using voucher neither.
DB files ate filled with the time info, when vouchers are used or activated.But :
With the option set, DB files are backed up ....
-
I have Fresh Install and everything created from Scratch. I have two site still running development version of 2.5 and there is no issue
I planned to upgrade them to 2.5 stable but facing problem with vouchers early expiring in two new deployments ( Built on FEB-2021 ). so Postponed it.
There is some kind of bug unable to trace yet.
-
@wazim4u said in Vouchers getting Expired before remaining Time:
I have Fresh Install and everything created from Scratch. I have two site still running development version of 2.5 and there is no issue
What is the actual snapshot date?
Could you provide more information and create a bugreport?
https://docs.netgate.com/pfsense/en/latest/development/bug-reports.htmlMay be related to https://redmine.pfsense.org/issues/97
-
Voucher system working perfectly on this snapshot
2.5.0.a.20210107.2142 ( Development )
up to 2500 captive portal users renew monthly basis and there is not a single issue reported about voucher's early expiry.
-
Aaargghhh. It happended again. Even the way I though will work (delete vouchers, upgrade to 2.5.1, create vouchers again) is failing. Around 280 vouchers expired way before the end of its validity period. And we have a lot of happy customers on our hotline ... :-(
The system was rebooted due to a power outage and this reboot may have caused the corruption of the voucher db. Just a thought ...
I am going to roll back all the respective sites to 2.4.5 now. This bug is severe - at least if vouchers are being used.
Regards,
Volker
-
Please create a bugreport:
https://docs.netgate.com/pfsense/en/latest/development/bug-reports.html -
Done!
https://redmine.pfsense.org/issues/11894
-
Hello!
Are you using RAM disks?
John
-
@refugeesonline said in Vouchers getting Expired before remaining Time:
Aaargghhh. It happended again. Even the way I though will work (delete vouchers, upgrade to 2.5.1, create vouchers again) is failing. Around 280 vouchers expired way before the end of its validity period. And we have a lot of happy customers on our hotline ... :-(
The system was rebooted due to a power outage and this reboot may have caused the corruption of the voucher db. Just a thought ...
I am going to roll back all the respective sites to 2.4.5 now. This bug is severe - at least if vouchers are being used.
Regards,
Volker
What is your complete configuration?
Are you using RAM disks or Captive Portal HA sync?
Please provide more info -
I can prove now that there is serious bug in system . given below are details for two vouchers showing expired today.
Voucher : 976522494 ( Expired after 2 days )
May 6 01:14:49 logportalauth 14356 Zone: smg_camp - FAILURE: 976522494, 6c:c7:ec:4b:11:c3, 172.17.228.225, voucher expired May 6 01:14:49 logportalauth 14356 Zone: smg_camp - 976522494 (6/50) already used and expired May 4 04:28:01 smg logportalauth[24061]: Zone: smg_camp - Voucher login good for 516420 min.: 976522494, 6c:c7:ec:4b:11:c3, 172.17.228.225
Voucher : 7929393872 ( Expired after 6 days )
May 6 08:04:05 logportalauth 61082 Zone: smg_camp - FAILURE: 7929393872, 2e:64:bd:c8:d3:4f, 172.17.120.210, voucher expired May 6 08:04:05 logportalauth 61082 Zone: smg_camp - 7929393872 (6/18) already used and expired Apr 29 12:03:47 smg logportalauth[46095]: Zone: smg_camp - Voucher login good for 517452 min.: 7929393872, 52:62:b0:80:78:00, 172.17.230.152
-
@serbus
Yes, /tmp and /var are RAM disks -
@viktor_g
Yes, /tmp and /var are RAM disks. No Captive Portal HA sync, only local dbs.What I noticed: System has been upgraded, all voucher rolls were deleted by the admin. No db-files in /var/db. Then the admin created 3 voucher rolls and the corresponding db-files are present in /var/db.
Then reboot. All of the sudden 286 vouchers expired (which is weird as only around 70 were active at that time). In /var/db only the db-file for one roll (the one containing active vouchers) is present - the others disappeared. Even more strange: All remaining active vouchers were set back to original validity (43.200 mins). It looks like the reboot has had a severe impact on the /var/db contents ...
Two things are a pain in the neck: Active vouchers are expired way earlier before the nominal validity (consequently, our hotline has a lot of calls). Secondly, the remaining vouchers start again with full validity. (consequently, we are giving access for free as active vouchers are kind of reset to square one). -
Tried to switch off RAM disks.
Result: Captive Portal does not work anymore ....May 6 15:11:51 logportalauth 360 Zone: zone_01 - Trying to modify DB returned error: table captiveportal has no column named traffic_quota
May 6 15:11:51 logportalauth 360 Zone: zone_01 - Trying to modify DB returned error: table captiveportal has no column named traffic_quotaI am closed to give up on this ....
-
@refugeesonline said in Vouchers getting Expired before remaining Time:
Yes, /tmp and /var are RAM disks
and then :
@refugeesonline said in Vouchers getting Expired before remaining Time:
It looks like the reboot has had a severe impact on the /var/db contents ...
True : RAM disks are lost during reboot.
edit : I presume that's a common knowledge .... Am I wrong ?When I read all this :
I do not have the confirmation that the captive data base files are retained.
Data (files) that are kept between reboots are mentioned !
The captive portal database files are NOT mentioned ....
So ..... I vote for : they are lost.
edit : the manual confirmed this.There is probably an issue, but using RAM disks and the using the captive portal - and using vouchers == asking for another issue.
@refugeesonline said in Vouchers getting Expired before remaining Time:
All remaining active vouchers were set back to original validity (43.200 mins).
Yep - this seems normal to me. These data base files - per portal and per role - are created when at least one of the vouchers of a roll is activated. The voucher code and the time stamp of their activation is recorded.
Loosing these file makes the voucher 'available' again with the original duration. -
@gertjan
I do not understand your comment. We are operating 170 sites and the problems I described ONLY appear on systems running 2.5.1 .... -
@refugeesonline said in Vouchers getting Expired before remaining Time:
and the problems I described ONLY appear on systems running 2.5.1 ...
== the systems you upgraded. This will zap de ram disk.
Or : the other system never reboot ( ? ) .... so the RAM disk stays valid.Also :
See https://github.com/pfsense/pfsense/commits/master/src/etc/inc/voucher.incThe last time this file changed was may 2020. That's before 2.4.5-p1 ...
And that code decides if the log message "already used and expired" is thrown out :May 6 08:04:05 logportalauth 61082 Zone: smg_camp - 7929393872 (6/18) already used and expired
I've created some vouchers and roles myself, and play with them.
I can't really create vouchers with a "517452 minutes " == one year ( !!!! ) and test them.Btw : the voucher code could change tomorrow, and break the bit-nitting** of of the expiration test, so making vouchers "517452 " will always give troubles. The unknown factor is just "when".
** Just look - for the fun - at the function voucher_expire() and voucher_auth() in
/etc/inc/voucher.inc -
@gertjan said in Vouchers getting Expired before remaining Time:
@refugeesonline said in Vouchers getting Expired before remaining Time:
Yes, /tmp and /var are RAM disks
and then :
@refugeesonline said in Vouchers getting Expired before remaining Time:
It looks like the reboot has had a severe impact on the /var/db contents ...
True : RAM disks are lost during reboot.
edit : I presume that's a common knowledge .... Am I wrong ?When I read all this :
I do not have the confirmation that the captive data base files are retained.
Data (files) that are kept between reboots are mentioned !
The captive portal database files are NOT mentioned ....
So ..... I vote for : they are lost.
edit : the manual confirmed this.There is probably an issue, but using RAM disks and the using the captive portal - and using vouchers == asking for another issue.
Please create a feature request for storing captive portal / voucher data:
https://docs.netgate.com/pfsense/en/latest/development/feature-requests.html -
@gertjan
None of the reporters of this "bug" (wazim4u and refugeesonline=me) is using a validity period of one year, we are using 43.200 mins = 30 days.
I am sorry I don't have the time to look in GitHub (although I would love to be able to contribute), I am just an operator of 170 appliances.
And concerning reboots: All our sites are refugee camps, power outages (and consequently reboots) are daily business. We never had this problem before. I can't judge whether this is a bug or a configuration problem. All I am saying: I did NOT change the configuration, I just did an upgrade to 2.5.1. and ran into problems. And I had to notice that the configuration had to be changed, too. 2.5.1 has a totally different handling of concurrent logins (which I figured by accident as many customers were complaining). So my conclusion is: For my case the upgrade caused the problem. I will have to physically visit al lot of sites and do the roll-back to 2.4.5. (Else, I am open for any advice how to do this remotely ....) -
Hello!
A cursory look through some of the code indicates that some voucher info is stored in the config during a graceful shutdown/reboot. Even that wont help with ram disks, or if there is a power loss.
Check on the requirements that forced you to move from 2.4.5 to 2.5.x and see if any of those can be worked around.
Try building a fresh 2.5.x without ram disks and see how it works.
Moving back to "no ram disks" after initially using them might work with a little scripting.
Using the current rc.backup* and rc.restore* scripts as a template for manually saving and restoring the /var/db data...
While ram disks are still on...
rc.backup_db#!/bin/sh # : ${DBPATH:=/var/db} : ${CF_CONF_PATH:=/cf/conf} : ${RAM_Disk_Store:=${CF_CONF_PATH}/RAM_Disk_Store} # Save the logs database to the RAM disk store. if [ -d "${DBPATH}" ]; then echo -n "Saving DB to RAM disk store..."; [ -f "${RAM_Disk_Store}/db.tgz" ] && /bin/rm -f "${RAM_Disk_Store}/db.tgz" if [ ! -d "${RAM_Disk_Store}" ]; then mkdir -p "${RAM_Disk_Store}" fi /usr/bin/tar -czf "${RAM_Disk_Store}/db.tgz" -C / "${DBPATH#/}/" echo "done."; fi
Turn off ram disks, reboot, manually run...
rc.restore_db#!/bin/sh # : ${CF_CONF_PATH:=/cf/conf} : ${RAM_Disk_Store:=${CF_CONF_PATH}/RAM_Disk_Store} /usr/bin/tar -xzf "${RAM_Disk_Store}/db.tgz" -C / 2>&1
Restart captive portal and see if it picks up your newly restored /var/db
The standard system rc.restore_ramdisk_store might even catch & restore the db.tgz you created if it runs when rams disks are off (???).
This is a highly experimental process with lots of unknowns. Use at your own risk.
Maybe someone with more/better knowledge of the startup/shutdown and ram disk operations could chime in.John