Enabling MIM causes Authentication Error for voucher based logins in Captive Portal
-
@rlinnemann said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
glad it sounds like you've worked around it, but my spidey sense is still tingling here.
Having identified a config version as the cause is the conclusion from my perspective.
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
24.11 actually uses 23.6.
The chart says config ver 23.3 for Plus 24.11 but I am quite satisfied that all restores to 24.11 must be done by restoring from or through (if it is CE 2.7.2) Plus 24.03.
Attempting to answer your questions is what lead to the final diagnosis here, it is comforting to know that existing installations have an upgrade path that includes MIM.
-
@EDaleH said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
The chart says config ver 23.3 for Plus 24.11
Yeah that page needs to be updated when 24.11 is released but currently it's using 23.6.
So it could be failing to upgrade the config at import....
-
Doesn't look like is it though. The search continues...
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
doesn't look like is it though. The search continues...
Well, your search may continue for the "fix" but the cause is clearly identified.
This morning I built a 24.03, restored the CE 2.7.2 backup onto it, tested it worked and then Backed it UP. I then upgraded it to 24.11 Beta and it does not display the authentication error when MIM is turned on.
Next, I built a new 24.11 Beta and restored that 24.03 backup onto it and voila! the authentication error is there every time you turn MIM on. Conclusive proof that the only way to get a stable 24.11 Beta in my case is to go through 24.03 and do a GUI upgrade.
-
Right which really seems like config upgrade issue at restore. It's not doing something that is done at system upgrade.
But it's more complex than that because I tried exactly that with a basic config and it still worked fine.
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
But it's more complex than that because I tried exactly that with a basic config and it still worked fine.
I have been unable to duplicate it with a fresh install either. This install is as complex as it gets for me and runs flawlessly. I am just trying to ensure it continues to do so under 24.11, Kea and MIM. Lots of lab testing left!
-
@stephenw10
As part of my testing of 24.11 Beta, I had a step to do a backup, fresh install and restore to confirm functionality. I moved that to the top of my list due to the restore issues I had encountered. I can confirm that a backup of a working 24.11 install (i.e. one that came through a 24.03 upgrade) will restore to a fresh 24.11 and work properly without displaying the authentication error.That suggests the format/processing of the backup config file (V23.3) is by far the most likely cause.
-
Exactly it appears that when you import the 24.03 config into 24.11 it's not being upgraded correctly. But only when the config is sufficiently complex.
Are you able to compare a failing config in 24.11 with a working one?
That looks identical in my testing here but clearly something in your config is hitting an issue.
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
Are you able to compare a failing config in 24.11 with a working one?
Well, comparing proved difficult as I had to be extremely careful to build exactly the same setup. When I finally succeeded and had one working (24.03->24.11 restore/upgrade) and one not working (24.11 direct restore) install that was backed up immediately BEFORE any testing, all I came up with was this missing line in the install that didn't work:
</notifications> <qinqs></qinqs> <-- This line is not there in the "BAD" config backupIt did fail on other items like dhcp leases "db", one captive portal encrypted "db" section, time of the last revision, and pkg repo conf path. Other than that, they were identical.
-
Hmm, none of that should make any difference.
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
Hmm, none of that should make any difference
OK, if it doesn't make any difference then, as I had two appliances, one with a good install and one with a faulty install, I simply took the good backup and restored it to the faulty install and took the faulty system backup and restored it to the good system install.
Well, good stayed good and faulty stayed faulty. The issue is not in the backup, it is in the 24.11 Beta install itself and once "broke", it stays broke.
I will follow up with the results of a restore of the original CE 2.7.2 backup to the good system when I have time. That restore in the past has always been to a fresh 24.11 Beta install, this time it will be to an existing, good 24.11 Beta install. Stay tuned.....
-
Hmm, to be clear, it now looks like a system with a clean 24.11 install fails when given the config from a system that was upgraded to 24.11?
-
@EDaleH said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
restore of the original CE 2.7.2 backup to the good system
OK, the original CE 2.7.2 Backup file repeatably results in a faulty 24.11 Beta (when that installation is fresh/new) resulting in a Voucher/local DB authorization error when MIM is on.
Restoring that file to a 24.03->24.11 upgraded installation, that does not display the authorization error, results in a good installation that does NOT display the authorization error either.
In other words, we have narrowed this down to occurring only when the V23.3 config file from CE 2.7.2 (or Plus 24.03) is restored to a brand new install of 24.11 Beta. If the install was a result of an upgrade from 24.03 (where the V23.3 config file from CE 2.7.2 / Plus 24.03 was already restored in advance) to 24.11 Beta, the authentication error does not occur when MIM is enabled.
Once a 24.11 Beta install is present that does not display the authentication error, an installation of a V23.3 config file does not "break" it.
Again, the only safe way to move CE 2.7.2 or Plus 24.03 to 24.11 Beta is to upgrade through the GUI. A fresh install of 24.11 Beta does not consistently result in MIM compatibility if restored from a prior version backup.
-
Ah, OK! So still looks like an upgrade issue in the config then. Even though the config itself does not look significantly different.
And to be clear still only happens when MIM is enabled?
-
Are you able to take status/diag files from the failing instance and upload them to us to analyze?
If so you can upload it here: https://nc.netgate.com/nextcloud/s/2poNFGxGJ7QZF8C
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
Are you able to take status/diag files from the failing instance and upload them to us to analyze?
Done, one successful login MIM disabled, one unsuccessful login MIM enabled. See captive portal authentication log.
FYI: Lab systems use XG-7100-1Us and Production systems are on XG-1541 Maxes. No internal switches are used, all are physical interfaces, specifically igc0, ix0 and ix1 for both Lab and Production installations.I will leave this "broken" installation of 24.11 Beta running on a currently spare XG-7100-1U for a while in case you want more log info. I do have a separate, working installation too if you want logs from an MIM enabled successful login on a "good" 24.11 Beta install to compare to. That system is identical to the one that produced the logs I uploaded.
-
This morning I did an update on the "faulty restored" 24.11 Nov 12 Beta to 24.11 Nov 21 Stable and the problem was corrected. However, I also did a fresh install of 24.11 Nov 21 Stable and then restored the CE 2.7.2 backup to it and the issue returned, so any GUI update corrects the problem but a direct restore of a V23.3 config file (2.7.2 or 24.03) appears to still create the Authentication Error if done through diagnostics, backup & Restore.
My advice is to restore a CE 2.7.2 or a 24.03 Plus to a 24.03 Plus and then do the GUI Update to 24.11 Plus just in case. This appears to be an installation specific backup config versioning issue.
-
Hmm, we still haven't replicated it here. We may simply not have a config sufficiently complex to hit it. IIRC you also didn't hit it on a basic config?
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
you also didn't hit it on a basic config?
Not this time, but in the past a 4 portal setup did NOT reproduce the problem, it worked fine. So far it is just this 8 port config that is not restoring but I have used at least 4 different backups and restored from both CE 2.7.2 and Plus 24.03 with the same authentication result if restored directly to a new 24.11 install (both beta and the stable releases) through Backup & Restore. A backup of the same setup in 24.11 does restore to a new 24.11 without any problems.
I am satisfied we have a way to work around it and will simply avoid any direct restore to 24.11 of any earlier version backups.
-
After many hours of troubleshooting, it turned out that the backup package restore was not respecting the changes made to /usr/local/captiveportal/index.php by the 24.11 install.
This is our fault for backing it up in the first place as it is considered part of the pfSense installation itself and in the case of 24.11 it is updated to support MIM. We backup the entire /usr/local/captiveportal directory as it contains all the custom code for the captive portal login pages. This resulted in unintentionally backing up the index.php as well. Thus a restore of our custom files also unintentionally overwrote the index.php with a prior version.
Do not backup or restore /usr/local/captiveportal/index.php if you use the backup package prior to installing Plus 24.11, it will break captive portal authentication when MIM is enabled.
Thank you for the assistance in locating this error.
