Enabling MIM causes Authentication Error for voucher based logins in Captive Portal
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
ust to confirm when you restored the config into 24.11 was that the full config via the webgui? In other words was the config upgrade script run against it?
The interfaces match on the production and lab units so it is a simple webgui restore that runs without any further intervention and provides a working unit (gateway for Wan has to be changed, which is simple to edit in the config file first, that's it).
To be honest, I don't know what you are referring to as an upgrade script. If that provides an output log, it would be excellent to run it and look over what it changes, not to mention if it fixes the symptoms.
This afternoon I built a 2.7.2 single portal and restored it to a 24.11 directly and did not reproduce the problem. Time permitting, I will make the installation multi-portal and try again.
-
If you import a config that has an older config version that whatever is current for the pfSense version it gets run through a script to upgrade it to current. That includes code for each config version step.
However the config version is only help in the main <system> section of the config. If you import the full config file the version is seen and any required upgrades are run. But if you import only some section of the config (other than system) the version is unknown and no upgrades are run. That can result in an invalid config.
The fact it worked for you importing into 24.03 first hints at a config version problem because it has the same config version as 2.7.2.
https://docs.netgate.com/pfsense/en/latest/releases/versions.html24.11 actually uses 23.6.
-
@rlinnemann said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
glad it sounds like you've worked around it, but my spidey sense is still tingling here.
Having identified a config version as the cause is the conclusion from my perspective.
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
24.11 actually uses 23.6.
The chart says config ver 23.3 for Plus 24.11 but I am quite satisfied that all restores to 24.11 must be done by restoring from or through (if it is CE 2.7.2) Plus 24.03.
Attempting to answer your questions is what lead to the final diagnosis here, it is comforting to know that existing installations have an upgrade path that includes MIM.
-
@EDaleH said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
The chart says config ver 23.3 for Plus 24.11
Yeah that page needs to be updated when 24.11 is released but currently it's using 23.6.
So it could be failing to upgrade the config at import....
-
Doesn't look like is it though. The search continues...
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
doesn't look like is it though. The search continues...
Well, your search may continue for the "fix" but the cause is clearly identified.
This morning I built a 24.03, restored the CE 2.7.2 backup onto it, tested it worked and then Backed it UP. I then upgraded it to 24.11 Beta and it does not display the authentication error when MIM is turned on.
Next, I built a new 24.11 Beta and restored that 24.03 backup onto it and voila! the authentication error is there every time you turn MIM on. Conclusive proof that the only way to get a stable 24.11 Beta in my case is to go through 24.03 and do a GUI upgrade.
-
Right which really seems like config upgrade issue at restore. It's not doing something that is done at system upgrade.
But it's more complex than that because I tried exactly that with a basic config and it still worked fine.
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
But it's more complex than that because I tried exactly that with a basic config and it still worked fine.
I have been unable to duplicate it with a fresh install either. This install is as complex as it gets for me and runs flawlessly. I am just trying to ensure it continues to do so under 24.11, Kea and MIM. Lots of lab testing left!
-
@stephenw10
As part of my testing of 24.11 Beta, I had a step to do a backup, fresh install and restore to confirm functionality. I moved that to the top of my list due to the restore issues I had encountered. I can confirm that a backup of a working 24.11 install (i.e. one that came through a 24.03 upgrade) will restore to a fresh 24.11 and work properly without displaying the authentication error.That suggests the format/processing of the backup config file (V23.3) is by far the most likely cause.
-
Exactly it appears that when you import the 24.03 config into 24.11 it's not being upgraded correctly. But only when the config is sufficiently complex.
Are you able to compare a failing config in 24.11 with a working one?
That looks identical in my testing here but clearly something in your config is hitting an issue.
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
Are you able to compare a failing config in 24.11 with a working one?
Well, comparing proved difficult as I had to be extremely careful to build exactly the same setup. When I finally succeeded and had one working (24.03->24.11 restore/upgrade) and one not working (24.11 direct restore) install that was backed up immediately BEFORE any testing, all I came up with was this missing line in the install that didn't work:
</notifications> <qinqs></qinqs> <-- This line is not there in the "BAD" config backupIt did fail on other items like dhcp leases "db", one captive portal encrypted "db" section, time of the last revision, and pkg repo conf path. Other than that, they were identical.
-
Hmm, none of that should make any difference.
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
Hmm, none of that should make any difference
OK, if it doesn't make any difference then, as I had two appliances, one with a good install and one with a faulty install, I simply took the good backup and restored it to the faulty install and took the faulty system backup and restored it to the good system install.
Well, good stayed good and faulty stayed faulty. The issue is not in the backup, it is in the 24.11 Beta install itself and once "broke", it stays broke.
I will follow up with the results of a restore of the original CE 2.7.2 backup to the good system when I have time. That restore in the past has always been to a fresh 24.11 Beta install, this time it will be to an existing, good 24.11 Beta install. Stay tuned.....
-
Hmm, to be clear, it now looks like a system with a clean 24.11 install fails when given the config from a system that was upgraded to 24.11?
-
@EDaleH said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
restore of the original CE 2.7.2 backup to the good system
OK, the original CE 2.7.2 Backup file repeatably results in a faulty 24.11 Beta (when that installation is fresh/new) resulting in a Voucher/local DB authorization error when MIM is on.
Restoring that file to a 24.03->24.11 upgraded installation, that does not display the authorization error, results in a good installation that does NOT display the authorization error either.
In other words, we have narrowed this down to occurring only when the V23.3 config file from CE 2.7.2 (or Plus 24.03) is restored to a brand new install of 24.11 Beta. If the install was a result of an upgrade from 24.03 (where the V23.3 config file from CE 2.7.2 / Plus 24.03 was already restored in advance) to 24.11 Beta, the authentication error does not occur when MIM is enabled.
Once a 24.11 Beta install is present that does not display the authentication error, an installation of a V23.3 config file does not "break" it.
Again, the only safe way to move CE 2.7.2 or Plus 24.03 to 24.11 Beta is to upgrade through the GUI. A fresh install of 24.11 Beta does not consistently result in MIM compatibility if restored from a prior version backup.
-
Ah, OK! So still looks like an upgrade issue in the config then. Even though the config itself does not look significantly different.
And to be clear still only happens when MIM is enabled?
-
Are you able to take status/diag files from the failing instance and upload them to us to analyze?
If so you can upload it here: https://nc.netgate.com/nextcloud/s/2poNFGxGJ7QZF8C
-
@stephenw10 said in Enabling MIM causes Authentication Error for voucher based logins in Captive Portal:
Are you able to take status/diag files from the failing instance and upload them to us to analyze?
Done, one successful login MIM disabled, one unsuccessful login MIM enabled. See captive portal authentication log.
FYI: Lab systems use XG-7100-1Us and Production systems are on XG-1541 Maxes. No internal switches are used, all are physical interfaces, specifically igc0, ix0 and ix1 for both Lab and Production installations.I will leave this "broken" installation of 24.11 Beta running on a currently spare XG-7100-1U for a while in case you want more log info. I do have a separate, working installation too if you want logs from an MIM enabled successful login on a "good" 24.11 Beta install to compare to. That system is identical to the one that produced the logs I uploaded.
-
This morning I did an update on the "faulty restored" 24.11 Nov 12 Beta to 24.11 Nov 21 Stable and the problem was corrected. However, I also did a fresh install of 24.11 Nov 21 Stable and then restored the CE 2.7.2 backup to it and the issue returned, so any GUI update corrects the problem but a direct restore of a V23.3 config file (2.7.2 or 24.03) appears to still create the Authentication Error if done through diagnostics, backup & Restore.
My advice is to restore a CE 2.7.2 or a 24.03 Plus to a 24.03 Plus and then do the GUI Update to 24.11 Plus just in case. This appears to be an installation specific backup config versioning issue.
-
Hmm, we still haven't replicated it here. We may simply not have a config sufficiently complex to hit it. IIRC you also didn't hit it on a basic config?
