Stuck upgrade of squid after pfsense upgrade 2.4.5
-
Certainly appears to be upstream from you, though I don't quite see why/how your ISP would be routing that any differently.
-
Yeah very odd.. since they are both in the same netblock
-
Yeah, I agree. I guess I have to call them up again tomorrow to see why this is happening.
Do you guys have any ideas on my ACB restore question?
@kevindd992002 said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
Also, will restoring a backup from the Auto Config Backup (ACB) service reinstall the packages as well? I tried restoring a 2.4.4p3 config (a backup one day before I upgraded to 2.4.5) but it did not even attempt to reinstall the packages. The console logs say it was trying to start the packages and this will naturally fail because they weren't installed in a fresh install of pfsense in the first place. So why would restoring a physical config.xml file reinstall the packages as expected but not when restoring using a backup from ACB? I cannot find any documentation about the differences between these two restore options.
-
Not currently. The only method which triggers a package reinstall is using Diag > Backup/Restore.
-
Ok, thanks for the confirmation. If there are any corruption in a previous install, would backing up the config.xml of that corrupted install carry over any type of corruption when restoring to a new install?
-
From your trace and my trace that both go through that level 3 network 4/9 network, I would "guess" has something going on there..
-
I have a site-to-site VPN connection to the working box so I can route traffic to files00.netgate.com to the WAN connection of that box. How do you create a rule that would direct this traffic to the openvpn gateway? I mean, how do you create rules with a source of the pfsense box itself? Floating rules?
-
You can't policy route from the firewall itself but since the destination is static, you can add the package server IP addresses to IPv4 Remote network(s) for the OpenVPN Client. For example, add
162.208.119.40/32,162.208.119.41/32 -
And FYI- I had our IT staff check and both package servers can trace a route all the way back to you (or at least one hop out) so it does appear to be a problem with the traffic from your location coming this way.
Hopefully nudging that to use your VPN works around it.
-
Oh a completely unrelated topic, I think you are the first one I have seen using the home.arpa name space.. Nice! I guess you are a rfc reader.. You just didn't accidentally decide to use that - did you? Curious.
-
@jimp said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
You can't policy route from the firewall itself but since the destination is static, you can add the package server IP addresses to IPv4 Remote network(s) for the OpenVPN Client. For example, add
162.208.119.40/32,162.208.119.41/32@jimp said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
And FYI- I had our IT staff check and both package servers can trace a route all the way back to you (or at least one hop out) so it does appear to be a problem with the traffic from your location coming this way.
Hopefully nudging that to use your VPN works around it.
Ok, that seems to have worked! Though not optimal, I can settle with it for now until I can report it to my ISP tomorrow. The weird thing is that these two boxes use the same ISP. The affected box uses a public static IP assigned directly to pfsense's WAN interface and the working box is behind CGNAT. For what it's worth, here's the traceroute from the working box:
1 192.168.100.1 (192.168.100.1) 0.548 ms 0.371 ms 0.304 ms 2 10.188.16.1 (10.188.16.1) 3.142 ms 3.594 ms 3.606 ms 3 172.20.20.93 (172.20.20.93) 3.832 ms 5.295 ms 3.420 ms 4 181.1.49.161-rev.convergeict.com (161.49.1.181) 2.794 ms 2.719 ms 2.727 ms 5 69.1.49.161-rev.convergeict.com (161.49.1.69) 5.239 ms 10.829 ms 2.981 ms 6 161.49.1.250 (161.49.1.250) 15.138 ms 4.752 ms 4.578 ms 7 * * * 8 * * * 9 * * * 10 THE-NEW-YOR.ear1.Newark1.Level3.net (4.15.150.218) 205.096 ms 205.100 ms 205.004 ms 11 cs90.cs99new.v.ewr.nyinternet.net (96.47.77.218) 205.165 ms 205.674 ms 205.380 ms 12 162.208.119.41 (162.208.119.41) 205.012 ms 205.114 ms 204.837 ms1 192.168.100.1 (192.168.100.1) 0.674 ms 0.494 ms 0.322 ms 2 10.188.16.1 (10.188.16.1) 2.990 ms 2.920 ms 3.217 ms 3 172.20.20.93 (172.20.20.93) 60.350 ms 15.737 ms 3.726 ms 4 181.1.49.161-rev.convergeict.com (161.49.1.181) 3.000 ms 2.895 ms 2.916 ms 5 13.1.49.161-rev.convergeict.com (161.49.1.13) 10.875 ms 3.152 ms 3.102 ms 6 246.1.49.161-rev.convergeict.com (161.49.1.246) 4.571 ms 4.453 ms 4.556 ms 7 * * * 8 * * * 9 * * * 10 THE-NEW-YOR.ear1.Newark1.Level3.net (4.15.150.218) 208.253 ms 208.109 ms 208.061 ms 11 cs90.cs99new.v.ewr.nyinternet.net (96.47.77.218) 208.437 ms 208.363 ms 208.219 ms 12 162.208.119.40 (162.208.119.40) 207.967 ms 208.587 ms 208.425 msSo I guess it's time I edit my config.xml now to include the edits I made for the OpenVPN remote networks and start from scratch by restoring again. In case you missed my question about this:
If there are any corruption in a previous install, would backing up the config.xml of that corrupted install carry over any type of corruption when restoring to a new install?
@johnpoz said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
Oh a completely unrelated topic, I think you are the first one I have seen using the home.arpa name space.. Nice! I guess you are a rfc reader.. You just didn't accidentally decide to use that - did you? Curious.
You got it! I work as an MS Exchange sysad so I read up on RFC's from time to time. When I was optimizing my home networks, I researched on what the proper local domain name is to avoid any potential conflicts with any system and stumbled upon the arpa domain.
-
@kevindd992002 said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
If there are any corruption in a previous install, would backing up the config.xml of that corrupted install carry over any type of corruption when restoring to a new install?
It depends on what was corrupted. If it's something in the OS/FS then your config.xml is probably OK. If it was really corrupt in the classical sense it would be invalid XML and get tossed out automatically.
If there is a bad setting in there, however, the setting would be in config.xml and carry over. That isn't really "corruption", though.
-
@jimp said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
@kevindd992002 said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
If there are any corruption in a previous install, would backing up the config.xml of that corrupted install carry over any type of corruption when restoring to a new install?
It depends on what was corrupted. If it's something in the OS/FS then your config.xml is probably OK. If it was really corrupt in the classical sense it would be invalid XML and get tossed out automatically.
If there is a bad setting in there, however, the setting would be in config.xml and carry over. That isn't really "corruption", though.
Ok, got it. When you say the invalid XML gets tossed out automatically, you're saying that pfsense won't even allow it to be restored and would complain, right? The reason I ask about corruption is that some of my configs in ACB (when I Show Info) have this error and it's the first time I've encountered it:
Any ideas why? Or are these ACB bugs? It doesn't happen for all my backups and I'm not seeing a pattern as to which ones are getting affected.
-
That's a different case. Usually that would mean the password you used didn't match the password used to encrypt the configuration file. Or perhaps you copied the wrong text field.
-
@jimp said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
That's a different case. Usually that would mean the password you used didn't match the password used to encrypt the configuration file. Or perhaps you copied the wrong text field.
If it was the wrong password then I would get the same error for all, but that is not the case. What do you mean copied the wrong text field? For the Device Key?
Also, as for this notification message:
Does "Do not make changes in the GUI until this is complete" mean not to change any setting in the GUI of the packages only? Or does it apply for the whole pfsense GUI? While the package reinstall ongoing, I had to create a new outbound NAT rule earlier and I'm not sure if it had any effect to the package reinstall like what the notification message says.
-
And while I was upgrading to 2.4.5 on this affected box I actually was stuck at the "updating repository" part in the GUI but when I reloaded the GUI it said that it is already at 2.4.5. So I was under the impression that the upgrade went through just fine and there was just an issue with the GUI until I stumbled upon this issue with the packages.
I did not get a config.xml backup before the upgrade because I thought the ACB backups were enough in case I go through a botched upgrade. So the config.xml I'm using now is the backup I got "after" the upgrade. This is another reason why I'm worried that there might be some kind of corruption/incorrect setting in the xml that might carry over to this new install. So far, I don't see anything in particular. Everything works fine but is there any additional check that I should do?
-
@kevindd992002 said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
If it was the wrong password then I would get the same error for all, but that is not the case. What do you mean copied the wrong text field? For the Device Key?
I mean there are two boxes in ACB which show the encrypted backup and the decrypted backup. If you see content in the encrypted backup box but not the decrypted backup box, then the wrong encryption password is set on the Settings tab.
But the specific error you showed above could also indicate that whatever response was received from the ACB server didn't contain a valid backup file.
@kevindd992002 said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
Does "Do not make changes in the GUI until this is complete" mean not to change any setting in the GUI of the packages only? Or does it apply for the whole pfsense GUI? While the package reinstall ongoing, I had to create a new outbound NAT rule earlier and I'm not sure if it had any effect to the package reinstall like what the notification message says.
It means what it says. Do not make any changes in the GUI until it is complete. The package (re)install process will make changes to the config and might clobber changes made in the GUI if the timing is wrong. If your NAT rule is still in the config then it's probably fine.
@kevindd992002 said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
I did not get a config.xml backup before the upgrade because I thought the ACB backups were enough in case I go through a botched upgrade.
You should always keep a local backup for safety. ACB is nice but it's still a remote backup. If you were completely down and couldn't access ACB then you would have been stuck.
So the config.xml I'm using now is the backup I got "after" the upgrade. This is another reason why I'm worried that there might be some kind of corruption/incorrect setting in the xml that might carry over to this new install. So far, I don't see anything in particular. Everything works fine but is there any additional check that I should do?
The config is probably fine. You could look under Diag > Backup/Restore on the config history tab and check to see if one is there from before the upgrade as well.
-
@jimp said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
@kevindd992002 said in Stuck upgrade of squid after pfsense upgrade 2.4.5:
If it was the wrong password then I would get the same error for all, but that is not the case. What do you mean copied the wrong text field? For the Device Key?
I mean there are two boxes in ACB which show the encrypted backup and the decrypted backup. If you see content in the encrypted backup box but not the decrypted backup box, then the wrong encryption password is set on the Settings tab.
But the specific error you showed above could also indicate that whatever response was received from the ACB server didn't contain a valid backup file.
The config is probably fine. You could look under Diag > Backup/Restore on the config history tab and check to see if one is there from before the upgrade as well.
Yeah, so I don't even get to the point where it shows me those two boxes when it shows the error message I posted above. It just goes back to the page where it shows the list of available ACB backups. What could cause an invalid backup file being sent to the ACB servers multiple times?
After the restoration to the new install, the only zi have under Diag -> Backup/Restore -> Conf History are those from the time I first accessed the GUI of the NEW install. The xml restore or any other else didn't bring back any of the old backup, unfortunately.
-
@jimp Do you still have any ideas on my pending questions? Thanks.
-
I tried setting one up from scratch here and the only time I get that error is when I have the wrong password in the settings and I click on a backup. But it's possible maybe there is a timing or other issue at play where it doesn't get a proper response from the server.
Next time you try to access one and get that error, wait about 10-15 seconds without touching anything else and then refresh the page.
