Problem restoring backup of Pfsense +
-
Hi All,
I currently run Pfsense + 23.01-RELEASE (amd64) in a VM with an Intel card passed through and it works perfectly. LAN trunks to a USW-24-POE off which all my devices connect. It’s been running solid for a couple of years.
My setup is nothing fancy, OpenVPN client/server, 5 VLANS, snort, avahi, pfblocker, DYNDNS.
In testing my backup and restoration procedure this week (for no other reason than wanting to be sure it works) I have encountered a really curly problem that I cannot seem to solve.
I’m restoring to a bare metal (on a machine I used to run my setup on with dual Intel nics) procedure is also straight forward.
- Install CE
- Upgrade to Pfsense Plus
- Restore Config
- Fix Interface error on WAN / LAN and VLANS
- Kick my ISP connection to forget previous MAC
- Swap LAN/WAN cables from VM machine
- Reboot
Everything comes up, WAN and LAN IP, VLANS configured, VPN clients connect and get IPs. Everything is looking great on the console. Ping Google and internal clients across VLANS from the Shell all is well.
That’s where everything falls over. I cannot access the firewall GUI from LAN or from other VLANS that have rules that allow specific devices to do so. DNS isn’t working internal or external from devices (but is from Pfsense itself). No cross VLAN connectivity. Cannot ping, despite rules that allow. All the impressions of basically of no connectivity.
- Tried rebooting switch and AP’s - nothing.
- Tried wired connection both via switch and direct into LAN port on box - nothing
- Tried restoring using find and replace on interface names instead of using the warning method on restore - this has the config restore really quickly with no warnings - nothing
- Tried doing a config by config restore. As soon as I restore the VLANS - issue comes back.
- Put a stock install on the box and setup a few basic VLANS, VPN clients and server, works just fine - did this on the off chance there was a H/W failure since I used it as my main box. Not the hardware.
It’s really strange. I’m so glad I tested because I currently don’t have a restoration plan and get to debug this issue while everything is running fine.
I’ve seen a few reports of people trying to restore a backup with VLANS and appear to have hit something similar. All threads go stale once the person explains the problem.
Plug the LAN/WAN cables back into the original box and all is well.
Any insights from the community on this would be helpful.
Thanks
Dan
-
This post is deleted! -
The fact you can ping the LAN clients from pfSense and see replies shows this is not a layer 2 problem. VLANs must be working correctly.
Are the clients using static IPs? Are they trying to use the correct gateway? Can they ping the pfSense interface IPs directly?
Ultimately I would try running a packet capture on LAN and sending some pings at it from a LAN client.
Steve
-
@stephenw10 said in Problem restoring backup of Pfsense +:
The fact you can ping the LAN clients from pfSense and see replies shows this is not a layer 2 problem. VLANs must be working correctly.
Are the clients using static IPs? Are they trying to use the correct gateway? Can they ping the pfSense interface IPs directly?
Ultimately I would try running a packet capture on LAN and sending some pings at it from a LAN client.
Steve
Thanks for the reply. Next time I have some free time I’ll try it again and give your suggestions a try.
I’m confused though. The config is sound. It’s running as we speak on another machine. The hardware is fine, I can demonstrate it. My mind keeps going back to the backup restore process - it has to be there - like something just hasn’t restored properly.
I will keep troubleshooting though, and will report back on this thread what I find.
-
I would say it's more likely a difference between the VM and hardware setups. Something is not the same there. But it could be a problem when you reassign the interfaces to match. You might try just editing the config to match before restoring it. If that works it would prove the issue.
-
I wanted to report back on my restore journey. Resolved one issue, hit another.
I had been using the "Backup extra data." flag when performing back ups.
For no other reason than me doing some trial and error debugging of the error, I decided to do a backup without it and try a restore.
Following the same restore procedure as above, everything seemed to come back up.
All was well and working - I thought - until I noticed there was an issue when my packages were not downloading.
I noticed the error "Unable to retrieve package information." prompted by me noticing that snort, pfb and avahi were not downloading and starting.
So I did "pkg -d update" via the command line tool and DNS was fine as everything resolved BUT I got a "Bad Request" error whenever it tried to get a .pkf file.
e.g. "pkg: https://pfsense-plus-pkg01.atx.netgate.com/pfSense_plus-v23_01_amd64-core/meta.txz: Bad Request"
Following review of the forum I found a pointer that led me to check the license status.
Low and behold, on the Update page I see this "MessagesYour device has not been registered for pfSense+. Please purchase a pfSense+ subscription at the Netgate store to receive future updates."
As mentioned above, I had already registered. Everything about the installation appears to be working (sans the package manager) So, I went into the Register menu item and re-pasted my key from the email. It accepted the key fine but there was no change - except the key went light grey. I rebooted. There was no change, still no packages and still showing device not registered.
I have registered a ticket with TAC as I am hoping that my assumption that packages are not downloading is linked to the registration status of Pfsense +.
We shall see .... :)
-
TLDR: Last update of this thread. All is now well.
I was right, the reason the package manager wasn't working was because there was something wrong with the subscription.
According to Ryan from TAC the Netgate Device ID had changed between the time the token was redeemed and my current installation.
Because I have a Home subscription they were not able to migrate me to the new NDI but I was able to request a new token and enter the new token.
Once that was done, package manager worked and now everything is working.
I have to say, the support from TAC is amazing, especially for a home user who doesn't pay. I have just had a recent amazing support experience with Noctua that I thought would take some beating but this pips that by a nose. I can't imagine how good their paid support is.
Anyway, I now have a working and tested backup and restore strategy and I hope my documented little journey hear helps someone in the future.
-
@danioj Bigger takeaway: Netgate Device ID is based on your NICs and their MACs.
Add VLANs over interfaces to your VM guests -- you'll be happier long-term.
