Adding RAID long after original install - custom install partition sizes or factory defaults
-
@stephenw10 I have to say your suggestion is wrecking my plan! But everything you have helped with previously proved reliable.
Would you please, in your understanding why only mirroring with two drives, instead of my three would be best. I tend to like the concept of swapping without shutting down... however how often would I not shut down pfsense to work on the hardware. I also like more redundancy than less - amateur for sure. I mean how bad can it be, "more is better!"
It seems a great deal of disk space is unused because of the efficiency of pfsense. Are there ways to utilize the underutilized space?
-
If you have hot-swappable hardware raid then that might be an option. But if you want that level of resilience then you should have an HA pair anyway.
The issue with hardware raid is it often requires additional drivers and can make reinstalling significantly more complex if you need to.
-
@stephenw10 Ahhh, so I have a misconception. No, I have no hardware RAID, I assumed my experience with TrueNAS was similar to RAID in FreeBSD - in it was software driven and provided hot swappable capabilities. If this is true - it does not - then yes one of my reasons for RAID with three drives is out the window. Please clarify "HA pair?"
-
Most SSDs / drive controllers are not hot-swappable so you need to power down the system to change it out.
A High Availability pair of pfSense installs means two separate hardware devices that can both pass the traffic. This means that in the event of a hardware failure traffic still passes. It also means you can power off one node to swap out a drive and traffic still passes.
See: https://docs.netgate.com/pfsense/en/latest/highavailability/index.html -
@stephenw10 Ah, yeah, no. My wife likes her HULU and Amazon but not that much!
Back to mirroring... if there is an SSD failure, pfsense will continue to function till I shut down pull the failed drive and reinstall new. Then the RAID will be rebuilt across the new drive. However if both drives fail it is a reinstall plus a config file - just like if my current single drive fails.
So the redundancy is in the odds of both SSD's failing simultaneously.
Suddenly... I realized right now I have no plan for recovery if my single SSD failed. I need to archive the current image from netgate I have for installing pfsense via memory stick!
-
Well IMO the odds of two SSDs failing at the same time are far lower than some other hardware component failing taking out the firewall. But, sure, the odds of 3 SSDs failing are even lower.
-
@stephenw10 Well you are a disrupter (Which is good... in this case.) I could move the 2 110 GB SSDs from my NAS into the pfsense box and move the larger SSDs which I purchased into the NAs making a more efficient use of the GBs.
-
@stephenw10 Hello, I need help. I went through the install process rebooted - missed inserting the USB with my old config.xml. Inserted the USB and rebooted from console and It appeared to not like my config.xml file - the bootup got stuck - something about "bad." I hit enter it defaulted back to the new fresh config.
Are there parts of my config file which are "bad" in terms of going from a single drive to RAID? -
@The-Party-of-Hell-No
Here is the complaint at bootup, and Line 135 it points to.
The two jpegs are following, did not like pulling off my phone
-
-
Nope the config doesn't reference the boot disk(s) or install type. It shouldn't care.
You can try booting with a default config then restoring the config in the gui. That might give you more useful errors.
The error it's showing there is actually secondary. It's trying to open the file to queue a notice about the bad config and failing during boot. But that's not actually related to the import error.
-
@stephenw10 So, I followed your directions and it appeared to work... as in all the interfaces, interface ip's were correctly displayed at the console.
However I could not get to the GUI from any browser on any of the LAN segments through LAN cables.
I am guessing the DNS resolver was not able to resolve.
My plan is to take down the Gateway Groups, turn off WireGuardVPN and OpenVPN Gateways and Interfaces and revert to the WAN as the default Gateway and Default DNS servers not the Surfshark forwarded DNS servers. Save this config. Boot into the new RAID version of pfsense, reset to factory defaults, get back into the the GUI - since I have done that previously and see if this will allow me to get to the GUI after uploading this new config file. Then rebuild what I turned off.
Is this a plan or folly? Do you have a different recommendation before proceeding?A related question... when I do a factory reset do I loose the RAID setup? The reason I ask is when I did the factory reset the first time to clear out my config and connected to the GUI I noticed only one disk listed under S.M.A.R.T status.
-
@The-Party-of-Hell-No said in Adding RAID long after original install - custom install partition sizes or factory defaults:
when I do a factory reset do I loose the RAID setup?
Nope. The config is defaulted but that's unrelated to the filesystem.
@The-Party-of-Hell-No said in Adding RAID long after original install - custom install partition sizes or factory defaults:
I am guessing the DNS resolver was not able to resolve.
Were you trying to access it by FQDN? Did you try connecting by IP directly?
I would expect it to respond if the config imported cleanly. I can only imagine the interface order was not what you expected.
-
@stephenw10 I tried using the IP of the firewall and the interface order appeared correct at the console. However, I can compare the two by booting into the RAID configured pfsense and confirm the match.,
-
I would also try pinging out from the console to be sure the firewall itself has connectivity.
-
@stephenw10 The interface order was a match for the working config, the assigned IP's matched also.
When I pinged to the outside from the console names did not work (Google, Yahoo) but IP's did - 1.1.1.1 and 9.9.9.9 returned packets.
Internally I can ping all wired desktops from the console - through switches
Cannot ping wireless access points - which are wired through the switches
I can ping from desktop to desktop - through switches
Cannot ping the firewall from any wired desktop
I can connect wireguard from external source but no data transfer
OpenVPN - Road Warrior will not connect - TLS negotiation failure to occcur within 60 seconds. -
How do you have DNS configured. Is it setup to use Unbound locally only? If so in Unbound running?
-
@stephenw10 General setup has the two DNS servers use by Surfshark VPN (With the same Gateways as used below.)
The DNS Resolver - General Settings - Network Interfaces - all the LAN segments(3),, the OpenVPN Server, Wireguard Server and Localhost are being used by Resolver.
For the Outgoing Network Interfaces I am using ONE WireguardVPN Surfshark Gateway and an OpenVPN Surfshark Gateway with DNS Query Forwarding - Enable Forwarding Mode checked.
-
Oh, then you can only get DNS at all if the VPN is connected. And that may not happen until the second boot after restoring a config since the first boot creates the interface.
So, yeah, check the VPN status.
-
@stephenw10 To clarify... if I reboot a second time after loading the config file it should work? I have been booting into the raid setup for a while now and this has not happened.
Wondering if adding the WAN as an Outgoing Network Interfaces would bypass the VPN's if they are not connecting.?
Also wondering since I can ping outward to 1.1.1.1 and 9.9.9.9 from the console, does this mean my DHCP lease with my provider is good? Or is my WAN connected to my ISP's system? I have had difficulties in the past with having to reboot the modem with pfsense off and then turning on the router to connect.
