Lack of foresight killed the system
-
@MajorDumPerch I love it the first time a new guy scares themselves by destroying a network. They're so surprised. I'm grateful that I haven't shorted out the planet yet.
Don't Panic. Just don't.
Most of the work in getting pfSense set up right is accomplished by mapping your network, keeping in mind that you can create aliases in pfSense for everything, big and small, unique or grouped. Aliases are your friend.
On paper and in spreadsheets: Map your IP and DNS spaces: WANs (you have at least two ISPs, right?), LAN, perimeter, and no-mans-land networks and their subnets. Map DHCP assignments. Record the MAC IDs of every device on the network. Assign IP addresses by DHCP to every device you can (except servers, routers,... you know.)
Inventory applications that require internet access. What hosts, protocols and Privileged or Registered Ports ports to they require access to?
Finally, inventory any other internet services that are required. Knowing what traffic is required helps to identify what traffic is unnecessary. If there are servers you need to protect, they'll require restrictive Allow rules.
Forget the VPN. You'll do that last. The big babies with the executive suites can come into the office instead of phoning it in. Make sure they understand that it's a low priority with you. You're in charge of the project, and you want to do it right.
But if you thoroughly understand your network, configuring it in pfSense is easier.
After that, there's a how-to for everything. I get my best results from Google and DuckDuckGo. As long as I use search language from the pfSense manual, I'll find what I need.
Install pfSense on a powerful computer. Don't skimp on hardware. If it's got 8 or more cores, be aware that you must double some Flow and Stream memory caps for each interface in Suricata. Search for others' experiences setting up pfSense on your hardware.
Configure Advanced Settings. Change the ports used to access pfSense. I use 47 for https and 227 for ssh. Disable offloads to the NICs. I wish it worked, but offloading sometimes causes problems. Just make sure you have more CPU than you think you need.
Configure DNS and DHCP.
If you're setting up pfSense in an Active Directory environment, quadruple the amount of time you hope the project will take. Then double that. Stretch it out as long as you can and go slowly. S-l-o-w-l-y. Double-check everything before making any changes in AD. Your DCs should use pfSense as their only forwarder, and Unbound should be configured with domain overrides to the DCs for both DNS and RDNS.
Configure your firewall rules, organized in groups of machines, subnets, and networks, similar to how aliases are grouped. Make sure everything can get access to updates and that protected machines have only the access they need. This is where aliases are especially useful.
Don't even dream of configuring authentication yet. You'll do that after you have a week of relaxation, just before you set up the VPN.
Set up pfBlockerNG and Suricata. Don't block too much. Watch the logs. And for that, make sure you're set up the GUI's Dashboard with 3 well-organized columns. I like the Dark-beta theme.
Then take an extra day off. You'll get a list of problems that you can tackle one by one. When they're solved, take a few more days off.
Then you can do certificates (CA, servers, users), network authentication (LDAP or, preferably, RADIUS), and VPN.
Then ask for a raise.
Well, those are the steps that I recommend, anyway. I hope this helps.
-
We have two WAN connections from two different companies coming into the building from different directions. I followed the instructions in the manual and everything worked fine, including the rollover tests. I've seen where some folks had problems, but I haven't seen any.
-
Ok, so things got better then worse.
After a few reboots and restores, the system was almost at 100%.
I loaded up a restore to see if it would work. Surprisingly, it did. Packages appeared installed, internet access ok, and all I had to do was some basic troubleshooting.
Then I suddenly lost connection to the GUI and everything on the lan couldn't communicate with anything.
Did a full on reset. Restored a configuration I just saved when things were working well.
System on 2.3.5.
It now wants to download and install packages. But the wan isn't working.
I tried to follow the instruction on the wan connectivity troubleshooting document.
Nothing worked.A strange thing I found was the WAN was offline. It only goes online when I make the default gateway and the IP address the same. I know that can't be right because not only are we still not reaching the internet, but the values aren't supposed to be the same.. But if I change them to their correct values, the gateway goes offline. We have a static IP so this shouldn't be like this.
Edit: found in the logs that the gateway is giving a sendto error 64
Anyone have a solution?
-
@majordumperch said in Lack of foresight killed the system:
if I change them to their correct values, the gateway goes offline
Sounds like the gateway monitoring is triggering and taking it offline? Set the monitoring IP to something else like 8.8.4.4, or check Disable Gateway Monitoring for that gateway.
Did you read the docs on upgrading from old versions?
As I recall 2.4 removed support for 32 bit so if you're stuck on 2.3 check to see if the current install is 32 or 64 bit.
-
I'm going to try and remember things as I'm posting via my phone and can't see the response.
Did what you put up there.
Still getting send to error 64, no internet.
I keep getting messages about the DNS not resolving but I have DNS forward (all interfaces) on and two DNS addresses already set.
I'm getting a lot of ipv6 entries on the firewall but I didn't set up ipv6 on this device.
The version says 2.3.5 - amd64
Freebsd 10.3-release-p22I have the update setting on last stable release.
The packages have now disappeared and I'm about broken on how to get this system even back to just getting on the internet. More or less getting the vpns back up and running
-
@steveits
And needless to say, the system won't find the update if it can't access the internet.So I'll remember what you put there for that but getting this system online takes precedence
-
@majordumperch said in Lack of foresight killed the system:
the system won't find the update if it can't access the internet.
System is not going to find update no matter what if he is running 2.3.5 - those packages are no longer available.. 2.3.5 has been EOL for years.. The whole 2.3.x line went end of life October 31, 2018 - with like 2 years warning of said fact before that.
I'm with @Gertjan on this - this is not your Coffee machine.. Your firewall needs to be kept current.. I get it, I do.. His mention of PPTP.. That has been DEAD!! I mean DEAD - like not secure for what a decade now? I myself have some boxes out of date - but the only reason has been covid.. There is nobody there to allow for an update - I don't care if its a 10K cisco box, or a diy 100$ box, etc. Remotely updating something with no possible way of someone to be on site in the worse case scenario is not a good idea.. So I would never recommend anyone do a remote upgrade, or even local without the proper safety measures in place - as clearly called out in the docs, etc.
Get the current media 2.5.1 and do a clean install is your best solution here. Unless how ever old this box your running on is that old.. Then get a new box!! While there have been people saying 2.5 has issues.. I have had ZERO!! Other than I couldn't do a clean zfs install on the first 21.02 release.. Every package worked, zero issues with unbound.. no problems at all.. Not saying there are not some issues - but with how many users use pfsense, and how many crazy setups are out there.. Yeah there is going to be issues - and who posts.. Users that have issues! If everyone posted that had zero problems with their upgrades - nobody would ever notice the issues ;)
People forget pfsense/netgate is not freaking MS - yet even companies like MS have major problems with their "upgrades" ;) for basic stuff like a printing causing BSOD..
Suggesting someone not go 2.5 for a clean install doesn't make a lot of sense to me - unless that user has called out something specific that has been identified.. Like nating with multiple wan connections, etc.
My advice to @MajorDumPerch would be to take this time to get a new box!! Since I have to assume your box your running on - that you don't even know what it is exactly is quite dated.. Your best bet would be to get a netgate box that works for your connection requirements.
An intelligent man is sometimes forced to be drunk to spend time with his fools
If you get confused: Listen to the Music Play
Please don't Chat/PM me for help, unless mod related
SG-4860 24.11 | Lab VMs 2.7.2, 24.11 -
Right now, I'm going to wait for a few days while I get a USB to console cable from Amazon. Found the old console cable that came with the box. Now just needs the connector to my laptop. After that, it's clean install time after watching enough youtube videos and reading blogs.
I'll update after I try it but it won't be until Friday when I actually get the cable from the Amazon Overlords.
I hope this works and that I don't have to get another box. The office is already bearing down on me and it makes me want to play solo Russian Roulette with a loaded Bazooka. I think it should work (barring a complete no go of it) because the processor is a 64-bit one.
I'll talk with the upper management to see if I can get an updated box after this is all over so I can at least futz with it outside of the active work environment.
Until then... wish me luck.
-
Good luck and yeah console is a must..
Maybe the box is only a few years old.. That would be some good news for sure.. But a new box never a bad idea - possible you can use the situation to your advantage. Old box works with clean install. And you get a new box, and can use that old one as emergency spare on the shelf setup..
And always nice to be able to play with something offline without interrupting production.
Let us know how it turns out..
Maybe you can talk the higher ups into springing for some pfsense training ;) or atleast support contract..
-
@johnpoz said in Lack of foresight killed the system:
While there have been people saying 2.5 has issues.. I have had ZERO!!
Um, port forwards not working on anything but the default WAN is a major issue for most of my installs. (I'm on CE, I guess they fixed it on Plus)
I agree otherwise, OP should get an Netgate box or an APU (sounds like the old box was an Alix running nano) and restore onto a current version.
-
Actually it is a Netgate APU box. Just an older one with a console port.
I think we got it around... 2015.
-
Just a quick update for anyone stuck in this situation.
In order to make double sure that we get this right, my company approved of contracting an PFsense specialist for this situation. The only one in our city.
They had an opening for today and I met them there this morning.
First, the complete flash worked and brought us up to 2.5.1. It took a DB9 to USB cable, a few cursewords, and some unplugging and replugging but eventually the new CE version settled into the system.
However, it didn't solve a lot of problems other than having the system up and running. We still couldn't access the WAN.
A few phonecalls later and we found out that our ISP changed our static IP. When I asked when that happened, they said since late 2015. Both of us were left a bit speechless about it. How has this thing been on the wrong IP address for 6 years now while still working as if nothing happened?
Getting the right static, we then changed all the rules and gateways to match. And now the thing chugs even faster than it used to. There was some bloat from the initial setup that was removed and a few new security measures were put into place.
After asking the guy, he recommended not upgrading the box because this one works just fine and that the next Netgate we'd need would be about $1500 more than this one. I don't think the higher ups would have approved of that.
Before he left, he told me "We make our money doing these emergency network repairs. Next time, sign a service contract and I'll come down during your next update and make sure it goes without a hitch." I'm not going to argue that point and I have his number on speed dial for that occasion.
Thanks for everyone's help.
-
@majordumperch Now you're done and cool with the guy you've found, but I have been where you are, and the best thing you can do while figuring things out is getting ANY PC out there, slam another NIC, put a pfSense image or whatever you are willing to put, and configure it from scratch to allow the basic things to run, then you take your time dealing with the cable you need and everything else the way @EveningStarNM recommended you.
That way you don't have a lot of people around you waiting and poking you while you are sweating it and trying to learn.
Obviously that's also a bit tricky because your net may not be safe in the meantime.
-
@iampowerslave said in Lack of foresight killed the system:
getting ANY PC out there, slam another NIC
Great tip.
Make that a dual-or-more NIC card.
Most of use have Window Pro so Hyper-V is one click away : create a VM that runs pfSense. It will run in parallel with the host, making LAN's etc. perfect for trying out settings and other situations.
Except for de card, no extra hardware needed. -
@gertjan Yes, I have not mentioned virtualization but that is the idea, VirtualBox runs ok with pfSense, just beware and use the NICs as Bridged and ensure they are not putting any traffic in the Host (no IP, etc)
You can VBox as a service (with Linux) or AlwaysUp in Windows with a watchdog can keep the VM running if it crashes, there are many options. Immutable/Non-Persistent disks helps too.
