[SOLVED] Inconsistant pinging across OPT
-
SEE SOLUTION BELOW
Hey all, I have a PFSense box with several OPT interfaces.
I'm trying to do a "simple" test of having a machine on OPT1 ping a machine on OPT3. I can't make heads or tails of these results, and I'd appreciate it if you all can stop me from pulling my hair out! (Firewall rules are after results)
Machine 1 (on OPT1) can ping OPT3 but NOT the machine on OPT3.
OPT3 interface can ping the machine on OPT3.To make the test (more) straightforward, I have the following rules on ALL OPT ports. The goal here is to keep the OPT ports from using LAN or WAN to talk to each other, so ALL traffic to WAN/LAN is blocked. ONLY traffic between OPT ports is allowed, and these are the only two machines on any OPT ports right now.
Checking firewall logs, I see no traffic AT ALL regarding the 10.1 when I ping.
How can a route exist from 10.1.1.1/24 to 10.1.4.1 but NOT 10.1.4.11??? Also, why are the pings not in the firewall logs?
Solution is post #10 here (https://forum.netgate.com/topic/153258/inconsistant-pinging-across-opt/3) but the TL;DR is clear Table States in case you have bad routes.
-
Take care of the target you PING.
The Windows Firewall for example blocks any traffic outside local/known subnets. So for testing disable the host Firewall.-Rico
-
@Rico said in Inconsistant pinging across OPT:
Take care of the target you PING.
The Windows Firewall for example blocks any traffic outside local/known subnets. So for testing disable the host Firewall.
-RicoThanks for the reply Rico :)
I actually got excited because that seemed like exactly what could be doing this, but alas even after turning the firewall off it was not the case. (FWIW, it's linux not windows on both machines--firewall was ufw). I wonder if there's not something to this idea.... but
sudo systemctl stop ufw
did not allow the ping to complete. -
Show your OPT1 and OPT3 interface configuration and source + tagert machine ifconfig output.
Your Firewall screenshot shows OPT1?-Rico
-
@Rico said in Inconsistant pinging across OPT:
Show your OPT1 and OPT3 interface configuration and source + tagert machine ifconfig output.
Your Firewall screenshot shows OPT1?
-RicoScreenshots are below. Screenshots are in parts as the page is longer than my monitor >_> But both parts have IP address as a sanity check!
Firewall was from OPT1 but all OPT have same rules as I'm troubleshooting this!
-
I think that was everything, was there anything else that might help?
-
Had a possible thought that isn't easily Googleable--would I have to mark these as a gateway if there's no intention of reaching the internet but merely other parts of the Local Network? I know the internet is just "a network of networks" and I'm reaching outside the subnet of the device---but not reaching outside the router! So I'm unclear if this may be a source the issue!
-
Do you have any Floating Rules or custom NAT settings?
Generally speaking, the communication between internal Interfaces should work just out of the box with the default settings plus Firewall Rules.
I did this a hundred times...add the new Interface like OPT, add the Firewall Rules to OPT (like any-any for testing). After that OPT1 can reach LAN and the Internet. LAN can reach OPT and the Internet.
After this test you can do the fine tuning with Firewall Rules, like OPT is not allowed to talk with LAN, etc.
So your config is corky somehow.
Post as much information as possible what you did....or start over with pfSense factory default settings.-Rico
-
@Rico ok, if it should work then I'm going to start from scratch (again) this weekend. I will try my best to document EVERY STEP & post here.
Really the only thing that I can see making potentially unusual is that this is all virtualized with VirtualBox, so the VMs are connected with intnets (short for internal networks). This shouldn't make a difference as I've had success with different intnets & PFSense before.
Regardless, look for a post in a few days--hopefully a happy one :)
Oh, and no custom NAT or floating rules. I am running HAproxy, but that's a relatively simple config too (just reverse proxy from WAN to OPT1 based on subdomain).
-
Glad to report it's fixed. After several existential crises I found the problem. Ends up while I was doing 498758967456798430674551 different things, I swapped some ports around, and got some bad routes. For whatever reason, PFSense preferred the bad routes to the good one. So I just went to "Diagnostics --> States --> Reset States" and reset all (literally the only option).
Anyway, after giving everything a minute or two....it all just worked. I have no idea why those routes didn't clear out, but it's CERTAINLY a tool I'll remember in the future!!!