pfSense keeps blocking access from one subnet to another most of the time (but not always)
-
Hey guys,
my pfSense still behaves weirdly and I cannot find a solution anywhere. DIY pfSense box, dual interface Intel card = two physical subnets. One subnet (LAN) is for Wi-Fi, IP Cams and stuff, another, OPT1, is for wired "work devices". Right now (a few months actually) there are no deny rules between these subnets and there are allow rules -- they (subnets) should be able to see each other. And they can sometimes! But most of the time I cannot ping OPT1 devices when I'm on Wi-Fi (LAN). And occasionally -- boom -- LAN can communicate with OPT1 out of the blue with no pfSense settings changed. I can't predict it and figure out what causes the access to unblock, but most of the time it is blocked. But, what's more confusing, my pfSense box has 2 IP addresses and I can ping its OPT1 IP when I'm on LAN (Wi-Fi), but it's the only accessible OPT1 device. Again, not always.
Access from OPT1 to LAN works always.
The problem goes away when I connect my laptop to OPT1 cable -- all devices are accessible, but of course I want to be able to connect to my Synology while I'm on Wi-Fi and most of the time I can't. But, again, sometimes pfSense decides to allow LAN to OPT1 ))
And! I see these warnings in logs:
sshguard 80034 Attack from "[my Macbook Wi-Fi (LAN) IP]" on service unknown service with danger 10.
What the....
Thank you guys! Happy New Year!!
-
Forgot to mention: version 2.7.0, 2.6.0 behaved the same.
-
@throttlenerd If it works when you are connected wired, but not wireless, it sounds like a problem with your wireless infrastructure not your firewall. You didn't give any details regarding how that is set up.
https://docs.netgate.com/pfsense/en/latest/troubleshooting/index.html#connectivity-networking
-
@Derelict Hi Derelict, thank you very much for your suggestion, but no, the problem lies in LAN to OPT1 communication, doesn't matter if I'm connected to LAN interface via Wi-Fi or wired with ethernet cable (via switch connected to pfSense, not to lan outputs on my WiFi mesh of course). So if I'm on LAN physical subnet (wired/wireless) -- I can't access OPT1. If I'm on OPT1 -- I can access anything. And, again, sometimes pfSense decides to allow LAN to OPT1 and I can't figure out when and why it makes such decision )))
And it's not just between my laptop and the router! If I go to Diagnostics > Ping -- even pfSense can't ping both OPT1 devices no matter which "Source address" I select: 3 packets transmitted, 0 packets received, 100.0% packet loss. And, for instance, one Synology is on OPT1 acting as a UPS server, another Synology on LAN getting UPS info -- and most of the time it can't reach OPT1 Synology. But sometimes it can hehe ))))
Thank you for any suggestions! Happy New Year! ))
-
@throttlenerd Sounds as if there is no firewall rules to allow LAN to access Opt1 nor OPT1 to LAN...can you show your firewall rules?
-
@NollipfSense Didn't find a quick way to upoload a screenshot (if there is such possibility), but rules should be effective always, and my problem disappears from time to time, that's what bugging me. Currently active LAN rules are:
default anti-lockout;
defalt allow LAN to Any
and a few deny rules just for IP cameras aliasI tried a "force allow LAN to OPT1", force "allow macbook to Synology" -- nothng works )
-
@throttlenerd said in pfSense keeps blocking access from one subnet to another most of the time (but not always):
Didn't find a quick way to upoload a screenshot
is the button not quick enough
-
@throttlenerd Please note the upload tab below.
@throttlenerd said in pfSense keeps blocking access from one subnet to another most of the time (but not always):
I tried a "force allow LAN to OPT1", force "allow macbook to Synology" -- nothng works )
You don't need to force anything...a simple allow traffic tcp/udp LAN to Opt1 and OPT1 to LAN will do...
-
@throttlenerd said in pfSense keeps blocking access from one subnet to another most of the time (but not always):
"force allow LAN to OPT1"
What is that even.. There is no "force allow" ??
Do you mean you created a specific rule? Please post a picture of your rules on your lan, and do you have any rules in floating?
But here is the thing if you had a rule blocking it, it wouldn't "sometimes" allow it.. That is not how firewalls work. Oh let me block this 90% of the time.. but sometimes I will let it pass ;)
-
Oh I didn;t see this button ))
-
@johnpoz I meant I tried to create specific "allow" rules which I described here as "force allow". And yes you're absolutely right about the 90% of the time ))
And there aren't any floating rules set, btw.
-
@throttlenerd all of the rules below that rule that allows lan net anywhere - are pointless.. they will never trigger.
Your lan net rule would allow lan clients to go anywhere they want.. anything that is IPv4.
If you can not ping your nas on opt1, then you have firewall running on the nas, is not using pfsense as its gateway.. Or its mask is wrong, or its multihomed?
We go over this like every other day it seems.. Your saying from the lan, you can not even ping pfsense IP on your opt1 interface?
Rules are evaluated top down, first rule to trigger wins, no other rules are evaluated.. I take it your cameras have an IP on your lan, so the lan net any rule would let them go anywhere.
So pfsense Ip on your lan is what 10.0.73.1/24 ?? And what about your opt1 network? What is pfsense IP.. Your saying you can not ping that??
Example my pfsense IPs all end in .253, my lan is 192.168.9.253, my network were my wifi controller, etc. sit is 192.168.2.253/24.. see I can ping pfsense IP from my lan.
$ ping 192.168.2.253 Pinging 192.168.2.253 with 32 bytes of data: Reply from 192.168.2.253: bytes=32 time=1ms TTL=64 Reply from 192.168.2.253: bytes=32 time=1ms TTL=64 Reply from 192.168.2.253: bytes=32 time=1ms TTL=64 Reply from 192.168.2.253: bytes=32 time=1ms TTL=64 Ping statistics for 192.168.2.253: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 1ms, Maximum = 1ms, Average = 1msAnd I can ping my wifi controller that sits on 192.168.2.12,
$ ping 192.168.2.12 Pinging 192.168.2.12 with 32 bytes of data: Reply from 192.168.2.12: bytes=32 time=1ms TTL=63 Reply from 192.168.2.12: bytes=32 time=1ms TTL=63 Reply from 192.168.2.12: bytes=32 time=1ms TTL=63 Reply from 192.168.2.12: bytes=32 time=1ms TTL=63 Ping statistics for 192.168.2.12: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 1ms, Maximum = 1ms, Average = 1msOnly rules I have on lan are
The rules I have on the 192.168.2 interface are meaningless when it comes to creating the connection from the lan side, because the state pfsense creates when the traffic is allowed as it enters pfsense lan would allow the return traffic. As long as that device answers and sends the traffic back to pfsense.
With those rules you for sure should be able to ping pfsense IP on opt1 100% of the time... If you can not you got something really buggered up.. Is your client on lan your pinging from maybe on wifi and wire at the same time? Is it running some security software that would stop you from pinging stuff?
Did you set its IP up, maybe you have a mask wrong? or did it get IP from pfsense dhcp server? Do you maybe have some wifi router with a leg in lan that might have its dhcp server still running?
-
Thank you @johnpoz for clarifying with the rule order, I heard many times that the rules work from top to bottom I guess I'm too dumb to understand it practically hahahah )) Placed deny rules for IP cams on top.
About my pfSense problem:
@johnpoz said in pfSense keeps blocking access from one subnet to another most of the time (but not always):
If you can not ping your nas on opt1, then you have firewall running on the nas, is not using pfsense as its gateway.. Or its mask is wrong, or its multihomed?
No, it's not about NAS, right now I have three devices turned on on OPT1 -- TrueNAS on 10.0.74.12, NAS on 10.0.74.11 and iMac on 10.0.74.14 -- neither of them is accessible while I'm pinging them on my macbook on wifi via its terminal. Both subnets are /24, 255.255.255.0.
We go over this like every other day it seems.. Your saying from the lan, you can not even ping pfsense IP on your opt1 interface?
When I'm on LAN, I can't ping anything on OPT1 EXCEPT pfSense -- it is on 10.0.73.1 (LAN) and 10.0.74.1 -- I can ping 10.0.74.1 when I'm on 10.0.73.xx and I can access 10.0.74.1 on web interface, but other 10.0.74.xx devices are unreachable. And -- when I log on to pfSense via web interface and go to Diagnostics > Ping, and try to ping 10.0.74.12 (TrueNAS), 10.0.74.11 (Syno) and 10.0.74.14 (iMac) -- 100% packet loss. Again, this applies only when my Macbook is on LAN (wifi/wired -- doesn't matter).
Is your client on lan your pinging from maybe on wifi and wire at the same time? Is it running some security software that would stop you from pinging stuff?
No, either Wi-Fi either eternet (when trying to reach devices on OPT1 while on LAN)
Did you set its IP up, maybe you have a mask wrong?
My laptop's IP varies -- static on WiFi (DHCP Static based on MAC address), random on wired LAN (DHCP), static on OPT1, but, as I said, this whole issue goes away when I work wired to OPT1.
Do you maybe have some wifi router with a leg in lan that might have its dhcp server still running?
No, my Tenda Mesh Wi-Fi is in bridge mode, all routing/IP stuff is done on pfSense.
Thank you very much! ))
-
Forgot: there are no deny rules on OPT1, only allowing ones )
-
Dang! I connected my macbook to OPT1 by wire, checked that I can access everything on OPT1 (as expected). Disconnected the cable, went back on LAN Wi-Fi and now I can reach OPT1! It could be something with my laptop but it's not! When I had Mac mini wired to LAN it too couldn't get to OPT1 most_of_the_time and how on earth this situation could affect Diagnostics > Ping from web interface? If I'm logged on to pfSense Web GUI that means I'm inside the router and it should ping anything, but no, it can't ping OPT1 devices that "most of the time" but now, when I plugged my Macbook to OPT1 for a few seconds and disconnected -- everything works! I'm no IT specialist but I think this situation can be considered stupidly weird by an IT specialist either ))
-
@throttlenerd so question are you using say the new ethernet rules? Or captive portal?
have you setup any static arp entries?
I'm inside the router and it should ping anything, but no, it can't ping OPT1 devices that "most of the time"
This is true - from the firewall you should be able to ping anything connected to networks pfsense is attached too. Reasons why you might not be able to, firewall rules on the device your trying to ping. Not able to arp for the devices IP, or wrong mac in the arp table for that IP..
I was thinking what could cause interment sort of issues, invalid mac comes to mind, duplicate IP where you have the correct mac for what your wanting to talk to, and then other times have wrong one, etc..
There is has been some ongoing issues where if you set a static mac it goes away and looks like just dynamic mac, and then can set back to static on restart of setting the static arp, etc.
-
@johnpoz All devices on OPT1 have static IPs and they are set up in those devices' settings, no MAC-based static leases. I have MAC-based static leases on LAN interface -- for my laptop, mobile phones, Wi-Fi AP (3-unit mesh). Today I figured out the problem goes away if I (by "I" I mean my laptop) connect to OPT1 by wire and then unplug it and connect to LAN wirelessly -- everything is accessible. But since "most of the time" this issue is "live" -- I will ssh into my other NAS which is on LAN (Surveillance, Plex) and will try to reach my work NAS on OPT1 -- it shouldn't be able to ping it even though right now it can, because I was on OPT1 a few minutes ago. Yes yes this is weeeeird. Will keep you posted thank you very much for helping me! Happy New Year! ))
-
So I waited a bit, now the issue is alive again -- no LAN to OPT1 access. Right now I'm (laptop) on Wi-Fi. I turned on SSH on my other Syno NAS which is on LAN. I open Terminal on my Mac, ssh into 10.0.73.12 (LAN NAS), pinging 10.0.74.1 (pfSense on OPT1) -- all good. Pinging two devices on OPT1 -- no luck. But I was able to ping them both from my mac and/or LAN NAS just after disconnecting my laptop from OPT1. So, ehmmmm. Still more questions than answers ahaha )
-
@throttlenerd you sure your actually isolated at layer 2?
You shouldn't be seeing the mac of the devices if your on another L2 network.. I would plug your laptop back into opt network.. ping your devices, now you say you can ping them while your on lan? Look in your mac address table, a simple arp -a should show only the stuff that is on your current network.. You should no longer see mac from devices on opt network.
-
But I was able to ping them both from my mac and/or LAN NAS
Oh by saying "mac" I meant my Macintosh laptop )) The only time I dealt with MAC addresses was when I assigned static DHCP leases to some devices on LAN interface.
Yes, arp -a while I'm on LAN shows only 10.0.73.xx (LAN) devices, no OPT1
