Ports some days opened, other days all closed
-
Good to hear I'm now alone!
Sounds like a firewall issue indeed.
I'm also up to date and not running any additional plugins, so no squid.
I boot and shutdown using IPMI (Super Micro)Question is how to debug this?
-
I tried digging around SystemAdvancedNetworking
If I set "disable Disable hardware checksum offload" I'm cut off the firewall: only the 443 openvpn rule is working but even if i can successfully connect to the vpn my lan subnet is unreachable. I can only see the webgui.
-
I would check the pfSense bug-tracker to see if this is a known bug, but I would focus on user error as well since both of you seem to be new to pfSense.
I know that I always have strange things happen when I first begin using something that is unknown or new to me.
-
Don't mess with advanced settings trying to troubleshoot something that's hit and miss as described, at best it won't do anything, at worst it'll make things worse and harder to troubleshoot.
When it's not working, packet capture on WAN filtered on the port in question, see if it's getting to you at all. If not, problem is upstream. If so, see if a state is created, see if the internal host is replying.
-
The problem is difficult to reproduce and is not always the same. Every time I reboot I keep my finger crossed because I'm not sure that all networks interfaces will be working fine.
I got the followings cases:
A) Firewall is up but unreachable from lan. DHCP is working and I get internet connection, but nmap from host to firewall shows host down.
B) From LAN everything is ok, but all forwarded ports are closed from wan. Tested with Shields Up - GRC.com and everything is stealth.
C) From LAN is ok and wan forwarded ports are ok, but from secondary LAN the firewall is unreachable. The clients get IP address like case A but the captive portal is unreachable so no internet connection. I tried to add MAC address to authorized hosts but still no internet.
Every time after several reboot the firewall start to work fine again.
Once I had to restore a previous backup to get things work again, but I had not changed the configuration between.My dmesg
http://pastebin.com/N4sisRx8 -
The problem is difficult to reproduce and is not always the same. Every time I reboot I keep my finger crossed because I'm not sure that all networks interfaces will be working fine.
I got the followings cases:
A) Firewall is up but unreachable from lan. DHCP is working and I get internet connection, but nmap from host to firewall shows host down.
B) From LAN everything is ok, but all forwarded ports are closed from wan. Tested with Shields Up - GRC.com and everything is stealth.
C) From LAN is ok and wan forwarded ports are ok, but from secondary LAN the firewall is unreachable. The clients get IP address like case A but the captive portal is unreachable so no internet connection. I tried to add MAC address to authorized hosts but still no internet.
Every time after several reboot the firewall start to work fine again.
Once I had to restore a previous backup to get things work again, but I had not changed the configuration between.My dmesg
http://pastebin.com/N4sisRx8Why does nmap fail? No ping response?
Can you share your network topology?
-
Yes, no ping response. In this case I cannot ping any lan/opt2 host from the pfsense console too. And DHCP and nat are working because I have internet on lan.
WAN (wan) -> pppoe0 -> v4/PPPoE: 88.149.xxx.xxx/32
PRIMARY BROADBAND CONNECTIONLAN (lan) -> re0 -> v4: 192.168.1.250/24
LAN CONNECTION. SQUID ENABLED BUT NOT TRANSPARENTOPT1 (opt1) -> ppp1 -> v4: 10.90.209.120/32
BACKUP MOBILE CONNECTION (/dev/cuaU0.0)OPT2 (opt2) -> re1 -> v4: 192.168.2.250/24
GUEST CONNECTION WITH CAPTIVE PORTAL AND TRANSPARENT SQUIDOPENVPN 172.16.0.0/24 on port 443 TCP, shared with a port forward
configured through wizardGateways
WAN_PPPOE 81.174.0.21 8.8.8.8 17.315ms 13.176ms 0.0% Online Interface WAN_PPPOE Gateway
OPT1_PPP 10.64.64.1 8.8.4.4 54.214ms 4.702ms 0.0% Online Interface OPT1_PPP GatewayFirewall (no traffic shaper, no schedules, Automatic outbound NAT, no floating rules)
Please ignore the duplicated ssh entry: I was using a mobile browser and my tap was not precise. Now there's again only one ssh forward.
-
I upgraded this morning and did the capture (see attachment). Ports did arrived in the capture. Strange this is this time I DID see the traffic in the firewall logs.
Now the strange thing: it was blocked because the destination wasn't 10.0.0.107. Now that's a rule PFsense created himself by the NAT rule.
After adding the same rules but with a wildcard destination it all seems to work for now. It was blocked because the destination wasn't the local address, but the public IP.
Now I can understand it was blocked, only it's strange that:- The NAT rule used the local destination and not a wildcard? And you can't change it also because it's greyed out.
- Most strange things: it used to work 50% of the days and 50% of the days it didn't. So why did it worked sometimes? Before the wildcard destinations didn't exist.
- Why wasn't this logged in the firewall logs in previous version?
Firewall now:
-
Seems that it isn't solve. Problem got worse in that case that instead of closed ports I now end up with opening my firewall to the internet: https://forum.pfsense.org/index.php?topic=114731.0
-
You for sure do not want or need wild card dest for your nat stuff. Your nat rules go first, then that traffic would hit your firewall and its dest would be where the nat is sending it.
So I am curious how many connections do you have open due to the tarpitting? Maybe your running into a connections limit issue? Or issue with memory due to connections staying open?
-
Regardless of what's been said here I'm 100% sure that pfSense is not the problem unless there's some serious misconfiguration involved. Start looking at the equipment that is in front of pfSense on the WAN side, contact your ISP and ask if they can assist with the problem in case there's something wonky on their side.
-
Well first thing I would fix is a HARD boot everyday?? For what possible reason would anyone do that, does power go off? That for sure needs to be fixed!
2nd thing would be anything enolved with dynamic IP - is your IP changing? So if some fqdn points to 1.2.3.4, your IP changes to 5.6.7.8 no shit forwards wouldn't be working if still going to 1.2.3.4 ;)
-
Hi
The box goes off for about 7 hours a day because it's running off grid on batteries with solar cells. I think that shouldn't be a problem?
The IP is dynamic, but changes at night when it's powered off. I can see this in the statistics of my ISP.
My ISP is a big one and doesn't want to help me: what happens behind there lines is my problem they say (don't have much choose of ISP in Belgium).
The reason I added the wildcards is because it doesn't work if it isn't. In my previous post you can see a ZIP with traffic captures.
As you open that in Wireshark, you can see it was blocked because the destination isn't 10.0.0.107 but the public dynamic IP. Hence the wildcard.
The capture was done on pfsense itself and then exported and opened locally with Wireshark.
Or is there another way to solve this?
-
What I can tell you for FACT is that when you do a nat, you do not need a rule with dest Any.. You need a firewall rule that is to the IP to where the nat is sending the traffic. That is how it works, that is how the nat creates the rule when you create the nat. That is how mine are setup and they work just fine.
I don't know what you saw in your traces. Most likely you did not have rule or nat active? See example below, I created a nat to 22, it created rule on my wan that says the private IP. As you can see from the sniff on my wan the packet is set to my public IP.. but still gets through to my box behind on the private.
So if your power goes off because its on battery you need to look into doing a graceful shutdown.. Hard off anysort of device is rarely a good idea. Especially when the media the device is running off of it writable.. Is this running off some ro cf card or something?
If the battery runs out of juice, I would setup so your devices gracefully shutdown before the juice runs out.. And then comes back online when there is sufficient juice to run.. Or get a bigger battery to last while the sun is down ;)
-
Hi
I removed the wildcards and I changed the shutdown script to a shutdown -p now and now all ports are blocked again some days (not all days).
The days they are blocked, it shows in the Firewall log? So they are blocked because the destination is my WAN address?
What extra rule do I need?
-
None of those rules show any hits on them at all.. Why is floating and Wan lite up.. Do you have any rules in floating?
What rule is blocking those. Turn on showing the rule in the log..
Status / System / LogsSettings
Are you seeing any errors in system log about the loading of the rules?
-
What do you mean by "Why is floating and Wan lite up"?
When I click "floating", the list is empty:No floating rules are currently defined. Click the button to add a new rule.
.
Which option in Logs > Settings do you mean?
There are errors, but they seem to be IPv6 related, so not linked I guess:
-
Nothing with "rule" in it after booting:
-
The one in the picture I attached ;) And the one you have set to not show… What I meant by lit up is they are both have the red bars under them..
So your log is flooded with this?? That doesn't seem right. But again in that picture you have both general and resolver lit up.. So what is that log from general or resolver.. Only the tab your on should be lit..
if your logs are being flooded how do you know if your rules failed to load? Or issues with them..
-
Ah ok, I understand! They both light up or have the bar below them because WAN was active and I was hovering Floating at the time of the screenshot. Sorry for the confusion.
It was a screenshot of the general. DNS resolver was just hovered by the mouse.
My second screenshot was after a reboot, before the IPv6 errors. I did a search and no logs mentioned "rule" in the description.
