I just had a client visit yesterday that this appears to be very similar:

Default WAN port flaps (Flapping) every 1min or less

resulting in a state of constant resets that it cannot stabilize to the second WAN interface and the network is Internet dead.

Hardware is a Supermicro - X10SDV-TLN4F – D-1541 2.1-2.7 Ghz 12MB L3 -  8 cores / 16 threads

WAN0 -  is igb0 - DHCP  - Comcast Business class gateway/router in Router mode - 10.1.10.1/30
WAN2 - is igb1
LAN1  - ix0
LAN2 -ix1 (empty port)

Steps to Resolve:

1.  pulled down pfSense 2.3.4-Release box - Tested with laptop direct on same cable and port to Comcast modem - no problems - stable.

2.  Repowered Comcast modem and put pfSense box back into the mix per above and flapping started immediately on WAN0 - igb0

3. Decided to test another port on Comcast - no change in status

4. Changed pfSense WAN0 port from DHCP to Static 10.1.10.2/30 and it stabilized and forced set the gateway IP to 10.1.10.1. - rest i believe were unchanged default settings.

Here is the key part of the log IMHO:

Jun 27 11:55:52 php-fpm 88701 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 10.1.10.2 -> 10.1.10.2 - Restarting packages.
Jun 27 11:55:51 kernel igb0: link state changed to UP
Jun 27 11:55:51 check_reload_status Linkup starting igb0

Gateway Log entries that repeat over and over seconds back to back:
Jun 26 12:18:26 dpinger WANIGB0COMCAST_DHCP 73.211.120.1: sendto error: 65
Jun 26 12:18:25 dpinger WANIGB0COMCAST_DHCP 73.211.120.1: sendto error: 65
Jun 26 12:18:25 dpinger WANIGB0COMCAST_DHCP 73.211.120.1: sendto error: 65
Jun 26 12:18:24 dpinger WANIGB0COMCAST_DHCP 73.211.120.1: sendto error: 65
Jun 26 12:18:24 dpinger WANIGB0COMCAST_DHCP 73.211.120.1: Alarm latency 22427us stddev 5835us loss 50%
Jun 26 12:18:24 dpinger WANIGB0COMCAST_DHCP 73.211.120.1: sendto error: 65
Jun 26 12:18:23 dpinger WANIGB0COMCAST_DHCP 73.211.120.1: sendto error: 65
Jun 26 12:18:22 dpinger WANIGB0COMCAST_DHCP 73.211.120.1: sendto error: 65
Jun 26 12:18:22 dpinger WANIGB0COMCAST_DHCP 73.211.120.1: sendto error: 65
Jun 26 12:18:21 dpinger WANIGB0COMCAST_DHCP 73.211.120.1: sendto error: 65
Jun 26 12:18:20 dpinger send_interval 2000ms loss_interval 8000ms time_period 240000ms report_interval 0ms data_len 0 alert_interval 4000ms latency_alarm 500ms loss_alarm 40% dest_addr 10.5.0.1 bind_addr 10.5.22.1 identifier "LAN3igb1GW "
Jun 26 12:18:20 dpinger send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 73.211.120.1 bind_addr 73.211.120.82 identifier "WANIGB0COMCAST_DHCP "

Doing searches on 11:55:52 line item - /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection:

A.  Bug #4474 - OpenVPN client connection causing - this was not the case for me at this time of the error - though OpenVPN is set and actively listening.  I am remote OpenVPN now getting this log message and the logs are not showing this bug error.

B.  Bug #6656 - similar?

My thought is that "rc.newwanip" code does not run once the interface is set for static ip versus DHCP?

Ok, kinda solved.

I had changed the size of the log files and saved that and reset the log files (back on the 19th). At the time I initially made them 1GB each and quickly decided that might be a bit too big. I went thru the process and changed them down to 100MB.

Tonight I thought I would change them again and have further reduced it to 50MB and they are now logging again.

I may try and raise it a little and play with the size and see what happens, but I'll leave that for another time. For the moment, it works with 50MB log files.

You might get a better response if you provide details regarding your config on both ends .  The request is very vague and difficult for someone to respond without understanding how you have configured you're a piece of connection.  Vague questions are usually answered with Vague answers

Not sure abou VMware but virtual machines running pfsense (I use Hyper-V) and I have never had an issue so would agree with jimp.

If you are looking for this for a single site you are better off using something like a cisco firewall which can provide reporting.  If you are looking at this to provide as a solution for MSP or many locations you might be able to post a bounty though that might be difficult or costly.

Thanks for all the replies. I went the 'nuclear' option by doing a complete re-install. Of course I was an idiot and didn't have a recent backup of the config so I had to run a memstick installer in recovery mode to copy the latest config first from the drive.

I've also hard pulled the plug hundreds of times on this bare metal box without issue…so idk.

I am back to using fwbuilder.  Not really much benefit using pf.

I did a litle workaround…

I edited the file /usr/local/etc/raddb/scripts/otpverify.sh and inserted the string below at line 86:

echo "FreeRADIUS: Authentication failed! Too many wrong password attempts. User is locked! To unlock delete /var/log/motp/users/$USERNAME" | mail.php -s"FreeRADIUS alert"

PS: The mail settings are configured under System/Advanced/Notifications

Thanks

I am using ping from pfsense to switch, ping from 2 machines to the gateway and switch. (direct connected patch cord 1,5m cat6)

from any manchine to switch is always 1ms, only pfsense seens to have variations.

I changed the cable, port, even to another switch but any ping from pfsense or to pfsense is instable, for me it looks like a software problem because its only start happing after all services from pfsense is up.

But I disabled almost all non essentials services but no luck.

my nic has 4 ports, 3 wan ports with avg of 0,5ms!!!! I even changed ports to see if anything change, no luck!

From this i'm guessing it's related to the IPSEC and openbgpd issue that's ongoing.
https://redmine.pfsense.org/issues/6223

I was forcing a tivo through an OpenVPN that egresses from AWS Oregon until about a week ago and it worked fine for geo-shifting MLB.Tv. Probably just a matter of time. (Don't have the tivo any more.)

Didn't try any other streaming services and tivo updates seemed to be fine.

Hard for me to fathom why tivo would care where you get updates from. The streaming apps all have their own enforcement methods I would think.

You could tailor the rule to only put traffic sourced from the tivo and destined for port 8080 out WAN.

And once you're done studying up on that, check out the ACME Package so you can easily get a free trusted certificate for your firewall:

https://doc.pfsense.org/index.php/ACME_package

Not at the moment.

@kpa:

@Harvy66:

My laymen's understanding. It's not an inherent security flaw, it just means one of the anti-exploit defenses does not work as well as expected.

It is definitely an inherent security flaw. An unprivileged process should never be able to play games with the system's memory management and trick it into allocating more stack pages from an area of memory that the process already had access to. If the attacker can do that it opens up many opportunities for compromise because the stack contains the return addresses for function calls and if you manage to manipulate those anything is possible. The classic case is the (possibly the world's first such incident) Morris worm:

https://en.wikipedia.org/wiki/Morris_worm

Yeah, turned out it was something more nefarious. It wasn't just about smashing stacks in an application's own virtual memory, but being able to access kernel memory, allowing for priv esc attack.

i have done the following and it works:

NAT - Port Forward:

Interface: the interface the dash buttons are on (wifi-net)
Protocoll: Tcp
Source Adress: The IP of the Dash button
Souce Ports: *
Destination Adress: *
Destination Ports: 443 (as the dash buttons try to establish a ssl-connection to amazon when pressed)
Nat IP: The IP of the Computer on the Net which shall receive the info that the dash buttons try to connect to the internet aka have been pressed
NAT Ports: 4321 (anyone does, no portrange needed, as the buttons only try to connect to :443)
Corresponding Firewall Rule: Pass

On the Nat IP-machine I can receive the connection requests using scapy in python:

from scapy.all import *
p = sniff(filter="tcp and port 4321", store=0)
print p[IP].src

Every Button Press generates 5 requests.

Problem : Scapy uses a lot of ressources, will take ~30% CPU on a Raspy B.

Problem2: I didnt manage to use socket  module, as the buttons dont really connect, they just send ssl-syn and receive some multiple acks from the nat-ip.

Here's what Wireshark shows (running on the Nat-IP machine; *.127 is the dash button, *.125 is the Nat-IP client machine):
https://ibb.co/hwwi55

@fleece:

My son stays up too late gaming.  Could I use pfSense to restrict his Internet access during days of week and time of day, say from midnight to 6AM?  I can give him the same IP address through reserved DHCP or something.

Yes.

In Services/DHCP server you can give your son a static IP.

Then, in Firewall/Schedules you can create a schedule.

Then, in Firewall/Alias, you can create aliases with adresses your son is allowed to go to (the gaming, for example).

Finally, in Firewall rules, you can:

1. Add the alias to allow him to game;
2. Add, in advanced settings (at the bottom) the schedule) which limits the time he can do that.

So after that time, the can still google his home work (sorry, I still can't live with that thought, I'm old fashioned, back in my days we had books  :-[ ) but can't game.

Or, of course, even beyond that: he can't internet at all.

Or, beyond that, with two schedules:

Firewall rule 1: he can game until 4 PM with a schedule.
Firewall rule 2: he can game from 9 PM-10PM with a schedule.