performance impact of clicking "apply changes"

binary_bandit

Hi All,

Can anyone give me feedback on how much packet loss I should expect when clicking "apply changes" after modifying an alias of a firewall rule?

If I'm pinging a host from outside of the LAN about 50% of the packets drop for about 10 seconds. This is confirmed by dropped VoIP calls as well as screen sharing sessions.

As a workaround I've set all firewall changes to occur after hours.

Should this be expected though? When changes are applied is it normal that data flow is interrupted?

thanks,

James

JKnott

@binary_bandit said in performance impact of clicking "apply changes":

As a workaround I've set all firewall changes to occur after hours.

That should always be the norm. You don't make changes when the network is busy.

binary_bandit

Thanks @JKnott. Good change management is critical, agreed.

Can anyone share their experience with the technical impact of clicking the "apply changes" button?

I have a feeling that the 10 seconds of packet loss are normal but would like confirmation.

SteveITS

Are you on 2.4.5? https://redmine.pfsense.org/issues/10414

binary_bandit

@teamits, you are the man!

That bug describes much of what we're seeing.

I've been combing our logs and cleaning up a few things that are showing up but I'm 99% sure that what we are seeing is related to this bug.

We're running on Proxmox, using 2.4.5, PFBlocker and blocking bogons .... we check all the boxes described in the bug. Oh, and when I upgraded the firewall it took 15+ minutes for the first boot followed by what felt like another 15 min for the CPU utilization drop and the firewall to process packets.

I'm considering the workarounds now.

best,

James

binary_bandit

update:

Yesterday evening I shut the firewall down, dropped the CPUs from 6 to 1 and rebooted it.

It was noticible how much faster it booted. Before this there were two points in the boot where the firewall paused when loading firewall rules ... I'd have to watch a boot again to identify the exact message .... regardless this did not occur with only 1 CPU assigned to the VM.

I'll report back in a week if we're in the clear or sooner if the issue that we were seeing repeats itself. Assuming that I have reason to change a firewall rule again, I'll test the packet loss as well.

binary_bandit

We're up and running with zero issues. Dropping to 1 CPU has definitely solved our problems.

I've done some testing for packet loss when applying firewall rules. This has been eliminated as well.

@teamits, it looks like the issue marked as resolved in the link that you posted:
https://redmine.pfsense.org/issues/10414

What does the process to move this to an update for 2.4.5 look like? I imagine that I can download and apply the fix without waiting (I see a link in the issue) but I'd like to know when it will show up in our firewall's software update status.

Thank you again for posting that link.

SteveITS

@binary_bandit said in performance impact of clicking "apply changes":

What does the process to move this to an update for 2.4.5 look like

Don't know, I would think if it was easy Netgate would have a 2.4.5-p1 out pretty quickly once FreeBSD was updated, at which point it would show on your dashboard. Otherwise I suppose you'd have to figure out what needs to be compiled/updated for the FreeBSD used by 2.4.5.

The other workaround discussed here in various posts is to reduce the number of entries used, e.g. turn off the bogons block for IPv6 and potentially pfBlockerNG if that's in use.

binary_bandit

Got it. I have no issues waiting for p1 to appear I'm just trying to look at this as a learning opportunity.

Thanks for all your help @teamits.