NG6100 MAX - pf+22.05 "wired" memory increasing over time
-
I believe we have found an issue in the code that pulls ethernet rules. It's particularly apparent where captive portal is running which is probably why you're seeing it more than most.
So we shouldn't need any more data but I'll reach out if we do.Yes, this:
https://github.com/freebsd/freebsd-src/commit/0044bd90f2397dfad5f4bbd12c64be86e0b7eb4a -
@stephenw10
Thanks for getting another cp related bug squashed.Will contact you again when I stumble upon the next rodent in the attic ....
-
@stephenw10 is there a redmine to follow up on the status of this bug?
(Didn't find one myself - but didn't look very hard) -
No, Kristof just went straight to the source! But there should be one to track it. I'll add one....
-
@stephenw10 when will a patch be released?
just got called in because school went offline:
tried to ssh remotely to reboot - then it got stuck somewhere
had to drive on site to powercycle the ng6100(yes i forgot to reboot before memory ran out ... )
-
It's patched in 23.01 if you're able to test that?
It's unlikely to be patched in 22.05 at this point because it would require a point release, the fix is in kernel.
Steve
-
@stephenw10 said in NG6100 MAX - pf+22.05 "wired" memory increasing over time:
It's patched in 23.01 if you're able to test that?
Is this Patch also in 2.7.0?
-
-
@stephenw10 test 23.01 on a production system with roughly 1k users ?
also: it's not mentioned in the 23.01 release notes posted alongside the beta announcement
-
Likely because it hadn't been marked as feedback yet. It is in 2.7 and 23.01 though. I've updated the ticket.
Steve
-
We need someone who was hitting this in 22.05 to test it in 23.01 to confirm it is fixed there.
Otherwise if it's not fixed in the current code it may not be before release.
Steve
-
@stephenw10 is netgate willing to verify/test the config of the school that hit this issue on 23.01 before i attempt the upgrade?
i can not afford to upgrade to a non/semi-working system that has bigger issue's then what i currently have.
so the verify / test would have to include the upgrade itself & basic connectivity appears to work in a lab environment. there are no addon packages installed.also: schools in belgium close down for the holidays. So between now & january 9th there will be no clients.
memory seems to increase only when there are "lots" of clients connected to CP. i can not observe memory-leakage during the weekends.also2: i read that the current builds still have some debugging enabled that cause decrease performance. how much % are we talking about on a ng6100?
conclusion: i'm willing to test this from january 9th if netgate can confirm there don't appear to be any issues upgrading from 22.05 to 23.01 with the config i have.
-
Well I can test a config in a 6100 running 23.01 but I obviously can't test it in your network environment.
-
@stephenw10 what's the preferred way to send you that config ?
-
You can upload it to me directly here:
https://nc.netgate.com/nextcloud/s/6ZZEPy4RsnWi6mb -
@stephenw10 you probably received the config at the link you provided.
there is a cron to restart dpinger to work around a different bug.
believe its https://redmine.pfsense.org/issues/11570 - anyhow in some scenarios dpinger keeps saying gateway is offline but in reality gw has been back up for a while - it has something todo with a PID getting stuck or something ?)could you let me know if the provided config seems to work on the alpha/beta/rc versions (whatever it is at this time).
-
The config looks good. It installed all the packages. The only issue I saw importing it was I had to reboot to populate the aliases. On the first boot there were a number of alerts indicating they had not been populated yet.
Is there anything specific you want me to test here? -
it would be good if the interfaces work and it doesn't crash :)
what is the expected performance hit with the debugging still enabled ?
-
Well it's been up for 20hrs without crashing but that's with no traffic. The interfaces look to be configured correctly but without your infrastructure connected to it I have no way to really test that.
I guess maybe the VLAN_BEZOEKERSWIFI interface with the captive portal on it is the one you might most expect to have issues? -
@stephenw10 said in NG6100 MAX - pf+22.05 "wired" memory increasing over time:
I guess maybe the VLAN_BEZOEKERSWIFI interface with the captive portal on it is the one you might most expect to have issues?
i hope i won't have issues with the interface itself.
i'm guessing there will be issues with CP ... that seems to be a trend ;)would it be possible to easily rollback using the zfs "snapshot"/boot environment thing that is builtin ? (in case of catastrophic failure during/after the upgrade)
does it still handle >=1Gbit NAT with debugging enabled?
