Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0
-
@netblues I do not reveal any specific IP addresses, especially one of an IT workstation.
I'll retest with HTTP
-
@netblues Still slow with HTTP, must not be isolated to HTTPS.
-
@sweber Any chance there is odd routing going on? You're connecting to the Management interface LAN IP?
It's normal that without rules no traffic is allowed on an interface. The exception is LAN which has a default allow-to-any rule.
The one time I've seen slow connections/dropped packets is a setup that has when trying to access the second router in an HA config from outside that office. I have not spent any time on it because I can just connect in to a PC in that office, but I figure it's a routing problem because of the setup. For the primary I can access IP:port1 which we NAT to LANIP:443. For the secondary it is set up to NAT IP:port2 to LAN2IP:443 but it isn't very functional...partial page loads, slow, etc. (the client has a web server on WAN:443, and the pfSense routers use private WAN IPs with the public IPs via CARP, so it's a bit of an odd setup)
Don't know if it helps but I found thread https://forum.netgate.com/topic/103666/ssl_write-failed-ssl-issues-on-secondary-node/6
-
Is it slow to every page or just the initial connection?
That sort of sounds like the behaviour you see when the client has an IPv6 address nut cannot connect using it. Are you trying to connect using the host name or IP?
What firewall rule was it hitting that was blocking it initially?
What rule did you add?
What rule was passing that traffic in 2.5.2?
Steve
-
@stephenw10 Slow to every page, sometimes I can at least log into the dashboard, but can't go anywhere else after that point, and sometimes it just hangs while logging in.
There was no firewall rule blocking it initially, I have a rule on my LAN interface that allows IT workstations to connect to the MANAGEMENT subnet with HTTP, HTTPS, etc, so traffic coming out of LAN (where my computer is) is PASS to anywhere (including my secondary pfSense instance). Before, my secondary pfSense wouldn't block it, but now it's blocking it using a rule I don't even see in my ruleset, it's bizarre.
I added a rule in the MANAGEMENT interface on my secondary pfSense instance that's a complete copy of the original rule in LAN.
The original rule above was letting traffic pass for IT.
-
@sweber said in Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0:
(including my secondary pfSense instance)
How about you draw out what you have actually setup... Because its not just a single pfsense, sounds like you might have asymmetrical routing, etc.
How exactly is this connected together?
-
@johnpoz PF01 is currently 2.5.2, I'm testing 2.6.0 on PF02:
(When I wrote "FINE" or "Laggy", I'm referring to accessing the webconfigurator through that interface)
-
Yup, I'd guess asymmetric routing.
You try to access PF02 on the management interface from a client on LAN. The client uses it's default gateway to reach it, which is the PF01. PF01 routes it to PF02 via the MGMT interface. But then PF02 tries to reply directly to the client because it has an interface in the LAN subnet.
That is blocked outbound on LAN in PF02 because it doesn't have a state for that. Hence the permission denied error from nginx.
So I would expect that to have shown blocked outbound in LAN in the firewall logs on PF02?Nothing should have changed since 2.5.2 though.
Steve
-
@stephenw10 Interesting, although nothing changed since 2.5.2 we've never had this issue and my rules didn't change, so what happened? That's the question...
Both firewalls are using pfSync, and I didn't find any outbound connections being blocked on PF02 going back to my client PC.
I checked all the logs for services n such on PF02 after I updated it, but just the fact that I'm having this issue is making me nervous on relying on it for failover, haha! I didn't change anything on 2.5.2, and after PF02 upgraded to 2.6.0, I started having this webconfigurator issue.
-
I just double-checked and all my settings, firewall configurations, everything, is as it was before on 2.5.2 and also identical to PF01's configuration (where they sync).
Something must've happened with this update, either a change/fix in the update or an error while it was updating, that's causing this.
-
I temporarily allowed ICMP to PF02 and perform another tracert from my workstation to PF02, and it reached successfully as it should through PF01:
I keep doing multiple tracerts and every time it routes the same, so this seems like an issue on the pfSense side on PF02.
This issue
-
@sweber said in Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0:
I checked my firewall and noticed it was blocking the connection from my workstation to the firewall through the MANAGEMENT interface.
Do you have those block logs still or can you recreate them? Is that the only blocked traffic you saw on either node?
Am I correct that you are testing this from a client in the LAN subnet? Only there?
Steve
-
@sweber said in Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0:
PF02, and it reached successfully as it should through PF01
But what about the answer? Since pf02 has an interface in 10.30.0..
-
@stephenw10 The only blocking I saw was on PF02 on the MANAGEMENT interface after it came from PF01. I'll also try accessing from a client on MANAGEMENT out of curiosity, give me a few minutes.
@johnpoz I'll packet capture and look at the answer
-
@sweber yeah such a setup would scream asymmetrical to me
Why are you not just accessing the gui via the Lan IP of pf2?
-
@johnpoz I just tested directly on the MANAGEMENT interface and it's working fine, so it's some routing issue, but again nothing changed so it's so weird it's suddenly not working anymore with our design we've had for years
Gonna capture some packets
-
I can confirm that PF02 is talking back to my workstation on the SECURE interface on PF02, bu it seems to only work when I'm on the login page, enter credentials, and log in to the dashboard; that's as far as I can get. If I try going into any other menu, it just spins and loads forever, the URL doesn't even populate in the address bar, and I don't see any firewall logs showing my workstation talking to PF02 asking for the page and PF02 responding on SECURE.
-
Ohhh, interesting thing!
Once i load into the dashboard, when I try to access other pages, those following connections to PF02 are getting blocked on PF01. So, the initial connection is using TCP:S to log in, then access the dashboard and everything else is TCP:A? I've never seen this before...
Here's the rule PF01 is saying blocked TCP:A:
That's the weird thing, we've always had a rule allow TCP/UDP from IT Workstations to Management net and management ports (HTTPS, SSH, all that jazz):
So why in the heck is it blocking it if I have an allow rule for TCP? (That should apply to all subs of TCP, including TCP:S, TCP:A, etc, right?) -
@sweber said in Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0:
So why in the heck is it blocking it if I have an allow rule for TCP?
When you see a block on non Syn, you either have asymmetrical traffic or the state went away.. The state is opened by the syn.. If the state goes away and the client continues to send traffic it would be blocked because there is no state.. This would be blocked by the default deny.
Are you doing outbound filtering? those little arrows mean outbound filtering..
-
@johnpoz Outbound filtering? Not that I know of, I don't recall configuring that.
So, this must mean the connection state between my workstation and PF02 is getting dropped, and that's why it's breaking? I wonder what would be causing that..
