Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0

johnpoz

@sweber said in Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0:

PF02, and it reached successfully as it should through PF01

But what about the answer? Since pf02 has an interface in 10.30.0..

ColdBrew

@stephenw10 The only blocking I saw was on PF02 on the MANAGEMENT interface after it came from PF01. I'll also try accessing from a client on MANAGEMENT out of curiosity, give me a few minutes.

@johnpoz I'll packet capture and look at the answer

johnpoz

@sweber yeah such a setup would scream asymmetrical to me

Why are you not just accessing the gui via the Lan IP of pf2?

ColdBrew

@johnpoz I just tested directly on the MANAGEMENT interface and it's working fine, so it's some routing issue, but again nothing changed so it's so weird it's suddenly not working anymore with our design we've had for years

Gonna capture some packets

ColdBrew

I can confirm that PF02 is talking back to my workstation on the SECURE interface on PF02, bu it seems to only work when I'm on the login page, enter credentials, and log in to the dashboard; that's as far as I can get. If I try going into any other menu, it just spins and loads forever, the URL doesn't even populate in the address bar, and I don't see any firewall logs showing my workstation talking to PF02 asking for the page and PF02 responding on SECURE.

ColdBrew

Ohhh, interesting thing!

Once i load into the dashboard, when I try to access other pages, those following connections to PF02 are getting blocked on PF01. So, the initial connection is using TCP:S to log in, then access the dashboard and everything else is TCP:A? I've never seen this before...

Here's the rule PF01 is saying blocked TCP:A:

That's the weird thing, we've always had a rule allow TCP/UDP from IT Workstations to Management net and management ports (HTTPS, SSH, all that jazz):

So why in the heck is it blocking it if I have an allow rule for TCP? (That should apply to all subs of TCP, including TCP:S, TCP:A, etc, right?)

johnpoz

@sweber said in Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0:

So why in the heck is it blocking it if I have an allow rule for TCP?

When you see a block on non Syn, you either have asymmetrical traffic or the state went away.. The state is opened by the syn.. If the state goes away and the client continues to send traffic it would be blocked because there is no state.. This would be blocked by the default deny.

Are you doing outbound filtering? those little arrows mean outbound filtering..

ColdBrew

@johnpoz Outbound filtering? Not that I know of, I don't recall configuring that.

So, this must mean the connection state between my workstation and PF02 is getting dropped, and that's why it's breaking? I wonder what would be causing that..

netermin

@sweber Do you have the zabbix agent installed?

johnpoz

@sweber said in Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0:

Outbound filtering? Not that I know of

Do you have rules in floating?

ColdBrew

@netermin No Zabbix.

@johnpoz No rules in floating.

johnpoz

@sweber said in Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0:

No rules in floating.

Well have no idea how your blocking on the outbound direction then? Lack of state? The default deny rules are in/out

if you look in the full rule set

#---------------------------------------------------------------------------
# default deny rules
#---------------------------------------------------------------------------
block in  inet all ridentifier 1000000103 label "Default deny rule IPv4"
block out  inet all ridentifier 1000000104 label "Default deny rule IPv4"
block in  inet6 all ridentifier 1000000105 label "Default deny rule IPv6"
block out  inet6 all ridentifier 1000000106 label "Default deny rule IPv6"

talking back to my workstation on the SECURE interface on PF02,

What interface is that? lan or magement? Still not understanding why you would not just access the gui from the lan interface? Your having to setup a rule to allow it to talk to management from lan anyway? So why not just allow it to the lan IP? Gui going to listen on all IPs anyway.

ColdBrew

@johnpoz LAN/SECURE are the same thing.

I mean, I can just access it through SECURE, the previous admin just set it up with the management IP cause it made more sense I guess.

johnpoz

@sweber said in Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0:

with the management IP cause it made more sense I guess.

Yeah if you were on the management network... But accessing it the way you are from lan is going to be asymmetrical that is for sure.. And you showed that with your sniff..

To be honest I don't know how it would have actually worked correctly.. Where you source natting before? If you source natted the traffic going to management IP from the lan, so it looked like it came from the other pfsense management IP then your return traffic wouldn't be asymmetrical

Or if you host routed on your client to use either pf01 or pf02 lan IP to get to the specific management IP.. then again the return traffic wouldn't be asymmetrical.

If the carp vip was owned by the specific pf you were going through for your gateway where the management IP was located it should work as well. But if you access the other pf managment IP from the lan its going to be asymmetrical return traffic - unless you could set reply-to on the traffic. Which I don't think you can do unless its a wan, this get set on a wan interface, unless you disable it. But I am not sure you could set specific reply-to on just normal interfaces?

The easy solution is just access the lan IP of the pf you want to access the gui of when your on the lan network.

ColdBrew

@johnpoz I'll just start accessing it through the SECURE interface. I'm checking my rules though to double-check and make sure other SECURE clients can't access the web interface - I have a rule in here block any from SECURE to "This Firewall (Self)" after all my other PASS rules for things that do need to talk to the pfSense firewall explicitly (its own IP, versus using the CARP gateway IP).

But it's still letting clients go through PF01 to get to PF02's web interface, even though it has the same rules. Can I just make an alias with PF01 and PF02's SECURE addresses and block all traffic to those specifically (after all my PASS rules, of course)?

And just to double check, if you have DHCP server set up on both pfSense instances, when clients announce on the network that they're looking for a DHCP server, does pfSense respond with its own firewall address (10.30.0.2, for example), or does it respond with the CARP address (10.30.0.1)?

johnpoz

@sweber

https://docs.netgate.com/pfsense/en/latest/solutions/reference/highavailability/clusterconfiguration.html#setup-dhcp

ColdBrew

@sweber Wait nevermind, I just noticed I said clients are going through PF01 to get to PF02's web interface when it shouldn't be, since it doesn't need routing for a same-network connection.

I'm confusing myself, haha!

stephenw10

@johnpoz said in Webconfigurator slow through MANAGEMENT interface after upgrade from 2.5.2 to 2.6.0:

Well have no idea how your blocking on the outbound direction then? Lack of state?

Exactly it passes the TCP:SYN outbound on MGMT on PF01.
Then the SYN:ACK from PF02 goes dircet.
Then the client responds ACK, there must be something to allow that in on PF01 LAN, but it's blocked outbound on MGMT because it never saw the SYN:ACK.

Usually in that sort of setup you end up having to add sloppy-state/all-flags rules on every interface in the route, In and Out.

You might be able to just check the box though:
https://docs.netgate.com/pfsense/en/latest/troubleshooting/asymmetric-routing.html#automatic-fix

Steve