Why does protocol "Any" not match?
-
"Our finance machines are in the DMZ (effectively what pfsense would call the WAN). They have their own Juniper firewall."
Please draw up this network - its smells of asymmetrical to me.. Does pfsense nat to this dmz network? What is the gateway of the machines in the DMZ? Do they have host routes telling them how to get back to networks behind pfsense, or do they just have default gateway to their Juniper Firewall?
Turning of state tracking on your stateful firewall would be the LAST POSSIBLE thing you would ever want to do.. But its not possible to help you find the root of your problem without understanding your traffic flow.
So you have this?
internet –- juniper --- dmz --- pfsense --- machines.
And your machines behind pfsense are RDPing to the machines in the DMZ? Unless you are host routing or natting at pfsense to the dmz this would be asymmetrical. And you could see all kinds of oddness in such a setup. How are these machines accessing the dmz machines? Via their actual dmz IP or FQDN that resolves to the dmz IP.. Or are they using some nat refection and accessing them via IP/FQDN that juniper sends back in?
-
fyi, source and destination is always relative to the interface.
-
fyi, source and destination is always relative to the interface.
What? I figured source and destination were the source and destination that were in the packet headers. Are you saying that's not so?
-
No – it looks more like the attached (crappy) picture.
Wow, really crappy. That picture was right side up in preview, because the camera knows about orientation. Sorry about that.
-
They are but they are evaluated as traffic enters the interface the rule is on.
You are going to have a hell-of-a hard time getting that to work. There should be one network with nothing but routers on it. They will all need routes to what is behind the other routers on that network.
There should be no servers on that network. The servers should be behind a router or they will also need to maintain a proper routing table above and beyond their default gateway.
You will need to defeat all the asymmetric routing you are introducing and battle with route-to/reply-to on pfSense WAN.
In that environment removing the gateway from the WAN interface (it will still exist in the routing table, but without route-to/reply-to involved) would be a good first step.
pfSense would no longer know it is a WAN any longer so you want to enable manual outbound NAT first to create all the NAT that currently exists before removing the gateway from the interface.
-
"Tracking state when you literally trust ALL THE traffic, and don't need to track connection state for NAT is pointless."
Running a firewall at all when you trust ALL the traffic seems pointless as well ;)
What do you think a stateful firewall would do, but track states ;)
So you have pfsense in a carp setup it looks like. But if you don't want it to track states and use it just as a router - then just disable firewall. I don't think it will work as carp with that off though.. Have never looked into such a setup.
But it looks like you have asymmetrical to me if the DMZ is between your pfsense and the core, and then you have servers listed as between the juniper and the core? Am I reading that wrong and your just trying to say what is behind the pfsense and what is behind the juniper?
If your going to have downstream routers from your core, then there should be nothing on these transits to your downstream. Looks like you have dmz and servers sitting on what should be your transit networks.. So unless of these devices are doing host routing to the stuff sitting behind the pfsense and juniper your going to have a asymmetrical mess!
-
"Tracking state when you literally trust ALL THE traffic, and don't need to track connection state for NAT is pointless."
Running a firewall at all when you trust ALL the traffic seems pointless as well ;)
What do you think a stateful firewall would do, but track states ;)
Let's pick this apart first using one example.
When I said ALL the traffic, I specifically meant "ALL the traffic from our finance /28" – suggesting that I ignore state-tracking and intead simply add an "allow" rule in either direction, as a means of eliminating the state mechanism as a potential cause of my dropouts.
As another example: If I know google DNS is my configured resolver, I could bother having pfsense set up a state entry for every single DNS packet I send, which will have a unique source port, and get a buttload of state entries. OR I could add one rule on my DMZ interface that says "allow udp packets from 8.8.8.8 port 53 to my desktop network".
Okay, maybe we don't trust google fully, and we might say "a rogue google DNS server could then probe my network as long as it has source-port 53." That's the tradeoff we use by not using state. But In our case, our own DNS resolvers are off on our server subnet (which we trust fully).
It seems PF isn't very protocol-aware, and doesn't seem to know when to expire UDP state.
-
So you have pfsense in a carp setup it looks like. But if you don't want it to track states and use it just as a router - then just disable firewall. I don't think it will work as carp with that off though.. Have never looked into such a setup.
But it looks like you have asymmetrical to me if the DMZ is between your pfsense and the core, and then you have servers listed as between the juniper and the core? Am I reading that wrong and your just trying to say what is behind the pfsense and what is behind the juniper?
If your going to have downstream routers from your core, then there should be nothing on these transits to your downstream. Looks like you have dmz and servers sitting on what should be your transit networks.. So unless of these devices are doing host routing to the stuff sitting behind the pfsense and juniper your going to have a asymmetrical mess!
No, we have a large service network where we host servers, and then a smaller network with its own vlan to speak to the PFsense machines. The path to the Juniper is across our server network, but it could as easily be via a dedicated transit vlan.
There's only one link between the Juniper and the core. There's only one path out from the finance machines to the Juniper. There's only one path from the Pfsense boxes to the core. (And the secondary pfsense box is completely passive – I would experience the same issues if I powered it off (and I have).
There is no secondary link out from the finance servers to any other part of the network. They are an "island" of service with their own IP space.
Trust me, as much as you want to use the term asymmetric routing, it doesn't apply here. There's precisely one route out of the pfsense box, precisely one route out of the finance subnet, and neither of those devices or the ones in between are rewriting any packets.
Interestingly, I think the answer to my question (why are my "Any" rules not matching) is this:
-
The state table is checked before any rules are checked.
-
There is no way to create a rule that is checked before the state table
-
Changing your rules does not update your state table.
-
Thus: Unless you flush state, You will see no hits on a rule that was otherwise matched by your state table.
I think there's more to it than that, but that hasn't actually been answered.
-
-
What does any of that have to do with your asymmetrical mess?
View the timeouts if you wish - they can be adjusted.. What do you have the firewall set to?
[2.4.0-BETA][root@pfsense.local.lan]/root: pfctl -st
tcp.first 120s
tcp.opening 30s
tcp.established 86400s
tcp.closing 900s
tcp.finwait 45s
tcp.closed 90s
tcp.tsdiff 30s
udp.first 60s
udp.single 30s
udp.multiple 60s
icmp.first 20s
icmp.error 10s
other.first 60s
other.single 30s
other.multiple 60s
frag 30s
interval 10s
adaptive.start 58200 states
adaptive.end 116400 states
src.track 0s
[2.4.0-BETA][root@pfsense.local.lan]/root:For its optimization? Normal, High-Latency, Aggressive, Conservative? 30sec UDP seems about right to me..
Are you having state issues?? If so then you should prob adjust the timeouts if you think they are too long.
"There's only one link between the Juniper and the core. There's only one path out from the finance machines to the Juniper."
What does that have to do with your setup?? Where is your DMZ and Servers?? From your drawing they seem to sit between your Juniper and Core and Pfsense and core.. There should be NOTHING on these networks - it would be a transit network. As I stated before from your drawing (yes really really crappy) I can not tell if your saying the networks behind pfsense are your dmz, and the finance stuff is your server networks? Or if these are what your calling your transit(s) and you have devices on them??
"Our core issue is Microsoft Remote Desktop resets when idle."
From what talking to what? Where is the device that creates the RDP to some other device, and where does that device sit?
-
What does any of that have to do with your asymmetrical mess?
It's not asymmetrical. Period. You keep using that word. I do not think it means what you think it means.
There is only one route between any two systems, and it is through the same path of devices on the same interfaces.
Thanks for your attempt to help, but you're not getting this.
-
I think I see what you're talking about. That our juniper firewall, sits in a network amongst the rest of the unix servers. This could be made into a dedicated transit VLAN (and you're correct, probably should) but the net result would be exactly the same.
The juniper is also a router. The finance subnet (which exists completely behind it) is statically routed to it.
If any machine on the server subnet tries to reach the finance machines, they they may be asymmetrically routed, as their requests will go via the core router, and their responses will come back directly from the juniper, but that works fine.
As far as the route between my desktop and the finance boxes, tho? Show ip route and netstat across the board prove solid.
-
A request please. Could you please remove the edited picture of my network drawing from your thread? Even though there's no IP addressing or anything like that in it, I'd prefer not to have our network diagrams up long-term.
Thanks,
-Dan
-
You can do it. Use the modify button there.
-
I only see the modify button for my own posts. Maybe as a moderator you see it for all of them.
-
Yeah. OK. John reposted it I see now. Done.
-
" Even though there's no IP addressing or anything like that in it"
It was some boxes? That said desktop and finance?? DMZ on it…
"and their responses will come back directly from the juniper, but that works fine."
that is completely beyond the point when your talking about states.. And not to forget the stupid hairpin you now have.. Fix it!! How can anyone run a network like that?? So now you see that its wrong - why would you not fix it?
And you will yes run into problems with states with such setup.. While one side sees the syn to start the connection, it never sees any return traffic - so yes the state will time out.. And now you would have to reopen that state with another syn.
If your talking from desktop to finance - then no that should not be asymmetric from your drawing.. But how is you know your RDP idle time out is do to state expired? More than likely its just your idle session timeout setting on your host your remote desktop to.. From the client side you can also set the keepalive function
Have you sniffed to see if the keepalives are being sent, what interval?? If you want pure router then do that.. I showed you where to disable the firewall aspect and therefore the states a few posts back. But wouldn't it be better to just adjust the idle timeout, and or set your client side boxes to send keep alives?
-
fyi, source and destination is always relative to the interface.
What? I figured source and destination were the source and destination that were in the packet headers. Are you saying that's not so?
Both statements are true, but pfSense only cares about the first packet that creates the state-pair. Once a state is created, the packets for that state will never be evaluated again.
-
What does any of that have to do with your asymmetrical mess?
It's not asymmetrical. Period. You keep using that word. I do not think it means what you think it means.
There is only one route between any two systems, and it is through the same path of devices on the same interfaces.
Thanks for your attempt to help, but you're not getting this.
Am I missing something or was there a private message involved? Without knowing the design, the physical route does not matter, only the logical route. A common source of frustration with asymmetric routing is a Layer 3 switch.
-
He had his drawing of some blocks with dmz, server, desktop, finance labels of where his networks sit removed.. (his tinfoil hat is cutting of blood flow to his brain maybe?) ;)
He has devices that sit on what should be a transit - so yes there is possible asymmetrical routing.. And agrees here
"If any machine on the server subnet tries to reach the finance machines, they they may be asymmetrically routed, as their requests will go via the core router, and their responses will come back directly from the juniper, but that works fine."
There is no "may be" about it but doesn't seem to understand how this can be a problem with stateful firewalls… Maybe its the blood flow issue with that tin foil hat of his ;)
-
Name-calling really doesn't become you.
I know my network better than you guys do. You only know what I told you, and you're making assumptions about what was going on.
No matter how I re-architect the links between our pfsense boxes, and our juniper firewalls, the issue will persist. The routing tables on all boxes confirm this.
My question (which was about a class of rule not logging) quickly devolved into a "your doing it wrong" debate where the original issue was not addressed.