Some ICMP redirect issues. Fixable or not?
-
Hi.
Unfortunately, I have to state this first, as otherwise the thread will most likely quickly detoriate and becoem off topic: Please refrain from telling me how ICMP redirects are evil and should not be relied on. That's just plain not the topic here. Let's just accept for this thread that ICMP redirects are part of the TCP/IP protocol suite, and as such ought to work properly. You may have a different opinion, and are fully entitled to it. But please stay on topic here.
So, here are two rather severe issues I see in pfSense regarding ICMP redirects:
1. In a carp setup, ICMP redirects are always sent from the primary IP instead of the carp VIP. Most, if not all, receiving OS's will ignore such redirects leading to enormous unnecessary load and often performance issues. That has been posted before here, without a reply.
2. ICMP redirects (at least here) seemingly are only sent when the communication is initiated from the network in which the alternate router lives, aka when pfSense sees the initial connection setup. If the communication is started from the other end, pfSense doesn't send an ICMP redirect, and also doesn't re-route the traffic, resulting in broken communication.
Let me give an example. Let's say there's LAN1, 192.168.1.X/24, with pfSense being the default gateway at 192.168.1.1.
There's also another LAN2, 172.21.1.X/24, which is connected to LAN1 through a different gateway than pfSense. Let's say that gateway has 192.168.1.254 on the LAN1 side, and 172.21.1.1 on the LAN2 side. In LAN2, 172.21.1.1 is the default gateway, in LAN1 192.168.1.1 (pfSense) is the default gateway.
Reminder: Please, refrain from posting to change the setup. Above is perfectly legal and needs to work. I'm fully aware of potential security implications, thank you.
Now, in above setup, if communication between the two LANs is inititated from LAN1, it works. It goes to pfSense first, receives an ICMP redirect to use 192.168.1.254 as the next hop gateway instead, goes there, and works.
BUT: If the communication initiates from LAN2, it comes in through the other gateway of course, then the local reply (usually a TCP SYN/ACK as part of the three way handshake) goes to pfSense. In that case pfSense does not send an ICMP redirect to that packet, as such the whole communication fails. I can only assume that's because pfSense didn't see the initial SYN?!
Especially the second one is nasty. Any ideas if, and if yes, how this might be fixable?
Thank You.
-
BUT: If the communication initiates from LAN2, it comes in through the other gateway of course, then the local reply (usually a TCP SYN/ACK as part of the three way handshake) goes to pfSense. In that case pfSense does not send an ICMP redirect to that packet, as such the whole communication fails. I can only assume that's because pfSense didn't see the initial SYN?!
Yes, that is a bit tricky. I imagine pfSense (pf) is going to drop the packet because there is no already-established state and it is a reply-type packet, not a state-establishing packet. And if the processing of ICMP-redirect comes after the packet has got through pf, then obviously it will not reach the point in the protocol stack where the routing layer puts out ICMP-redirect.
And yes, I agree with your analysis. If this all worked then it would be possible to turn on "acceptance of ICMP-redirects" in clients (ones that ignore ICMP-redirect by default), and then LAN with multiple gateways and potential asymmetric routing scenarios would "just work".
-
Thanks for your reply.
The core question for me here is, is there anything that can be done in the configuration or firewall rules to make the second situation work?
Thanks.
-
So did you try a rule with sloppy state?
-
"There's also another LAN2, 172.21.1.X/24, which is connected to LAN1 through a different gateway than pfSense."
While I completely understand this scenario, normally you would NEVER see it, because normally you wouldn't ever set it up that way. You would create a transit connection to this downstream router from pfsense so you never have asymmetric routing and you also control access from this downtream network to your lan network. With a downstream router pfsense has no control over traffic initiated from that network in to your lan network.
Or you don't even use that downstream router and just connect that segment direct to pfsense.
While I agree with you that icmp redirects are part of the suite - I don't think they are intended for such a design flaw from the get go ;)
I see that redirects can be useful - but I would never use them in this case, such a case should never be setup. Another option is if you want host on lan to use that downstream router to get to some downstream network, then just create a route on it telling it so, this is not the best solution - but also another way to solve asymmetric routing problems.