I can ping both directions but only access servers one way...?

[Gblenn]

@viragomann Thanks for the tip, turned it on set to default but no change. What do you mean playing with packet size in ping? What will that give me?
If I try ping -s 1400 I get Frag needed and DF set (mtu = 1400)
If I do -s 1500 I just get the ping times, no additional info.

[viragomann]

@Gblenn
For testing you can try a low value like 1100.If this doesn't work, there might be another reason for the problem.

[Gblenn]

@viragomann I tried various settings but no difference that I can see...

I did some PCAP when trying to access Site C from Site A and this is what it looks like in pfsense.
The problem is the same regardless of which server I'm trying to acces, be it pfsense itself or in this case a switch. I just chose it for simplicity and cleanliness of the pcap.

🔒 Log in to view

And the downloaded file from within Wireshark

🔒 Log in to view

If i google pfsense blocking SYN on port 80/443 I find an old redmine issue that clearly has been resolved since way back? So it must be something different, or? Why do I not see this in the 2.7.2 setup?

I should add that in the firewall log I see a PASS on the IPSec interface of 10.10.2.3:PORT to 10.10.177.5:80
🔒 Log in to view

[viragomann]

@Gblenn
Are the routes set properly at C?

[Gblenn]

@viragomann Yes, site C being the pfsense side, the only thing to do there is to set the static route 172.168.1.0/24 to the IPSecVTI Gateway that was automagically created.

🔒 Log in to view

On Sophos it looks pretty similar and it's been quite obvious when setting this up a few times that whenever I forgot this part, I don't get near where I am now...

[Gblenn]

I have been trying to go through the logs in Sophos as well and everything I can find on that side shows green as in all packets are passed by the correct fw and nat rules for this.

[Gblenn]

When trying to log in via SSH on a Linux machine on Site C , I get the following...
Simlar to before but it looks like there is a response initally from 192.168.1.31?

🔒 Log in to view

[viragomann]

@Gblenn
So even if the destination machine was sending an ACK, the SYN packet is re-transmitted.

I'd suspect, that the ACK packet does not arrive at the machine, which initiated the connection. But why?

BTW: I noticed, that the LAN at A is a public network range.
However, this shouldn't matter if the routes are added properly.

[Gblenn]

@viragomann said in I can ping both directions but only access servers one way...?:

@Gblenn
So even if the destination machine was sending an ACK, the SYN packet is re-transmitted.

It appears that way doesn't it... however having run the same test again several times I don't see the ACK coming back. So perhaps it was an anomaly?

I'd suspect, that the ACK packet does not arrive at the machine, which initiated the connection. But why?

BTW: I noticed, that the LAN at A is a public network range.
However, this shouldn't matter if the routes are added properly.

Ah you are right, it is in the public IP range... I made the simplest possible change from 192 to 172 and I should have changed to 16 as well. But I don't suppose pfsense would care? And it's working towards site B anyway...

I started thinking there was some sync issue since site A and C are connected via a switch. Site A is for testing purposes at the moment. But latency is below 1 ms when pinging between them. So I added a limiter in order to add a bit of latency to get it closer to what I have between the other sites, but that doesn't seem to change things.

[Gblenn]

I have sort of given up on getting this to work, for now at least. Ended up deploying Wireguard on a VM on the Sophos LAN and have a connection going that way instead...

[Gblenn]

@viragomann
So this is definitely a 24.03 thing since a change to 2.7.2 solved the issue immediately... No change in the configuration on either side, just a switch to CE.
And I guess I have to see if the patches I just found out about for IPsec VTI will fix the issue.

[Gblenn]

And now it's working also on 24.03 after implementing patches for Redmine #15449, #15430 and #15606...

[viragomann]

@Gblenn
Great info. Thanks.
I never had this issue, even running multiple IPSec site-to-site connections to other routers than pfSense and didn't apply IPSec patches.

[Gblenn]

@viragomann Well, first time I have used IPSec since Sophos doesn't do WG...

One thing to note though, this is Routed VTI connection, which is different from "Tunnel IPv4" that you have as default Mode in pfsense...

[viragomann]

@Gblenn
Ah ya, you mentioned above.
I never set up a VTI to be honest, I just have some small knowledge about from the docs.

[Gblenn]

@viragomann Well, since I couldn't get the default one to work, I tried VTI and it worked. Not sure what I did wrong with the other method but I did find VTI a bit more like the WG tunnels I have set up in the past. With the gateway and routing settings at least...