Dropping connections through static route after 20-30 seconds

[bufu]

Hi,

so I am using a physical machine running pfsense Plus 22.05 as my home router (pfsense01), and a pfsense VM on Proxmox running the same OS as the router for my virtual lab network (pfsense02). My home network is on the 192.168.0.0/24 subnet, while the networks on Proxmox are all in the 10.10.0.0/16 range. The pfsense VM has the IP address 192.168.0.10 assigned via a static mapping.

I am trying to set up a static route between these two networks. I added a gateway on the LAN interface on pfsense01 and set it to the IP address of psense02 (192.168.0.10). I then added static routes to each network I need to access via this gateway (e.g. 10.10.30.0/24). I then added firewall rules on both pfsense01 and pfsense02 to allow the traffic.

This setup works, but only for an initial 20-30 seconds during a connection, after which it just cuts out. When I am trying to SSH from my machine (192.168.0.5) to a machine in the VM network (10.10.30.3) , the initial connections gets made, but hangs after a few seconds and I get the error message

client_loop: send disconnect: Connection reset

I checked via firewall logs, states and pftop, but they all show the connection being made correctly, then just dropping. Nothing is getting actively blocked. I then captured the packets, and I can see the initial connection, then some TCP Retransmissions, then an ARP Broadcast message from pfsense01 on who has 192.168.0.5 (my machine). Then the connection gets terminated drops.

I have looked around and found Netgate's documentation on Troubleshooting Asymmetric Routing. I tried the fixes there, but no luck. I also saw the most recent video from Lawrence Systems on https://www.youtube.com/watch?v=acDvlzmsnaE, but that also did not fix the issue. I also tried disabling NAT for the destination network 10.10.30.0/24, but no luck.

Sicne I am running Tailscale and exposing routes with it, I thought maybe that was causing the issue, but after disabling it, the issue persists.

I have also noticed that SSH on target machine shows the connection sometimes coming from my host 192.168.0.5, while other times coming from the gateway interface on pfsense02 10.10.30.1. This one confuses me even more.

I am honestly at a loss here. The setup seems very straight forward to me, especially as when I configure the routes on my host machine it works flawlessly, and using Tailscale VPN and exposing the routes works fine as well. I uploaded the packet capture here if that helps you guys find the issue.

For now I will stick to using Tailscale, but would still really appreciate if you guys could help me solve this issue.