Odd SSH behavior when pfSense is in the mix
I have a home network setup with a server implementing two networks:
Bonded 4X1Gb interface on 192.168.0.0/23 network for serving content
1Gb interface on point-to-point 192.168.3.0/31 network for management
I have a cisco 3750 L3 switch that also participates as a routed interface at 192.168.1.241. This switch is also the routed interface for the mgmt connection to the server using 192.168.3.0.
pfSense acts as the router/default gateway for 192.168.0.0/23 using 192.168.0.1, and has a static route to 192.168.3.0/31 permitted through gateway 192.168.1.241
When I SSH to the mgmt interface (remote network) from my work station (1.240) on 192.168.0.0/23 with pfSense as the default gateway, the session starts, but then drops and eventually the server terminates the session. TCP Dump on the server shows the connection stop, and send new ARP requests for my work station IP, which are replied to by the switch (appropriately).
When I put in a static route in my work station for 3.1 via 1.241 (the switch), SSH works without issue. This leads me to believe the asymmetric network path of WS->PF->SWITCH->SERVER->SWITCH->WS, meaning the removal of PF on the return from the server, is introducing an issue.
I'd prefer not to have a persistent static route in my work station for SSH since I plan on expanding the number of remote networks that the workstation can join. I've attached an image to illustrate the network path examples. Please note that, when directly connected, my work station does not use the WiFi network.
Is there a flag or a state that pfSense is inserting as the gateway that I can tweak? Am I missing some other SSH fundamental in asymmetric routing that isn't exposed when only the Cisco L3 interface is in the mix?
Thank you for any thoughts.