SMB/NFS/iSCSI between VLAN<->LAN only works with synproxy enabled

MichaelLong

Hello,

I am experiencing a weird issue with VLAN->LAN communication in relation to file sharing protocols (as of now with SMB/CIFS, NFS and iSCSI): In order to create and maintain a successful connection I have to select "State Type: Synproxy" in the advanced settings of the related firewall rule that allows traffic between the two networks.

The following server systems residing on the LAN segment:

Synology NAS, hosting SMB, NFSv4 and one iSCSI-Target
Linux PC, Kernel 5.4.40, hosting SMB.
Windows 10 Professional, hosting SMB

Clients on VLANs:

Linux laptop, kernel 5.6.12, accessing NFS and SMB shares
Another linux laptop accessing the iSCSI-Target on the NAS
several iOS-devices using VLC media player using the internal SMB-client to access shares

Network hardware:

pfSense 2.4.5 (amd64) on a Protectli FW4B system:
1x UniFi USW-8-POE 60W
2x UniFi AP-HD

What I did so far:

When having the firewall pass rule on default values (state type: keep) while trying to access the different kinds of shares, I can see lots of TCP retransmissions in wireshark. In case of iSCSI, the initiator runs into a timeout, mounting a samba-share is quick as normal but listing folder content stalls for minutes but eventually shows the content (otherwise it is not usable), NFS is hanging forever.
I already toggled the hardware offloading settings under System -> Advanced -> Networking -> with no difference.
Also did a reset of the pfSense settings, created only one VLAN and only the necessary firewall rules -> same effect

What I plan to test next:

Get other hardware and restoring the config
Trying out a previous version of pfSense (2.4.4)

Note: The Protectli pfSense system replaced a Unifi USG 3P gateway device that ran pretty much the same network configuration without any issues. Other applications/services, e.g. HTTP/HTTPS, iperf, DNS, SIP or things like gaming in general seem not affected.

Advice appreciated.

Cheers

MichaelLong