NFS pfctl -d wierdness
Ok, the blurb… take a seat...
NFS client LAN: clients have an allow all TCP/UDP rule to the NFS servers in the NFS server LAN
NFS server LAN: default deny out rule, with exceptions for DNS lookups and SMTP from the NFS servers
Clients mounting shares from the servers fine for several months.
I cant get to the GUI directly for some odd reason, so I ssh and temporarily disable PF (pfctl -d) while I add a new static route. pfctl -e afterwards.
some (not all) NFS clients now cant access the NFS servers. Some get full access, some can only mount half of their shares. pfctl -d on the PFSense box, and it all works again.
I've removed the route and rebooted the firewall, and even rolled back the config to prior to any of this, and still exactly the same. So I'm almost 100% sure the route is not the offending party here.
A complete stop of all NFS activity on the clients, pfctl -e, and restart of NFS seems to cure the situation.
Any ideas what's up? My theory is that stopping pf means it loses track of the connection state regarding NFS, so upon starting is denying out reply packets from the NFS servers to the clients (hence some of them - which happen to be the "quieter" boxes - dont seem to be that affected). this makes sense, although adding an allow all rule for servers back to clients didnt seem to rectify the situation, which I would have expected.
It is working again now, but I'm struggling to get a decent explanation as to what happened (logs dont show much except some of the blocked packets from server -> client)
Any ideas, I'm open to pretty much anything at this stage...