No route to host, until I ping back from Server

[SeRiusRod]

I'm having a strange issue, where after I while I lose connectivity to a server from stations on the same vlan.

The problem happens with a NAS VM hosted on an Esxi server. It had an unused 10Gbs Connectx-3 Mez card, so I created a vswitch and a port group and connected the VM to it.
I have three vlans (lan, sys, iot) and I setup the switch and esxi ports as untagged pvid 2 (lan) for this Machine.

All goes right but after a while (I guess after inactivity) clients on the same vlan lose connectivity to the NAS. Pings reply "No route to host".
pfSense can ping the NAS as all other clients on the two remaining vlans.

If I log into the NAS and ping to one disconnected station, it recovers connection to the NAS, but all other remain disconnected.

I tried to change the vlan mode to tagged, but stays the same.
This is weird, and I don't know where to look.