@jknott I know the difference between UDP and TCP. I have started to isolate traffic.
You gave me an idea. I will get a raspberry pi, and will connect it to ISP router, and will record any interruptions, to see if it occurs at the same time of my netgate.
I've being using pfsense for the pass 5 years (VMs and netgate boxs from small to medium sizes like 7100 series) and never encounter any issue like this, but you know, sometimes after a while we start questioning all the parts.