Periods with sudden drop in DL speed

[DocDAT]

Hi all,

I've just gotten a FTTH connection of 1000/100 and built a pfsense box (Shuttle DH170 + I5 7600t and 4 GB RAM) that's connected directly to the mediaconverter and without the ISP supplied router. Supposedly the only thing necessary to get this working is to change VLAN tagging to 101 on the WAN interface. Other than that, it's installed with everything as default - no VPN, loadbalancing etc.

So I set this up and everything seems to be working fine until I started to download large files. Typically after about 3 minutes the speed suddenly drops from around 900 mbit/s to 30-50 mbit/s and stays like this for about an hour, unless I reboot pfsense. The interface never changes from 1000 full duplex and there are no collisions or other errors as far as I can see. CPU never exceeds about 7%.
Interestingly, when I try the ISP supplied router, this slowdown doesn't happen although it's noticeably slower in general.

I'm a n00b when it comes to networking, so I hoped someone could help me diagnose whether the problem is caused by an issue with the settings in pfsense or if it's some incompatibility with my ISP.

Thanks!