@stephenw10 Yeah that's what I'm thinking, maybe the ONT itself can't handle it or something along those lines.
I know many ISPs do throttle torrents, but you'd usually see that as the torrent traffic itself having higher latency and stuff, not just dropped packets on the entire connection, though it doesn't appear the later is unheard of.
Pretty confident at this point it isn't pfSense, so at least that's good. May also see if my ISP can get a tech out, after I test both VPNs and possibly direct fiber connectivity instead of the ONT.