@TheNarc definitely not what I'm seeing in my setup. Probably something related to your VPN connection. My latency isn't affected at all.
Did you try TCP Tunnel? I'm having latency issues and changes in speed as well when combining torrent with UDP tunnel.
I wouldn't disable gateway monitoring, ping should just work and is a usefull quality indicator.