Are you experiencing across the across the board packet loss (the internet doesn't work at all), or is it particular to your monitor target address?
If you are not experiencing across the board packet loss, the easiest guess is that the problem is to particular your monitor address. Try using a different address, such as a google public address such as 8.8.8.8 for monitoring and see if the issue goes away. If it does, you can use traceroute to help you choose an appropriate monitor address.