100% of bandwidth in use for no reason? Bug? Virus? Malicious?
-
The ISP I'm with has datacaps, and our family is on a 200GB cap. Last month we ended up capped 2 weeks into the month, which was quite strange, but I assumed it was just my sister torrenting.
We just got uncapped a few days ago on sunday, and our bandwidth has been quite poor, which I again assumed was my sister torrenting. I was constantly checking traffic graphics, and the bandwidth in was alway high, but when I looked at the WLAN interface I couldn't pin it on her.
Today she was out, so I shut off her PC, and the traffic was still at max.
I checked the RRD graph and saw 100% of our bandwidth has been in use every day since we've been uncapped, and the usage just before we were capped last month was quite high.
I checked packet capture, and all I could find was that our pfsense box was getting packets constantly from the IP of our service provider, but these packets were not going to any PCs on our network.
I'm really worried we're about to be capped again after only a few days, and really want to know what is causing this and how to stop it. What could it be?
-
Do you have ntp exposed to the WAN? Perhaps you've been sucked into the ongoing ntp amplification DDOS attacks that are currently making the news? Though that would be mostly outgoing traffic.
Look at the state tables, what sort of traffic is it?
Steve
-
I noticed that on the 3rd graph the numbers at the bottom were showing in-block and out-block with similar figures to in=pass and out-pass. I didn't believe that, so looked at the code and found bugs there. These code changes should fix the display calculation errors: https://github.com/pfsense/pfsense/pull/939
But you are getting lots of in-pass - so whatever it is the firewall is passing it in. If it is a DDOS attack then it would need to be to ports that are open on WAN, since the traffic ends up being counted as passed.
Does traffic also show on LAN? If you remove all devices from LAN for a while, is the traffic still hitting WAN? -
Do you have ntp exposed to the WAN? Perhaps you've been sucked into the ongoing ntp amplification DDOS attacks that are currently making the news? Though that would be mostly outgoing traffic.
Look at the state tables, what sort of traffic is it?
Steve
I'll post some screenshots from the tables and packet capture as i've never used states before so I'm not sure I can interpret it.
Last night the traffic seemed to be coming from my ISP, but today it was coming from 172.18.112.143, which I cant trace to anywhere. Blocking it in the firewall stopped it, but I'm worried it will just happen again. I need to sort it out, because my ISP has a ridiculously slow capped speed and does not offer data packs.
EDIT: It's happening again, now coming from 61.9.129.152 which is apparently deploy.akamaitechnologies. I believe this is where it was coming from last night, not my ISP. Blocking it seems to do nothing, unless i'm using the rules incorrectly.
EDIT 2: Blocking it worked, it just switched to a different IP on the 61.9.192.0 network. So I blocked the entire network, but it switched to 61.9.129.184. So maybe that network is subnetted? This is ridiculous.Does traffic also show on LAN? If you remove all devices from LAN for a while, is the traffic still hitting WAN?
No traffic shows on the LAN or WLAN, I have turned off all PC's except my Laptop to check and it was still happening until I blocked that IP. Are you also saying that I likely have some open ports I should have closed, and this is leaving me vulnerable?
-
124.176.225.110 is Telstra in Melbourne, Australia - http://ipaddress.is/124.176.225.1#.UwQ_VFO5F8w - I guess that is you.
207.46.7.252 is a Microsoft address - http://ipaddress.is/207.46.7.252#.UwRAY1O5F8w - so is something constantly trying to download some Windows thing over and over, or?
172.16.0.0 to 172.31.255.255 (172.16.0.0/12) is private address space. Assuming that is a capture on WAN, then where does that come from? Why would something be pinging 172.18.112.143 from the WAN and getting a response. That must be something going on internal to the Telstra network??? -
124.176.225.110 is Telstra in Melbourne, Australia - http://ipaddress.is/124.176.225.1#.UwQ_VFO5F8w - I guess that is you.
207.46.7.252 is a Microsoft address - http://ipaddress.is/207.46.7.252#.UwRAY1O5F8w - so is something constantly trying to download some Windows thing over and over, or?
172.16.0.0 to 172.31.255.255 (172.16.0.0/12) is private address space. Assuming that is a capture on WAN, then where does that come from? Why would something be pinging 172.18.112.143 from the WAN and getting a response. That must be something going on internal to the Telstra network???It's all really confusing me now, just before it was coming from the 61.9.129.0 network. I blocked the IPs one by one, and the network as well. Each time I blocked it, it switched to another IP. After a while it switched to another 61.X.X.X IP, so I've just shut off the pfsense box, and have no idea what to do.
I got an email from the ISP saying 160gb of our 200gb has been used, which I'm assuming I can't do anything about since I haven't been using their router. I'm going to switch back to the ISP router today, and if I end up capped tomorrow I have no idea what I'll be able to do.
All in all, extremely frustrating.
-
I thought that 61.9.x.y subnet looked familiar!
BigPond (Telstra ISP) has DNS servers and mail servers in that space. At 6Mb/s max I guess you are on BigPond ADSL.
Just because you aren't using their router doesn't mean you can't call to let them know about all the traffic being directed to your IP. Don't get hit with being slowed or charged an arm and a leg for data overrun if it isn't your fault.
Are you using dyndns or something similar that picks up your IP address changes?
-
I thought that 61.9.x.y subnet looked familiar!
BigPond (Telstra ISP) has DNS servers and mail servers in that space. At 6Mb/s max I guess you are on BigPond ADSL.
Just because you aren't using their router doesn't mean you can't call to let them know about all the traffic being directed to your IP. Don't get hit with being slowed or charged an arm and a leg for data overrun if it isn't your fault.
Are you using dyndns or something similar that picks up your IP address changes?
I'm not using dyndns or anything similar.
I'll call Telstra tomorrow, maybe since it's their addresses they'd feel responsible enough to reset my data cap.
In the mean time I'm attempting to block the 61.9.129.x network from my WAN port, but only my single host rules seem to work(or at least it seems to start coming from a different ip when i attempt to block). Could anyone give me an example that would block all of 61.9.129.x or even all of 61.9.x.x?
Update:
I installed the pfBlocker package
Then set incoming to block
Then made a .txt file containing "telstra:61.9.129.0-61.9.129.255"
I put the .txt file in a .gz archive and uploaded it to http://www.uploadhosting.co/ to get a direct link
I then made a list with the url of the direct link.this seems to have done nothing though, as packet capture still shows my router getting packets from 61.9.129.209
-
Make a firewall rule on WAN - Block protocol all source 61.9.129.0/24 or all of 61.9.0.0/16, destination any.
The trouble with these things is that the data has already come across your ISP link and been counted in your quota by the time you block it. So yes, as you say, you will need to contact Telstra and try to find out what is going on and why, and get your quota reset. -
What firewall rules do you have on WAN? Any unsolicited traffic on WAN should be blocked by default so why is it showing as 'in-pass'? The obvious answer the that is that it's not unsolicited, something on your LAN is asking for it. Why then is that traffic not showing up on an internal interface?
If you didn't know Akamai are a content distribution network used by many companies including Microsoft to deliver updates service packs etc.
I would guess that one or more machines on your internal network has got stuck in Windows update loop, downloading the updates, failing to apply them and then trying again.
If you go to Diagnostics: States: rather than state summary, and then filter by the offending external IP you should see the NAT state showing your internal machine requesting the traffic.
Steve
-
What firewall rules do you have on WAN? Any unsolicited traffic on WAN should be blocked by default so why is it showing as 'in-pass'? The obvious answer the that is that it's not unsolicited, something on your LAN is asking for it. Why then is that traffic not showing up on an internal interface?
If you didn't know Akamai are a content distribution network used by many companies including Microsoft to deliver updates service packs etc.
I would guess that one or more machines on your internal network has got stuck in Windows update loop, downloading the updates, failing to apply them and then trying again.
If you go to Diagnostics: States: rather than state summary, and then filter by the offending external IP you should see the NAT state showing your internal machine requesting the traffic.
Steve
I originally assumed it was a PC, tablet, etc on the LAN causing this like you said, but nothing ever appeared to be sending or receiving anything. It's stopped as of now so its a bit harder to check on, I am not sure what actually stopped it though, and whether or not I had any part.
Using States and filtering by a few of the IP addresses only showed traffic between my public IP and the offending IPs. I'm not sure if this is because it's been a few hours since it stopped, I assume it gets cleared out a bit?
Below is the Traffic graph for the last week on the WLAN(i dont have anything on the wired LAN).
