1000x WAN Traffic increase
-
I'm assuming you're running a 'full' install?
Usually when you run an upgrade packages get reinstalled. It's hard to believe that this is some 'stuck in a loop' problem because it contiunes across an upgrade. :-\You can follow the trail between the pftop table and the sockets list quite clearly though. The problem is I'm not familiar enough with havp to know if this is it's normal behaviour. ::)
Take 184MB transfer, we can see in pftop that the local WAN address downloaded it from 4.27.11.126:80. We also see it was tranfered in both directions between loacalhost services on port 3125 (havp) and port 6053 (?). Then looking down the sockets table we see that at least two of those processes are listed as havp.
I also see that you have at least 4 interfaces. Have you checked the RRD graphs for those other interfaces? Given it's still happening after an update it still points to some internal machine being the culprit here.
Steve
-
Specifically: 23.67.253.161 and 4.27.11.126
From the first IP, looks like it could be an update process?
I don't have those addresses in my Blocklists and I have alot of them.
https://www.virustotal.com/en/ip-address/23.67.253.161/information
2013-06-28 a4.mzstatic.com
2013-09-14 acs.pandasoftware.com
2013-07-10 aru-akam.oracle.com
2013-09-12 au.v4.download.windowsupdate.com
2013-07-02 ax.itunes.apple.com
2013-07-02 cbsbigbrother-lh.akamaihd.net
2013-06-27 cdn.mysql.com
2013-07-09 csd.aeriagames.com
2013-07-09 d.computerbild.de
2013-09-14 de.download.nvidia.comhttps://www.virustotal.com/en/ip-address/4.27.11.126/information/
2013-06-23 a.ligatus.com
2013-06-24 cdn.dli.trymedia.com
2013-06-19 cdn.kaisergames.de
2014-04-30 cdn.ricaud.com
2013-06-18 cdn.royale.spongecell.com
2014-02-20 cdn.static.cyclingnews.com
2014-04-30 cdn.thomascook.com
2013-06-18 cdn2.worldoftanks.com
2012-12-01 conflash.ribob01.net
2013-07-10 dl.wargaming.netEDIT : Guess I should have read the whole thread before posting a repeat!!
-
Worth pointing out again though. ;)
Do Snort or HAVP use a CDN to distribute their updates? I didn't think they did. In which case that's further evidence pointing to it being something behind pfSense.Steve
-
It was HAVP
I turned it off, and IMMEDIATELY the traffic stopped.
This is my whitelist, I wonder if that has anything to do with it.logitech.com/
navisite.net/
.lenovo.com/
.omniti.com/
clamav.net/
sourceforge.net/
70.38.0.134
188.121.46.128
alternate.mtrosemedia.org/*Also, is it possible that I'm trying to cache all of the virus DB? Not really sure about what I'm talking about, but I don't know why it is still downloading definitions from that URL.
I removed the following from the whitelist, and the problem is gone. Any explanation?
logitech.com/
navisite.net/
.lenovo.com/
.omniti.com/
clamav.net/
sourceforge.net/ -
Worth pointing out again though. ;)
Do Snort or HAVP use a CDN to distribute their updates? I didn't think they did. In which case that's further evidence pointing to it being something behind pfSense.Steve
Yes, that's my current hypothesis too. I suspect that a LAN client is making a request that squid is trying to cache - repeatedly. So squid (or possibly HAVP) just keeps looping the request over and over but failing to complete the process. If the bandwidth disappears after temporarily deactivating the service that would at least give the OP a place to start looking.
-
Sorry, that did not fix the issue. Turning off HAVP fixes the issue, but removing those lines from whitelist did not solve anything, and I didn't think that it would.
-
but removing those lines from whitelist did not solve anything, and I didn't think that it would.
I'm not sure what you mean by removing lines from a whitelist, but it's probably not relevant at this point.Oops.Sorry, that did not fix the issue. Turning off HAVP fixes the issue,
Ok, excellent. Now you know the source of the trouble, it's HAVP. Have you had a look at the HAVP logs to see if it's reporting errors? If not, we could probably increase it's debug level.
-
How do I view the logs for HAVP?
Going to status >> Package Logs show nothing.
System logs does not have anything specific to HAVP -
Also, I added a rule in my firewall, and it doesn't seem to work.
I added the ip
23.67.253.163, .167, .168 to the "Spammer_Hacker" alias
23.57.253.0/24 to the "Spammer_Network" alias
But it doesn't seem to block the traffic.
Attached are my rulesets and my current traffic graph
-
How do I view the logs for HAVP?
Going to status >> Package Logs show nothing.
System logs does not have anything specific to HAVPFrom the command prompt/console:
clog /var/log/havp/havp.log
-
I'm assuming the "(Bad address)" is when I disabled it.
05/06/2014 11:06:05 === Starting HAVP Version: 0.91 05/06/2014 11:06:05 === Mandatory locking disabled! KEEPBACK settings not used! 05/06/2014 11:06:05 Running as user: havp, group: havp 05/06/2014 11:06:05 --- Initializing Clamd Socket Scanner 05/06/2014 11:06:05 Clamd Socket Scanner passed EICAR virus test (Eicar-Test-Signature) 05/06/2014 11:06:05 --- All scanners initialized 05/06/2014 11:06:05 Process ID: 52553 05/06/2014 11:12:55 === Starting HAVP Version: 0.91 05/06/2014 11:12:55 === Mandatory locking disabled! KEEPBACK settings not used! 05/06/2014 11:12:55 Running as user: havp, group: havp 05/06/2014 11:12:55 --- Initializing Clamd Socket Scanner 05/06/2014 11:12:55 Clamd Socket Scanner passed EICAR virus test (Eicar-Test-Signature) 05/06/2014 11:12:55 --- All scanners initialized 05/06/2014 11:12:55 Process ID: 35010 05/06/2014 11:59:20 === Starting HAVP Version: 0.91 05/06/2014 11:59:20 === Mandatory locking disabled! KEEPBACK settings not used! 05/06/2014 11:59:20 Running as user: havp, group: havp 05/06/2014 11:59:20 --- Initializing Clamd Socket Scanner 05/06/2014 11:59:20 Clamd Socket Scanner passed EICAR virus test (Eicar-Test-Signature) 05/06/2014 11:59:20 --- All scanners initialized 05/06/2014 11:59:20 Process ID: 3913 clog: ERROR: could not write output (Bad address)
-
Also, I added a rule in my firewall, and it doesn't seem to work.
I added the ip
23.67.253.163, .167, .168 to the "Spammer_Hacker" alias
23.57.253.0/24 to the "Spammer_Network" alias
But it doesn't seem to block the traffic.
Attached are my rulesets and my current traffic graphIt's probably still in the state table. Try: Menu; Diagnostics; Show States; Reset States; "Reset"
-
[quote] I'm assuming the "(Bad address)" is when I disabled it. clog: ERROR: could not write output (Bad address) [/quote] My bad, I should have said "cat /var/log/havp/havp.log" Ok, that log seems reasonable enough. Maybe it's clamav, try: cat /var/log/clamav/clamav.log
-
Resetting the states seems to have done it.
However, it appears that HAVP really isn't doing too much of anything
With my workflow being:
Internet >> Snort >> pfBlocker >> Squidguard >> Squid >> Client
I'm thinking it may be best to uninstall HAVP. There seems to be a lot of issues with it from people on the forums.
I don't believe there are any alternatives -
clamav.log is empty.
Does it need to be running in order for it to generate logs, or are the old logs saved ? -
Resetting the states seems to have done it.
Bear in mind that blocking that IP address was only as a temporary diagnostic and will likely prevent HAVP from functioning correctly once, er, it is, er, functioning correctly…
clamav.log is empty.
Does it need to be running in order for it to generate logs, or are the old logs saved ?Yes, it needs to be running. And to properly diagnose it's error the temporary block(s) should be removed. There may also be additional logs in each directory:
ls /var/log/havp
and
ls /var/log/clamav
-
Hmm, curiouser and curiouser. ;)
Putting a firewall rules on the WAN interface will not block any traffic that is initiated by HAVP. Firewall rules on WAN only block new incoming connections. If you want to block new outgoing connection, like this, you need to use a floating rule.
Just to be perfectly clear you didn't respond to my question about other interfaces you have. How many interfaces do you have? Have you check the RRD graphs for those interfaces to make sure it's traffic from there?
Steve
-
Sorry. I checked the RRD Graph for those interfaces, and none of them were causing the Traffic
You can disregard the WAN2DHCP interface. I was trying to create a Gateway Group for all of my traffic. I have Comcast with a static IP, but apparently the DHCP IP still works as well. Which gives me (2) outbound connections. I was trying to set both as a shared outbound connection for all traffic, giving priority for WANGW, but the deployment did not work out very well. I don't have a good grasp on the workflow.
-
Putting a firewall rules on the WAN interface will not block any traffic that is initiated by HAVP. Firewall rules on WAN only block new incoming connections.
You're not saying that pfSense would allow a two-way connection to be established despite the WAN entry blocking traffic from that IP? That seems counter-intuitive to me. I would have expected the firewall to permit the outbound packets to be sent to the blocked IP but then to block any response coming from the blocked IP. i.e. one-way traffic only. I'd better hit the man pages again…
-
Putting a firewall rules on the WAN interface will not block any traffic that is initiated by HAVP. Firewall rules on WAN only block new incoming connections.
You're not saying that pfSense would allow a two-way connection to be established despite the WAN entry blocking traffic from that IP? That seems counter-intuitive to me. I would have expected the firewall to permit the outbound packets to be sent to the blocked IP but then to block any response coming from the blocked IP. i.e. one-way traffic only. I'd better hit the man pages again…
That's not how stateful tracking works. Pass decisions are made when the first "new" packet is seen. In TCP connections is the initial SYN packet and in UDP or other IP protocols it's the first packet that does not match any existing state. Block rules apply to any packets that are seen but they won't match packets that match an existing state. PfSense allows all outbound connections (as seen from the point of the interface) by default unless you restrict them with floating rules.