Large amount of data usage
-
Hey guys,
I've been using PFSense for a while now on my home network but recently I've noticed an issue with my ISP reporting large amounts of data usage that I can't account for. They're reporting two different IP's coming from my network and only one of them is assigned to my router.
My setup is just a Cat 6 cable straight from the wall to the router then to a switch going to the rest of my network. The issue is definitely with the router since I have removed it and replaced it with a wireless router and the second IP goes away.
The router is running an out of the box setup since I tried to reinstall the OS to fix the situation but it still has the same error.
Any help would be greatly appreciated.
-
Run packet capture on the WAN interface, to see if you are using that other IP. If you are, check the MAC, to see if it's actually your WAN interface. You have have to watch for ARPs or other broadcasts, if it's another device somewhere. If you have one, you could set up a managed switch with port mirroring and connect another computer, running Wireshark, to see what's actually on the wire between pfSense and the modem.
-
Pfsense out of the box is not going to generate any sort of large amount of traffic on its own. It will check for updates, it will update bogons..
Are you running say snort or pfblocker where rules or lists would be downloaded? Proxy?
How would pfsense have more than 1 IP - does your ISP give you more than 1 IP?
When you say large - put some context around that are you talking 100MB or 10TB?
I would suggest you install package(s) to help you figure out what could be using up your bandwidth. There is the traffic totals package, there is the bandwidthd package. Dartstat, Ntop, Traffic Summary are just a few off the top of my head that could help get a handle on the bandwidth your using.
example: Here is when I upload bunch of home video to backblaze ;)
-
So I just put my router back on the network. I took it down for a little while trying to diagnose this issue an also not running my data usage through the roof.
So I'm going to set it up with some monitoring packages and see what all I can capture with this situation.
I attached the usage chart provided by my ISP. Everywhere where the date is repeated is the second IP reported by my ISP, so apparently I am provided multiple IPs by them… I called tech support when I noticed the issue and the only thing that guy told me was that there was two IPs because my router was in "bridge mode."
To elaborate on the chart, my actual data usage is the smaller bar for each day. That is at least what I could tell from looking at the actual usage on all my devices. There's days where I have random spikes of over 100GB and I cannot account for that on anything that I own.
The part that I don't understand is that whenever I switch over to my wireless router only I only show one IP. Also my wireless router has MAC filtering and no one is able to access it that way. When the PFSense router is connected the LAN connection goes through a Dlink web smart managed switch and then to my devices.
Edit: I also wanted to add that my ISP only refreshed this page once a day so any changes made will take a whole day to confirm.
-
Ok if they give you multiple IP are those IPv4 or IPv6? Doesn't really matter but IPv6 would explain lots of IPs very easy..
So how exactly are you connecting this wireless router.. Are you leaving it connected the same time you using pfsense? If you get IPv6 address, pfsense for sure could be giving all your clients IPv6 and your ISP could be counting those.. But would think you would see more than 2 then..
But if your isp is giving out multiple IPv4 then maybe you have multiple devices on the ISP ipv4 network vs behind NAT (pfsense or your wifi router)..
As to "MAC filtering and no one is able to access it that way"
I sure hope you have actual security setup, WPA2 with a secure PSK?? Mac filtering is a control method and a joke to circumvent if your wifi network is not actually secured..
-
They should be IPv4 but I can disable IPv6 if you think that could be the cause.
Also it is connected at the same time as pfsense but it is only connected to the LAN port on the router with dhcp disabled.
Yes I do have WPA2 with a PSK, no names or room numbers or anything like that, along with the MAC filtering.
-
Mac filtering is pretty pointless.. Its good for say controlling which devices connect to which AP when you have more than one.. Or if you want to like shut off the kids wifi access at bedtime, etc. It is not or has it ever been an actual security feature… Its like putting some duct tape across your door jam after you have set the deadbolt -- that will make it harder for them to get in ;)
What it does do is make it harder to get on your own network... Oh sure billy you want to use my wifi, whats your mac address so I can go add it to the list..
" only connected to the LAN port on the router with dhcp disabled."
How exactly do you have it connected?? Your not going going through your switch right.. So you have connected like the left where switch is after the firewall... Or do you have it like the right side where your modem is connected to your switch and you have vlans setup to isolate traffic?
-
It is setup like the left. My internet comes from the ISP through an ethernet port in the wall then to pfsense then the switch then the wireless router.
-
Well then really the only way you could have 2 IPs being seen by the ISP would be from IPv6.. You can look on pfsense to see what IPs it has..
Prob have a IPv4 address and IPv6, etc.. Out of the box pfsense wold not grab more than 1 IPv4 on its wan interface… Unless you setup VIP?
You can see exactly what IPs pfsense has right on the dashboard.. Do you see IPv6 Addresses?
Another thought on your days you see 2 IPs.. You do understand that since your pfsense mac of its wan interface is different than the mac of your wifi router.. If you say where changing them in an out for testing that your ISP would see 2 IPs for those days... The IPv4 pfsense wan got, and then the IPv4 your wifi router got.
To be honest - that would be my guess to why your seeing 2 IPs on some days.
If you want to swap your pfsense and wifi router in and out of your connection and use the same IP.. You could prob setup mac clone on either pfsense or your wifi router so that you present the same MAC address to your ISP so via dhcp you should always get the same address. I use to do this all the time when running router as vm.. I could swap out different distros and even different copies of pfsense and always have the same public IP because I set all the VMs to use the same MAC on their wan interface.. Also meant I didn't have to reboot my cable modem when changing routers in and out.
-
There is no IPv6 address that shows up in the list.
Also the IP issue isn't caused by the routers being switched because I only switched after weeks of the dual IPs and then after two days pass it shows only the one IP. I only switched to try to narrow down the cause of the issue after the fact.
Also I don't have to worry about rebooting a modem because I don't have one.
Right now since I just connected it I will have to wait another two days to see if this problem still persists. I also have bandwidthd, Status_Traffic_Totals, and darkstat running now to try to see what I see compared to the ISP.
-
Well it could very well be your IP is just changing.. I would write down what your IP is currently on pfsense, and check it now and then to see if it is changing.. It could be that they are just handing you a different IP… Your wan on pfsense is set to dhcp right.
-
So I switched routers back to only pfsense on the 1st and waited until the 3rd to check the usage to be sure that it wasn't just two IPs from both routers being on in one day and it showed two IPs again.
I called customer support and of course they told me my router is in "bridge mode" again but that's how great outsourced tech support is..
They also gave me the two IPs I was pulling and one of them is the one that is assigned to my WAN interface in pfsense and the other one does not show up in the interfaces section. Is there any way to track down where this is coming from since I have that IP now?
-
So they told you your routers in bridge mode - so pfsense has a public IP on its wan..
And it didn't change? You checked on it now and then and made sure it didn't change like on day 2, etc.. This 2nd IP they say your using is also public..
You have nothing else plugged into your modem that is in bridge mode, and it has no WIFI on? Just the 1 wire from modem to pfsense wan?
You do not have pfsense setup in bridge mode do you? You can view all the ips pfsense would have on the diag, routes.. This would show you any vips you might of setup even.. See attached I created a vip 1.2.3.4 just to show as example…
-
Yes the IP has been the same since I plugged it in and it's still the same.
It's fiber so no modem but yes pfsense is the first thing the network touches coming into my room.
I have not setup any bridges or anything in pfsense. Straight out of the box install plus adding the monitoring packages.
I looked in the routes section and the second IP does not show up.
It does show up in bandwidthd as the second highest traffic amount right under the router itself. I attached a picture of its entry.
-
What is the IP - look to its mac in your arp table tell you what device it is.
-
Well, I found my issue. The IPMI interface on my board binds itself to the first ethernet port so the ipmi interface was pulling the second IP and causing that extra traffic on my network…
-
how did you track that down?
-
So once I got the IP from my ISP I found it showing up in the ARP table with the MAC address matching with my board's manufacturer. I decided to just type in the IP, which admittedly should have been something I did before, and it popped up with my IPMI web console. I did some research and figured out that by default the IMPI bonds to the dedicated port as well as the first ethernet port so I went in and disabled it.
Definitely not something I thought about at all honestly.
-
Good catch… For sure - you sure wouldn't want your ipmi open to the public internet..
Such an option should really be disabled in the bios out of the box..
-
You'd think but I guess since it's not really a board meant to be a router they just assume it's only going to be inside the network.
