Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Bounty $200: Monitor bandwidth use on IP adresses. NOW $250

    Expired/Withdrawn Bounties
    25
    151
    79988
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      backbone
      last edited by

      I want to know how many GB of traffic every Internal IP adresse is using true my Firewall.
      Also runing many VLANs so it have to support to get info on many VLANs as well.
      Or I want to define the ip adresse I want to get info on. something like add ip adresses I want to monitor.
      have to support many hosts at one time.
      And just get the info for total bandwidth inn/out last month. And total so far this month.
      And maybe an total show of bandwidth over the firewall as well, for last month and so far this month.

      maybe based on darkstat or something?

      1 Reply Last reply Reply Quote 0
      • O
        ollopa
        last edited by

        I'm pretty sure ntop can do this detailed level of reporting, and there's already an ntop package for PFsense…  http://www.pfsense.com/packages/All/

        ??

        1 Reply Last reply Reply Quote 0
        • B
          backbone
          last edited by

          @ollopa:

          I'm pretty sure ntop can do this detailed level of reporting, and there's already an ntop package for PFsense…  http://www.pfsense.com/packages/All/

          ??

          I can`t see how I can use Ntop on just the IP adresses I want.. and also how can I get It to show last month and so fare this month ??  ???

          Maybe ntop could be writen to do this?

          1 Reply Last reply Reply Quote 0
          • C
            cmb
            last edited by

            ntop can do this, but it doesn't run exceptionally well on FreeBSD because of threading bugs in ntop that the author doesn't care to fix.

            Plus, if you're running VLAN's, ntop requires putting your interfaces in promiscuous mode.  There's a bug in FreeBSD that will cause all your VLAN's to drop if you put a VLAN interface in promiscuous mode.

            This capability is really there already, you just need to collect the appropriate data and then report on it as you desire.  What you need to look at is the pfflowd package, and you need to get a NetFlow collector running on something.  For a quick solution, check out CactiEZ, it comes preconfigured with an awesome Cacti install plus a NetFlow collector enabled out of the box. 
            http://cactiusers.org/wiki/CactiEZ

            After you have the NetFlow stats collected, there are a bunch of reporting tools that will show you your NetFlow data, including ntop, which runs on CactiEZ (and works fine since it's Linux-based).

            Nice list of NetFlow related tools here:
            http://www.switch.ch/tf-tant/floma/software.html

            Personally, for my network monitoring at home, I run a CactiEZ install in a VM running on VMware Server.  Works great.

            Is this answer worth $200?  ;D

            1 Reply Last reply Reply Quote 0
            • S
              sullrich
              last edited by

              This looks like a possible solution.

              http://bandwidthd.sourceforge.net/

              Look okay?  I may take this one up.

              1 Reply Last reply Reply Quote 0
              • B
                backbone
                last edited by

                @sullrich:

                This looks like a possible solution.

                http://bandwidthd.sourceforge.net/

                Look okay?  I may take this one up.

                Thats more like it :)
                I just testet the demo….
                but I can`t see how I can get how many GB of traffic one Ip have use there? This mnd and last?
                I see that it can write to a database, maybe just get the info out from there again?
                Can the data be writen to an external mysql db on another server?

                If you could do this in one package for pfsense it would be nice.

                This is something I would pay to get done ;)

                1 Reply Last reply Reply Quote 0
                • C
                  cmb
                  last edited by

                  @sullrich:

                  This looks like a possible solution.

                  http://bandwidthd.sourceforge.net/

                  Except it'll put your interfaces in promiscuous mode, and since he's running VLAN's, it'll kill all his network connectivity because of the VLAN's + promisc FreeBSD bug.  Won't work.

                  The only thing I know of that doesn't use promisc is NetFlow.

                  backbone:  You need to look at what I suggested, as that's the only thing that's going to work for you in the foreseeable future.

                  1 Reply Last reply Reply Quote 0
                  • O
                    omegadraconis
                    last edited by

                    I ran across this: http://www.freebsd.org/cgi/query-pr.cgi?pr=72933
                    it's a patch to fix the problem for bge module. At the bottom of the page it linked to http://cvsup.pt.freebsd.org/cgi-bin/cvsweb/cvsweb.cgi/src/sys/dev/bge/if_bge.c
                    "Revision 1.148 / (download) - annotate - [select for diffs] , Mon Sep 18 22:18:21 2006 UTC (3 weeks, 3 days ago) by jkim
                    Branch: MAIN
                    Changes since 1.147: +28 -15 lines
                    Diff to previous 1.147

                    Do not strip VLAN tag in promiscuous mode."

                    It would seem to depend on the nic's your using, your going to have to figure out which kernel module your nic uses(http://www.freebsd.org/releases/6.1R/hardware-i386.html#ETHERNET) and see if it has been patched or is a problem.

                    1 Reply Last reply Reply Quote 0
                    • C
                      cmb
                      last edited by

                      Thanks for the pointer to that, omegadraconis.  Looks like drivers are getting fixed one by one.  We'll have to look at this again once 6.2 is out and we're using it.

                      1 Reply Last reply Reply Quote 0
                      • M
                        Mercredi
                        last edited by

                        i will pay another $50 for a package, that will help me to know how many GB of traffic every Internal IP address is using through my pfsense firewall with possibility to view statistics for a day, week, month, and also configurable local address table to exclude from calculation. i need this for my small office network and home use, there is not so much users and data to deploy NTOP and special accounting servers.

                        1 Reply Last reply Reply Quote 0
                        • S
                          sullrich
                          last edited by

                          Ok, if everyone wants to verify that their nics will work with the package I mentioned earlier, I'll go ahead and get started on bandwidthd.

                          1 Reply Last reply Reply Quote 0
                          • B
                            backbone
                            last edited by

                            @sullrich:

                            Ok, if everyone wants to verify that their nics will work with the package I mentioned earlier, I'll go ahead and get started on bandwidthd.

                            I have 2 onboard Broadcom BCM5721 cards, using the bge(4) driver?

                            1 Reply Last reply Reply Quote 0
                            • M
                              Mercredi
                              last edited by

                              @sullrich:

                              Ok, if everyone wants to verify that their nics will work with the package I mentioned earlier, I'll go ahead and get started on bandwidthd.

                              i am using intel pro100 management adapters, their name in system are fxp… thank you, sullrich.

                              1 Reply Last reply Reply Quote 0
                              • S
                                sullrich
                                last edited by

                                So we are a go?

                                1 Reply Last reply Reply Quote 0
                                • M
                                  Mercredi
                                  last edited by

                                  sullrich: what do i need to do? i allready need statistics on traffic of each local user :)

                                  1 Reply Last reply Reply Quote 0
                                  • B
                                    backbone
                                    last edited by

                                    @sullrich:

                                    So we are a go?

                                    If the nics are supported so yes :)

                                    but can this system get info for 1.jan to 2.feb ? I see that it has info for an Interval, but I need to know how many GB traffic it use last mnd and so on…. Maybe the sensor system has what i need?  It is the sensor system u want to make then? as a pfsense package?

                                    1 Reply Last reply Reply Quote 0
                                    • B
                                      backbone
                                      last edited by

                                      @sullrich:

                                      So we are a go?

                                      So what do u say? :)
                                      Something that could be done? :)

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        sullrich
                                        last edited by

                                        Yeah, I'll start on it in the next day or so.  However, I am not sure your nics work or not, honestly.  Thats why i was asking you all ..

                                        1 Reply Last reply Reply Quote 0
                                        • N
                                          narf
                                          last edited by

                                          Is there a list of drivers that have been patched to work with promisc + VLAN, or is the bge module the only one that has been fixed?

                                          I'm just curious as i should be doing some hardware shopping, and its alot easier to just buy hardware that works, instead of waiting for drivers to be fixed.

                                          //Richard.

                                          1 Reply Last reply Reply Quote 0
                                          • S
                                            sullrich
                                            last edited by

                                            I would say it is the opposite and that most nics should work fine.

                                            1 Reply Last reply Reply Quote 0
                                            • M
                                              Mercredi
                                              last edited by

                                              and any updated information?

                                              1 Reply Last reply Reply Quote 0
                                              • S
                                                sullrich
                                                last edited by

                                                Doubtful.  This thread went from promising to dead in the matter of a few days.  What was almost 2K is now 250$ .. If I was Bill (which I am not) then I would be less than interested at this point for the amount of work that this entails.

                                                1 Reply Last reply Reply Quote 0
                                                • B
                                                  billm
                                                  last edited by

                                                  @sullrich:

                                                  Doubtful.  This thread went from promising to dead in the matter of a few days.  What was almost 2K is now 250$ .. If I was Bill (which I am not) then I would be less than interested at this point for the amount of work that this entails.

                                                  Wrong thread :)  But this did go kind of dead all the same ;)

                                                  –Bill

                                                  pfSense core developer
                                                  blog - http://www.ucsecurity.com/
                                                  twitter - billmarquette

                                                  1 Reply Last reply Reply Quote 0
                                                  • S
                                                    sullrich
                                                    last edited by

                                                    @billm:

                                                    @sullrich:

                                                    Doubtful.  This thread went from promising to dead in the matter of a few days.  What was almost 2K is now 250$ .. If I was Bill (which I am not) then I would be less than interested at this point for the amount of work that this entails.

                                                    Wrong thread :)  But this did go kind of dead all the same ;)

                                                    –Bill

                                                    Second time this has happened to me.  Woops ;)

                                                    1 Reply Last reply Reply Quote 0
                                                    • N
                                                      narf
                                                      last edited by

                                                      @billm:

                                                      Wrong thread :)  But this did go kind of dead all the same ;)

                                                      –Bill

                                                      I was wondering why it went dead? I think we established that this would work with most NICs. I'd appreciate this functionality in pfsense. That way we'd have all the information in one logical place, instead of deploying multiply small appliances. I'd be willing to donate some cash to this effort as well… provided its still considered a possibility...

                                                      1 Reply Last reply Reply Quote 0
                                                      • M
                                                        mrks
                                                        last edited by

                                                        i've used ipfm before for this: http://robert.cheramy.net/ipfm/
                                                        freebsd port: /usr/ports/net-mgmt/ipfm

                                                        i'm just starting to play with pfsense and i think developers can do this better than me. maybe later on i can help with developing new packages.

                                                        Regards

                                                        1 Reply Last reply Reply Quote 0
                                                        • M
                                                          Mercredi
                                                          last edited by

                                                          i like this solution, this is what we want. it would be best if someone can do a package from ipfm.

                                                          1 Reply Last reply Reply Quote 0
                                                          • S
                                                            sullrich
                                                            last edited by

                                                            Alright, everyone else that pledged money in this thread agree?  I will seriously do this one.  It looks pretty helpful.

                                                            1 Reply Last reply Reply Quote 0
                                                            • N
                                                              narf
                                                              last edited by

                                                              Im not going to pretend to know anything about this, but can't the already built-in RRDtool graping functionality be tailored to do this for us?'

                                                              I mean a nice page that showes us the bandwidth statistics in text of each host on the network - and maybe checkboxes next to those hosts to create a nice RRD graph to see who (or in our case, what) is hogging the bandwidth over any given time frame. Maybe even functionality to define groups of hosts (read departments) for graphing for an easy overview of whats going on.

                                                              Regarding the pledges, I cannot speak for the others, but, given as this would be a valuable feature, I'd be willing to pledge $50 to this cause (cash is tight right now  :-\ )

                                                              1 Reply Last reply Reply Quote 0
                                                              • S
                                                                sullrich
                                                                last edited by

                                                                @narf:

                                                                Im not going to pretend to know anything about this, but can't the already built-in RRDtool graping functionality be tailored to do this for us?'

                                                                I mean a nice page that showes us the bandwidth statistics in text of each host on the network - and maybe checkboxes next to those hosts to create a nice RRD graph to see who (or in our case, what) is hogging the bandwidth over any given time frame. Maybe even functionality to define groups of hosts (read departments) for graphing for an easy overview of whats going on.

                                                                Regarding the pledges, I cannot speak for the others, but, given as this would be a valuable feature, I'd be willing to pledge $50 to this cause (cash is tight right now  :-\ )

                                                                Hrm.  This makes me think.  PFTOP + RRD would do something similar to this…

                                                                Scott

                                                                1 Reply Last reply Reply Quote 0
                                                                • M
                                                                  Mercredi
                                                                  last edited by

                                                                  here is my confirmation: i am agree.

                                                                  1 Reply Last reply Reply Quote 0
                                                                  • N
                                                                    narf
                                                                    last edited by

                                                                    @sullrich:

                                                                    @narf:

                                                                    Im not going to pretend to know anything about this, but can't the already built-in RRDtool graping functionality be tailored to do this for us?'

                                                                    I mean a nice page that showes us the bandwidth statistics in text of each host on the network - and maybe checkboxes next to those hosts to create a nice RRD graph to see who (or in our case, what) is hogging the bandwidth over any given time frame. Maybe even functionality to define groups of hosts (read departments) for graphing for an easy overview of whats going on.

                                                                    Regarding the pledges, I cannot speak for the others, but, given as this would be a valuable feature, I'd be willing to pledge $50 to this cause (cash is tight right now  :-\ )

                                                                    Hrm.  This makes me think.  PFTOP + RRD would do something similar to this…

                                                                    Scott

                                                                    Pretty graphs with useful information is what its all about ;)

                                                                    1 Reply Last reply Reply Quote 0
                                                                    • S
                                                                      sullrich
                                                                      last edited by

                                                                      Okay, I am ready to do this project.

                                                                      If everyone that has pledged money is serious, please paypal half of the money promised to sullrich@gmail.com

                                                                      Once I receive half of what everyone promised I will begin working on this.  Should be able to have the project knocked out in about 2-3 days time.  So if everyone can get the money to me sometime today then I will have all weekend to work on it.

                                                                      1 Reply Last reply Reply Quote 0
                                                                      • S
                                                                        sullrich
                                                                        last edited by

                                                                        I have only received 15$ so far.  If I do not receive the remaining money by tomorrow I will refund the 15$, lock this thread and will not be doing this package in the future.

                                                                        1 Reply Last reply Reply Quote 0
                                                                        • Y
                                                                          yoda715
                                                                          last edited by

                                                                          @sullrich:

                                                                          I have only received 15$ so far.  If I do not receive the remaining money by tomorrow I will refund the 15$, lock this thread and will not be doing this package in the future.

                                                                          Not trying to step on any toes Scott, but it's only been two days since you said you wanted the money. Perhaps a little more time is in order. :)

                                                                          1 Reply Last reply Reply Quote 0
                                                                          • S
                                                                            sullrich
                                                                            last edited by

                                                                            I had time this weekend, lots of it.  My schedule is filling up quick until the end of the year unfortunately.

                                                                            1 Reply Last reply Reply Quote 0
                                                                            • N
                                                                              narf
                                                                              last edited by

                                                                              I just paypalled off my half.

                                                                              I didn't see the new posts in this thread until now.

                                                                              Is it too late?

                                                                              I'd have to agree with sdale, this thread started Sept. 17th. Is been on and off since then. People aren't checking back as often as they were before…

                                                                              1 Reply Last reply Reply Quote 0
                                                                              • S
                                                                                sullrich
                                                                                last edited by

                                                                                I hear you all but I have a MAJOR development deal that I have to fulfill and I will not have any time before the end of the year.

                                                                                The package would have most likely already have been done if everyone was paying attention.  Oh well.

                                                                                1 Reply Last reply Reply Quote 0
                                                                                • M
                                                                                  Mercredi
                                                                                  last edited by

                                                                                  Hey Guys!
                                                                                  i cant paypal money right now, first of all i need to buy some eMoney (i am in russia, siberia), then convert it into something what can be paypalled. sorry for my english.

                                                                                  1 Reply Last reply Reply Quote 0
                                                                                  • M
                                                                                    Mercredi
                                                                                    last edited by

                                                                                    trying to send money from visa… hold on.

                                                                                    1 Reply Last reply Reply Quote 0
                                                                                    • First post
                                                                                      Last post