Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Arpwatch not downloading vendor ID's

    Scheduled Pinned Locked Moved Traffic Monitoring
    46 Posts 10 Posters 3.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • dennypageD
      dennypage @michmoor
      last edited by dennypage

      @michmoor said in Arpwatch not downloading vendor ID's:

      I can try out the FreeBSD package.

      Thanks. I've included the package below. Please let me know if you have any issues.

      Here is the notification script if you want it:

      #!/usr/bin/env php
      <?php
      require_once("notices.inc");
      
      $timestamp=$argv[1];
      $ifname=convert_real_interface_to_friendly_descr($argv[2]);
      $ipaddr=$argv[3];
      $old_hwaddr=$argv[4];
      $old_hwaddr_org=$argv[5];
      $new_hwaddr=$argv[6];
      $new_hwaddr_org=$argv[7];
      
      $hostname = gethostbyaddr($ipaddr);
      
      $msg = "ANDwatch notificaton\n\n";
      $msg .= sprintf("%22s: %s\n", "timestamp", $timestamp);
      $msg .= sprintf("%22s: %s\n", "interface", $ifname);
      $msg .= sprintf("%22s: %s\n", "hostname", $hostname);
      $msg .= sprintf("%22s: %s\n", "ip address", $ipaddr);
      $msg .= sprintf("%22s: %s %s\n", "old ethernet address", $old_hwaddr, $old_hwaddr_org);
      $msg .= sprintf("%22s: %s %s\n", "new ethernet address", $new_hwaddr, $new_hwaddr_org);
      
      notify_all_remote($msg);
      ?>
      

      I don't have anything to display a status page yet, but you can do a query via the command line like so:

      andwatch-query <ifname>
      

      That will give you a report of all the latest IP mappings.

      [Edit: Updated pkg to v1.0.1 to fix query bug with MAC addresses beginning with '0']
      [Edit: Updated pkg to v1.1.0 to change record update / age behavior. Details on GitHub.]

      andwatch-1.1.0.pkg.zip

      1 Reply Last reply Reply Quote 1
      • dennypageD
        dennypage
        last edited by dennypage

        Just because I hate packing...

        Here are a couple of files that will get you going for a database query UI. Note that since there is no configuration you will need to hand edit the list of interfaces you are running ANDwatch on. The list is near the bottom of andwatch.inc.

        /usr/local/pkg/andwatch.inc:

        <?php
        /*
         * andwatch.inc
         *
         * part of pfSense (https://www.pfsense.org)
         * Copyright (c) 2025 Denny Page
         * All rights reserved.
         *
         * Licensed under the Apache License, Version 2.0 (the "License");
         * you may not use this file except in compliance with the License.
         * You may obtain a copy of the License at
         *
         * http://www.apache.org/licenses/LICENSE-2.0
         *
         * Unless required by applicable law or agreed to in writing, software
         * distributed under the License is distributed on an "AS IS" BASIS,
         * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
         * See the License for the specific language governing permissions and
         * limitations under the License.
         */
        
        require_once("config.inc");
        require_once("functions.inc");
        require_once("util.inc");
        require_once("service-utils.inc");
        
        function andwatch_query_interfaces($ifnames)
        {
            $entries = array();
        
            foreach($ifnames as $ifname) {
                $real_ifname = get_real_interface($ifname);
                $friendly_ifname = convert_friendly_interface_to_friendly_descr($ifname);
            
                $pipe = popen("/usr/local/bin/andwatch-query $real_ifname", 'r');
                if ($pipe) {
                    while ($line = fgets($pipe)) {
                        list($date, $time, $age, $ipaddr, $hwaddr, $org) = sscanf(trim($line), '%s %s %s %s %s %[^$]s');
        
                        $hostname = gethostbyaddr($ipaddr);
                        if ($hostname == $ipaddr) {
                            $hostname = "";
                        }
         
                        $entry = [
                            'ifdesc' => $friendly_ifname,
                            'datetime' => "$date $time",
                            'age' => $age,
                            'hostname' => $hostname,
                            'ipaddr' => $ipaddr,
                            'hwaddr' => $hwaddr,
                            'org' => $org
                        ];
            
                        $entries[] = $entry;
                    }
                    pclose($pipe);
                }
            }
        
            return $entries;
        }
        
        function andwatch_query_all()
        {
            $ifnames = array("lan");
            //$ifnames = array("lan", "opt2", "opt3");
        
            return andwatch_query_interfaces($ifnames);
        }
        
        ?>
        

        /usr/local/www/andwatch_database.php:

        <?php
        /*
         * andwatch_database.php
         *
         * Copyright (c) 2025, Denny Page
         * All rights reserved.
         *
         * Licensed under the Apache License, Version 2.0 (the "License");
         * you may not use this file except in compliance with the License.
         * You may obtain a copy of the License at
         *
         * http://www.apache.org/licenses/LICENSE-2.0
         *
         * Unless required by applicable law or agreed to in writing, software
         * distributed under the License is distributed on an "AS IS" BASIS,
         * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
         * See the License for the specific language governing permissions and
         * limitations under the License.
         */
        
        require_once("guiconfig.inc");
        require_once("andwatch.inc");
        
        $pgtitle = array(gettext('Status'), gettext('ANDwatch'), gettext('Database'));
        include("head.inc");
        
        $entries = andwatch_query_all();
        ?>
        
        <div class="panel panel-default">
            <div class="panel-heading"><h2 class="panel-title"><?=gettext('Database')?></h2></div>
            <div class="panel-body table-responsive">
                <table class="table table-striped table-hover table-condensed sortable-theme-bootstrap" data-sortable>
                    <thead>
                    <tr class="text-nowrap">
                        <th><?=gettext("Interface")?></th>
                        <th><?=gettext("DateTime")?></th>
                        <th><?=gettext("Hostname")?></th>
                        <th><?=gettext("IP Address")?></th>
                        <th><?=gettext("MAC Address")?></th>
                        <th><?=gettext("MAC Organization")?></th>
                    </tr>
                    </thead>
                    <tbody>
                    <?php if (count($entries)) : ?>
                    <?php foreach ($entries as $entry): ?>
                    <tr class="text-nowrap">
                        <td><?=htmlspecialchars($entry['ifdesc'])?></td>
                        <td><?=htmlspecialchars($entry['datetime'])?></td>
                        <td><?=htmlspecialchars($entry['hostname'])?></td>
                        <td><?=htmlspecialchars($entry['ipaddr'])?></td>
                        <td><?=htmlspecialchars($entry['hwaddr'])?></td>
                        <td><?=htmlspecialchars($entry['org'])?></td>
                    </tr>
                    <?php endforeach; ?>
                    <?php else: ?>
                    <tr>
                        <td colspan="6"><?=gettext("No entries to display")?></td>
                    </tr>
                    <?php endif; ?>
                    </tbody>
                </table>
        </div>
        <?php include("foot.inc"); ?>
        
        1 Reply Last reply Reply Quote 0
        • dennypageD
          dennypage
          last edited by

          FYI, I've edited the original post to update the version of the package to 1.1.0. If you've pulled a prior version, please see the post containing the package to get an update.

          1 Reply Last reply Reply Quote 0
          • dennypageD
            dennypage
            last edited by dennypage

            For those that are interested, here is a quick update.

            Version 2.0.0 adds hostname for notify/query output, and support for user defined additions to the pcap filter. More detail can be found in the README.

            I've included an updated pkg below, plus a tarball that contains these supporting files:

            • /usr/local/pkg/andwatch.inc
            • /usr/local/pkg/andwatch-notify.php
            • /usr/local/www/andwatch_database.php

            Here is an example of how I am launching for my LAN interface:

            /usr/local/bin/andwatchd -s -F 'not net fe80::0/10 and not net fc00::0/7' -n /usr/local/pkg/andwatch-notify.php ix0
            

            I'm beginning work on the pfSense package now.

            andwatch-2.0.0.pkg.zip
            andwatch_pfsense_files.tgz

            M 1 Reply Last reply Reply Quote 3
            • M
              michmoor LAYER 8 Rebel Alliance @dennypage
              last edited by

              @dennypage
              Hey Denny,
              Is it possible to add as part of the notification message what ports and/or vlan the arp message pertains to?
              I sometimes get arpwatch messages about an IP that someone incorrectly self assigned to a machine and of course there’s a mac/arp conflict but it would be helpful to know which port this was seen on

              Firewall: NetGate,Palo Alto-VM,Juniper SRX
              Routing: Juniper, Arista, Cisco
              Switching: Juniper, Arista, Cisco
              Wireless: Unifi, Aruba IAP
              JNCIP,CCNP Enterprise

              dennypageD 1 Reply Last reply Reply Quote 0
              • dennypageD
                dennypage @michmoor
                last edited by dennypage

                @michmoor said in Arpwatch not downloading vendor ID's:

                Is it possible to add as part of the notification message what ports and/or vlan the arp message pertains to?

                By “ports and/or vlan”, do you mean interface? If so, the interface is already part of the notification. If you mean something other than interface, please explain more.

                Edit: This is what notifications look like with ANDwatch:

                            timestamp: 2025-02-22 16:25:18
                            interface: DEVICE
                             hostname: myhost.mydomain
                           ip address: 192.168.1.80
                 new ethernet address: f8:b9:5a:37:1c:1e
                     new ethernet org: LG Innotek
                 old ethernet address: ac:f1:08:5e:85:24
                     old ethernet org: LG Innotek
                

                DEVICE is the interface name as seen in pfSense.

                M 1 Reply Last reply Reply Quote 0
                • dennypageD
                  dennypage
                  last edited by

                  Hey all, here is a version of the pfSense ANDwatch package for testing.

                  You'll need to have the andwatch package itself installed (posted a few days ago here) before installing the pfSense package.

                  Note that you can remove /usr/local/www/andwatch_database.php as it is no longer used.

                  Feedback welcome. Bonus points for finding bugs. 🤠

                  pfSense-pkg-ANDwatch-2.0.pkg.zip

                  patient0P 1 Reply Last reply Reply Quote 1
                  • M
                    michmoor LAYER 8 Rebel Alliance @dennypage
                    last edited by

                    @dennypage
                    Yes I mean interface.
                    So it should say “igc3” or if using vlans “igc3.14”

                    Firewall: NetGate,Palo Alto-VM,Juniper SRX
                    Routing: Juniper, Arista, Cisco
                    Switching: Juniper, Arista, Cisco
                    Wireless: Unifi, Aruba IAP
                    JNCIP,CCNP Enterprise

                    dennypageD 1 Reply Last reply Reply Quote 0
                    • dennypageD
                      dennypage @michmoor
                      last edited by dennypage

                      @michmoor said in Arpwatch not downloading vendor ID's:

                      So it should say “igc3” or if using vlans “igc3.14”

                      There are three levels of interface names in pfSense:

                      Friendly interface name -> Internal interface name -> Real interface name.
                      

                      Some examples:

                      LAN -> lan -> ix0
                      GUEST -> opt2 -> igc0.2
                      DEVICE -> opt3 -> igc0.3
                      

                      You can see the mapping in Status / Interfaces. The Internal and Real names are shown in parens following the Friendly name.

                      Pretty much everywhere in the GUI the Friendly name is the one used. If you go to Service / DHCP Server for instance, you will see "LAN", GUEST", "DEVICE", etc. Unless you are defining an interface you generally don't see/use the Internal or Real names. Friendly names are what administrators are most familiar with.

                      ANDwatch uses the Friendly name in the notification for this reason.

                      M 1 Reply Last reply Reply Quote 1
                      • M
                        michmoor LAYER 8 Rebel Alliance @dennypage
                        last edited by

                        @dennypage nice!
                        Thanks for the quick response Denny

                        Firewall: NetGate,Palo Alto-VM,Juniper SRX
                        Routing: Juniper, Arista, Cisco
                        Switching: Juniper, Arista, Cisco
                        Wireless: Unifi, Aruba IAP
                        JNCIP,CCNP Enterprise

                        1 Reply Last reply Reply Quote 1
                        • patient0P
                          patient0 @dennypage
                          last edited by

                          @dennypage I installed the two pkg on a 2.7.2 CE (on Proxmox), not sure that's supported at all.

                          Great work! Looks cool to me :o) ... some feedback:

                          • there is no menu entry in the 'Status' menu, the status can only be access through the service 'ANDwatch'
                          • there are two interface in my installation, LAN and TEST. On the LAN interface it shows a client and pfSense itself, all with the correct IP, MAC and hostname.
                            For the TEST network the connected client shows up correct but for the pfSense interface for network TEST the hostname is displayed as "(unknown)" (IP 200.1 in the picture). The other client with name "(unknown)" is disconnected.
                          • maybe the icons search and clear icons are taller then they have to be :)

                          TEST with pfSense name as "unkown"
                          Screenshot 2025-02-24 at 17.14.42.png

                          Missing Status menu entry
                          Screenshot 2025-02-24 at 17.17.48.jpeg

                          dennypageD 1 Reply Last reply Reply Quote 0
                          • dennypageD
                            dennypage @patient0
                            last edited by

                            @patient0 said in Arpwatch not downloading vendor ID's:

                            there is no menu entry in the 'Status' menu, the status can only be access through the service 'ANDwatch'

                            Something is really wrong. Did you refresh the pages after the install? There is no way to access Status from the Services / ANDwatch page--did you type in the URL by hand?

                            If refresh doesn't bring it up, check your config please. Look for the following:

                                            <menu>
                                                    <name>ANDwatch</name>
                                                    <tooltiptext>ANDwatch Settings</tooltiptext>
                                                    <section>Services</section>
                                                    <url>/andwatch.php</url>
                                            </menu>
                                            <menu>
                                                    <name>ANDwatch</name>
                                                    <tooltiptext>ANDwatch Status</tooltiptext>
                                                    <section>Status</section>
                                                    <url>/andwatch_status.php</url>
                                            </menu>
                            

                            For the TEST network the connected client shows up correct but for the pfSense interface for network TEST the hostname is displayed as "(unknown)" (IP 200.1 in the picture). The other client with name "(unknown)" is disconnected.

                            If you log into your pfSense box, and run the command "host 10.99.200.104". This should respond with "Host 104.200.99.10.in-addr.arpa not found: 3(NXDOMAIN)"

                            maybe the icons search and clear icons are taller then they have to be :)

                            Very strange. You also don't have the icons. It should look identical to the Search panel on the DHCP Status page. Mine (24.11 & 25.03 beta) look like this:

                            ForScreenshot 2025-02-24 at 12.31.15.png

                            Two questions:

                            • What browser are you using?
                            • I don't have a 2.7.2 CE system. Can you run the following command on your system and send me the result please?
                            grep -i btn /usr/local/www/status_dhcp_leases.php
                            

                            Thanks.

                            patient0P 1 Reply Last reply Reply Quote 0
                            • patient0P
                              patient0 @dennypage
                              last edited by

                              @dennypage if it's too much work to support 2.7.2 CE just tell me, yes? Maybe it's easier to wait for 2.8.0 CE? Can only be a few months away.

                              Did you refresh the pages after the install? There is no way to access Status from the Services / ANDwatch page-

                              I did log out and in again, yes. The icon to access the status is in the service page, next to the "?":
                              Screenshot 2025-02-24 at 20.54.46.png

                              check your config please. Look for the following

                              That section appears exactly like that in /usr/local/pkg/andwatch.xml. I assume Netgate made changes to the GUI framework in the newer versions.

                              If you log into your pfSense box, and run the command "host 10.99.200.104". This should respond with "Host 104.200.99.10.in-addr.arpa not found: 3(NXDOMAIN)"

                              You are right - although I was talking about 10.99.200.1 - pfSense itself on the TEST interface is not getting resolved. That way ANDwatch can't know the name, of course.

                              It should look identical to the Search panel on the DHCP Status page

                              I see, again probably changes to the GUI framework. I'll compare it with other package configs and see if I can see a difference.

                              What browser are you using?

                              Firefox 128.7.0esr on Debian 12

                              grep -i btn /usr/local/.../status_dhcp_leases.php

                              Below is output of the call:

                              [2.7.2-RELEASE][root@pfsense.home.arpa]/root: grep -i btn /usr/local/www/status_dhcp_leases.php
                                                              <a id="btnsearch" title="<?=gettext("Search")?>" class="btn btn-primary btn-sm"><i class="fa fa-search icon-embed-btn"></i><?=gettext("Search")?></a>
                                                              <a id="btnclear" title="<?=gettext("Clear")?>" class="btn btn-info btn-sm"><i class="fa fa-undo icon-embed-btn"></i><?=gettext("Clear")?></a>
                                      <a class="btn btn-info" href="status_dhcp_leases.php?all=0"><i class="fa fa-minus-circle icon-embed-btn"></i><?=gettext("Show Active and Static Leases Only")?></a>
                                      <a class="btn btn-info" href="status_dhcp_leases.php?all=1"><i class="fa fa-plus-circle icon-embed-btn"></i><?=gettext("Show All Configured Leases")?></a>
                                      <a class="btn btn-danger no-confirm" id="cleardhcp"><i class="fa fa-trash icon-embed-btn"></i><?=gettext("Clear All DHCP Leases")?></a>
                                      $("#btnsearch").prop('type', 'button');
                                      $("#btnclear").prop('type', 'button');
                                      $("#btnsearch").click(function() {
                                      $("#btnclear").click(function() {
                                                      $("#btnsearch").get(0).click();
                              

                              Thanks you for taking the time!

                              dennypageD 1 Reply Last reply Reply Quote 0
                              • dennypageD
                                dennypage @patient0
                                last edited by

                                @patient0 said in Arpwatch not downloading vendor ID's:

                                That section appears exactly like that in /usr/local/pkg/andwatch.xml.

                                Can you check in your actual running config please? Like download it via Backup and see if both sections are in there?

                                @patient0 said in Arpwatch not downloading vendor ID's:

                                The icon to access the status is in the service page, next to the "?":

                                Ah, my bad. I forgot about the shortcuts.

                                @patient0 said in Arpwatch not downloading vendor ID's:

                                Below is output of the call:

                                Yep, that's got it. They changed from "fa" to "fa-solid" a couple of releases ago. Not sure why.

                                Anyway, if you edit andwatch_status.php and change occurrences of "fa-solid" to just "fa" it should look like it's supposed to.

                                patient0P 1 Reply Last reply Reply Quote 0
                                • patient0P
                                  patient0 @dennypage
                                  last edited by

                                  @dennypage said in Arpwatch not downloading vendor ID's:

                                  Can you check in your actual running config please? Like download it via Backup and see if both sections are in there?

                                  That section is not in the backup file, I'll investigate. Installed HAproxy and compared the config, looks really the same confused

                                  dennypageD 1 Reply Last reply Reply Quote 0
                                  • dennypageD
                                    dennypage @patient0
                                    last edited by

                                    @patient0 You might try a re-install of the package.

                                    patient0P 1 Reply Last reply Reply Quote 0
                                    • patient0P
                                      patient0 @dennypage
                                      last edited by

                                      @dennypage I dove into pkg create and tried a few things. In the end it was simple as often:

                                      On 2.7.2 CE in andwatch.xml the menu item name for "Status" can't be the same as for "Services". Renaming the <name> tag value of the menu item for Status from "ANDwatch" to "ANDwatch Status" resolved it. It still works works on 25.03-BETA :)

                                      Btw: the value of the <name> tag right after the </copyright> tag seems to be lower case for the other packages I checked (HAproy, Shellcmd, Avahi, LCDproc). Made no functional difference though.

                                      dennypageD 2 Replies Last reply Reply Quote 0
                                      • dennypageD
                                        dennypage @patient0
                                        last edited by dennypage

                                        @patient0 said in Arpwatch not downloading vendor ID's:

                                        On 2.7.2 CE in andwatch.xml the menu item name for "Status" can't be the same as for "Services". Renaming the <name> tag value of the menu item for Status from "ANDwatch" to "ANDwatch Status" resolved it. It still works works on 25.03-BETA :)

                                        Excellent catch. I'll change that. Thanks.

                                        @patient0 said in Arpwatch not downloading vendor ID's:

                                        Btw: the value of the <name> tag right after the </copyright> tag seems to be lower case for the other packages I checked (HAproy, Shellcmd, Avahi, LCDproc). Made no functional difference though.

                                        I think that one is okay. There are a few on my system that aren't all lower case:

                                        • System Patches
                                        • RRD Summary
                                        • Netgate Firmware Upgrade
                                        1 Reply Last reply Reply Quote 1
                                        • dennypageD
                                          dennypage @patient0
                                          last edited by

                                          @patient0 Here is an updated package. You should remove the prior package before installing the new version.

                                          Note that it still has the "fa-solid", so you'll need to make that change again, except the file name to edit is now status_andwatch.php rather than andwatch_status.php in order to match naming convention.

                                          pfSense-pkg-ANDwatch-2.0.pkg.zip

                                          patient0P 1 Reply Last reply Reply Quote 1
                                          • patient0P
                                            patient0 @dennypage
                                            last edited by

                                            @dennypage Thanks a lot, you are awesome :)

                                            For the fa-solid to fa conversion I have created a patch and applied via System patches.

                                            It's attached for reference if someone else would need it:

                                            andwatch-2.0_pfSense-CE-272-status-CSS-fix.diff

                                            In System / Patches, set "Path Strip Count" set to 0, leave the other settings as they are.

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.