Arpwatch not downloading vendor ID's

michmoor · Jan 24, 2025, 6:18 PM

@dennypage no worries. no rush. Just wondering if i could provide a helping hand along the way. Appreciate yah !

dennypage · Feb 4, 2025, 1:19 AM

Took a bit longer than I expected, but hopefully this will reward your patience, and give you a good idea of where I'm headed:

Github: The ANDwatch daemon

There were just too many problems to overcome with arpwatch.

FWIW, it may be two or three weeks before I can do the pfSense package due to travel.

Benjamin 3 · Feb 4, 2025, 1:19 AM

@dennypage this looks great! Thank you. Safe travels :-)

keyser · Feb 4, 2025, 9:49 AM

@dennypage This looks very exiting indeed. Thank you very much for investing your valuable time in creating such a great tool/package for all of us

dennypage · Feb 4, 2025, 11:14 PM

The submission for FreeBSD (upstream) is in. Not sure how long it will take, Usually it's pretty quick.

If anyone would like to give ANDwatch a spin before the pfSense UI is done, please let me know and I will send you a copy of the FreeBSD package.

I'll get to the pfSense UI package written when I return.

michmoor · Feb 5, 2025, 4:44 PM

@dennypage

denny denny denny...what would we do without you :)

I can try out the FreeBSD package.

dennypage · Feb 6, 2025, 5:58 PM

@michmoor said in Arpwatch not downloading vendor ID's:

I can try out the FreeBSD package.

Thanks. I've included the package below. Please let me know if you have any issues.

Here is the notification script if you want it:

#!/usr/bin/env php
<?php
require_once("notices.inc");

$timestamp=$argv[1];
$ifname=convert_real_interface_to_friendly_descr($argv[2]);
$ipaddr=$argv[3];
$old_hwaddr=$argv[4];
$old_hwaddr_org=$argv[5];
$new_hwaddr=$argv[6];
$new_hwaddr_org=$argv[7];

$hostname = gethostbyaddr($ipaddr);

$msg = "ANDwatch notificaton\n\n";
$msg .= sprintf("%22s: %s\n", "timestamp", $timestamp);
$msg .= sprintf("%22s: %s\n", "interface", $ifname);
$msg .= sprintf("%22s: %s\n", "hostname", $hostname);
$msg .= sprintf("%22s: %s\n", "ip address", $ipaddr);
$msg .= sprintf("%22s: %s %s\n", "old ethernet address", $old_hwaddr, $old_hwaddr_org);
$msg .= sprintf("%22s: %s %s\n", "new ethernet address", $new_hwaddr, $new_hwaddr_org);

notify_all_remote($msg);
?>

I don't have anything to display a status page yet, but you can do a query via the command line like so:

andwatch-query <ifname>

That will give you a report of all the latest IP mappings.

[Edit: Updated pkg to v1.0.1 to fix query bug with MAC addresses beginning with '0']
[Edit: Updated pkg to v1.1.0 to change record update / age behavior. Details on GitHub.]

andwatch-1.1.0.pkg.zip

dennypage · Feb 5, 2025, 9:59 PM

Just because I hate packing...

Here are a couple of files that will get you going for a database query UI. Note that since there is no configuration you will need to hand edit the list of interfaces you are running ANDwatch on. The list is near the bottom of andwatch.inc.

/usr/local/pkg/andwatch.inc:

<?php
/*
 * andwatch.inc
 *
 * part of pfSense (https://www.pfsense.org)
 * Copyright (c) 2025 Denny Page
 * All rights reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

require_once("config.inc");
require_once("functions.inc");
require_once("util.inc");
require_once("service-utils.inc");

function andwatch_query_interfaces($ifnames)
{
    $entries = array();

    foreach($ifnames as $ifname) {
        $real_ifname = get_real_interface($ifname);
        $friendly_ifname = convert_friendly_interface_to_friendly_descr($ifname);
    
        $pipe = popen("/usr/local/bin/andwatch-query $real_ifname", 'r');
        if ($pipe) {
            while ($line = fgets($pipe)) {
                list($date, $time, $age, $ipaddr, $hwaddr, $org) = sscanf(trim($line), '%s %s %s %s %s %[^$]s');

                $hostname = gethostbyaddr($ipaddr);
                if ($hostname == $ipaddr) {
                    $hostname = "";
                }
 
                $entry = [
                    'ifdesc' => $friendly_ifname,
                    'datetime' => "$date $time",
                    'age' => $age,
                    'hostname' => $hostname,
                    'ipaddr' => $ipaddr,
                    'hwaddr' => $hwaddr,
                    'org' => $org
                ];
    
                $entries[] = $entry;
            }
            pclose($pipe);
        }
    }

    return $entries;
}

function andwatch_query_all()
{
    $ifnames = array("lan");
    //$ifnames = array("lan", "opt2", "opt3");

    return andwatch_query_interfaces($ifnames);
}

?>

/usr/local/www/andwatch_database.php:

<?php
/*
 * andwatch_database.php
 *
 * Copyright (c) 2025, Denny Page
 * All rights reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

require_once("guiconfig.inc");
require_once("andwatch.inc");

$pgtitle = array(gettext('Status'), gettext('ANDwatch'), gettext('Database'));
include("head.inc");

$entries = andwatch_query_all();
?>

<div class="panel panel-default">
    <div class="panel-heading"><h2 class="panel-title"><?=gettext('Database')?></h2></div>
    <div class="panel-body table-responsive">
        <table class="table table-striped table-hover table-condensed sortable-theme-bootstrap" data-sortable>
            <thead>
            <tr class="text-nowrap">
                <th><?=gettext("Interface")?></th>
                <th><?=gettext("DateTime")?></th>
                <th><?=gettext("Hostname")?></th>
                <th><?=gettext("IP Address")?></th>
                <th><?=gettext("MAC Address")?></th>
                <th><?=gettext("MAC Organization")?></th>
            </tr>
            </thead>
            <tbody>
            <?php if (count($entries)) : ?>
            <?php foreach ($entries as $entry): ?>
            <tr class="text-nowrap">
                <td><?=htmlspecialchars($entry['ifdesc'])?></td>
                <td><?=htmlspecialchars($entry['datetime'])?></td>
                <td><?=htmlspecialchars($entry['hostname'])?></td>
                <td><?=htmlspecialchars($entry['ipaddr'])?></td>
                <td><?=htmlspecialchars($entry['hwaddr'])?></td>
                <td><?=htmlspecialchars($entry['org'])?></td>
            </tr>
            <?php endforeach; ?>
            <?php else: ?>
            <tr>
                <td colspan="6"><?=gettext("No entries to display")?></td>
            </tr>
            <?php endif; ?>
            </tbody>
        </table>
</div>
<?php include("foot.inc"); ?>

dennypage · Feb 6, 2025, 6:19 PM

FYI, I've edited the original post to update the version of the package to 1.1.0. If you've pulled a prior version, please see the post containing the package to get an update.

dennypage · Feb 19, 2025, 9:59 PM

For those that are interested, here is a quick update.

Version 2.0.0 adds hostname for notify/query output, and support for user defined additions to the pcap filter. More detail can be found in the README.

I've included an updated pkg below, plus a tarball that contains these supporting files:

/usr/local/pkg/andwatch.inc
/usr/local/pkg/andwatch-notify.php
/usr/local/www/andwatch_database.php

Here is an example of how I am launching for my LAN interface:

/usr/local/bin/andwatchd -s -F 'not net fe80::0/10 and not net fc00::0/7' -n /usr/local/pkg/andwatch-notify.php ix0

I'm beginning work on the pfSense package now.

andwatch-2.0.0.pkg.zip
andwatch_pfsense_files.tgz

michmoor · Feb 24, 2025, 2:48 AM

@dennypage
Hey Denny,
Is it possible to add as part of the notification message what ports and/or vlan the arp message pertains to?
I sometimes get arpwatch messages about an IP that someone incorrectly self assigned to a machine and of course there’s a mac/arp conflict but it would be helpful to know which port this was seen on

dennypage · Feb 24, 2025, 4:02 PM

@michmoor said in Arpwatch not downloading vendor ID's:

Is it possible to add as part of the notification message what ports and/or vlan the arp message pertains to?

By “ports and/or vlan”, do you mean interface? If so, the interface is already part of the notification. If you mean something other than interface, please explain more.

Edit: This is what notifications look like with ANDwatch:

            timestamp: 2025-02-22 16:25:18
            interface: DEVICE
             hostname: myhost.mydomain
           ip address: 192.168.1.80
 new ethernet address: f8:b9:5a:37:1c:1e
     new ethernet org: LG Innotek
 old ethernet address: ac:f1:08:5e:85:24
     old ethernet org: LG Innotek

DEVICE is the interface name as seen in pfSense.

dennypage · Feb 24, 2025, 5:44 AM

Hey all, here is a version of the pfSense ANDwatch package for testing.

You'll need to have the andwatch package itself installed (posted a few days ago here) before installing the pfSense package.

Note that you can remove /usr/local/www/andwatch_database.php as it is no longer used.

Feedback welcome. Bonus points for finding bugs.

pfSense-pkg-ANDwatch-2.0.pkg.zip

michmoor · Feb 24, 2025, 3:27 PM

@dennypage
Yes I mean interface.
So it should say “igc3” or if using vlans “igc3.14”

dennypage · Feb 24, 2025, 4:09 PM

@michmoor said in Arpwatch not downloading vendor ID's:

So it should say “igc3” or if using vlans “igc3.14”

There are three levels of interface names in pfSense:

Friendly interface name -> Internal interface name -> Real interface name.

Some examples:

LAN -> lan -> ix0
GUEST -> opt2 -> igc0.2
DEVICE -> opt3 -> igc0.3

You can see the mapping in Status / Interfaces. The Internal and Real names are shown in parens following the Friendly name.

Pretty much everywhere in the GUI the Friendly name is the one used. If you go to Service / DHCP Server for instance, you will see "LAN", GUEST", "DEVICE", etc. Unless you are defining an interface you generally don't see/use the Internal or Real names. Friendly names are what administrators are most familiar with.

ANDwatch uses the Friendly name in the notification for this reason.

michmoor · Feb 24, 2025, 4:04 PM

@dennypage nice!
Thanks for the quick response Denny

patient0 · Feb 24, 2025, 5:20 PM

@dennypage I installed the two pkg on a 2.7.2 CE (on Proxmox), not sure that's supported at all.

Great work! Looks cool to me :o) ... some feedback:

there is no menu entry in the 'Status' menu, the status can only be access through the service 'ANDwatch'
there are two interface in my installation, LAN and TEST. On the LAN interface it shows a client and pfSense itself, all with the correct IP, MAC and hostname.
For the TEST network the connected client shows up correct but for the pfSense interface for network TEST the hostname is displayed as "(unknown)" (IP 200.1 in the picture). The other client with name "(unknown)" is disconnected.
maybe the icons search and clear icons are taller then they have to be :)

TEST with pfSense name as "unkown"
login-to-view

Missing Status menu entry
login-to-view

dennypage · Feb 24, 2025, 8:43 PM

@patient0 said in Arpwatch not downloading vendor ID's:

there is no menu entry in the 'Status' menu, the status can only be access through the service 'ANDwatch'

Something is really wrong. Did you refresh the pages after the install? There is no way to access Status from the Services / ANDwatch page--did you type in the URL by hand?

If refresh doesn't bring it up, check your config please. Look for the following:

                <menu>
                        <name>ANDwatch</name>
                        <tooltiptext>ANDwatch Settings</tooltiptext>
                        <section>Services</section>
                        <url>/andwatch.php</url>
                </menu>
                <menu>
                        <name>ANDwatch</name>
                        <tooltiptext>ANDwatch Status</tooltiptext>
                        <section>Status</section>
                        <url>/andwatch_status.php</url>
                </menu>

For the TEST network the connected client shows up correct but for the pfSense interface for network TEST the hostname is displayed as "(unknown)" (IP 200.1 in the picture). The other client with name "(unknown)" is disconnected.

If you log into your pfSense box, and run the command "host 10.99.200.104". This should respond with "Host 104.200.99.10.in-addr.arpa not found: 3(NXDOMAIN)"

maybe the icons search and clear icons are taller then they have to be :)

Very strange. You also don't have the icons. It should look identical to the Search panel on the DHCP Status page. Mine (24.11 & 25.03 beta) look like this:

For login-to-view

Two questions:

What browser are you using?
I don't have a 2.7.2 CE system. Can you run the following command on your system and send me the result please?

grep -i btn /usr/local/www/status_dhcp_leases.php

Thanks.

patient0 · Feb 24, 2025, 9:20 PM

@dennypage if it's too much work to support 2.7.2 CE just tell me, yes? Maybe it's easier to wait for 2.8.0 CE? Can only be a few months away.

Did you refresh the pages after the install? There is no way to access Status from the Services / ANDwatch page-

I did log out and in again, yes. The icon to access the status is in the service page, next to the "?":
login-to-view

check your config please. Look for the following

That section appears exactly like that in /usr/local/pkg/andwatch.xml. I assume Netgate made changes to the GUI framework in the newer versions.

If you log into your pfSense box, and run the command "host 10.99.200.104". This should respond with "Host 104.200.99.10.in-addr.arpa not found: 3(NXDOMAIN)"

You are right - although I was talking about 10.99.200.1 - pfSense itself on the TEST interface is not getting resolved. That way ANDwatch can't know the name, of course.

It should look identical to the Search panel on the DHCP Status page

I see, again probably changes to the GUI framework. I'll compare it with other package configs and see if I can see a difference.

What browser are you using?

Firefox 128.7.0esr on Debian 12

grep -i btn /usr/local/.../status_dhcp_leases.php

Below is output of the call:

[2.7.2-RELEASE][root@pfsense.home.arpa]/root: grep -i btn /usr/local/www/status_dhcp_leases.php
                                <a id="btnsearch" title="<?=gettext("Search")?>" class="btn btn-primary btn-sm"><i class="fa fa-search icon-embed-btn"></i><?=gettext("Search")?></a>
                                <a id="btnclear" title="<?=gettext("Clear")?>" class="btn btn-info btn-sm"><i class="fa fa-undo icon-embed-btn"></i><?=gettext("Clear")?></a>
        <a class="btn btn-info" href="status_dhcp_leases.php?all=0"><i class="fa fa-minus-circle icon-embed-btn"></i><?=gettext("Show Active and Static Leases Only")?></a>
        <a class="btn btn-info" href="status_dhcp_leases.php?all=1"><i class="fa fa-plus-circle icon-embed-btn"></i><?=gettext("Show All Configured Leases")?></a>
        <a class="btn btn-danger no-confirm" id="cleardhcp"><i class="fa fa-trash icon-embed-btn"></i><?=gettext("Clear All DHCP Leases")?></a>
        $("#btnsearch").prop('type', 'button');
        $("#btnclear").prop('type', 'button');
        $("#btnsearch").click(function() {
        $("#btnclear").click(function() {
                        $("#btnsearch").get(0).click();

Thanks you for taking the time!