Arpwatch not downloading vendor ID's
-
@dennypage
Hey Denny. Any assistance you need when looking at this package? I dont know how far along you are in the review. -
@michmoor said in Arpwatch not downloading vendor ID's:
Hey Denny. Any assistance you need when looking at this package? I dont know how far along you are in the review.
Hey. Sorry, I haven't forgotten, and I have been working on it. But as a friend of mine once said, "Some home improvement projects may be larger than they appear." This is one of those cases, rather like avahi -> mdns-bridge. I should be able to do a write up about where I'm headed sometime next week. Hopefully, people will find their patience rewarded.
In the interim, if you want the arpwatch patch I'm currently running with in production I've included it below. Note that this patch requires the Zero padded ethernet addresses option to be enabled.
--- arpwatch.xml.org 2024-11-27 11:19:46.000000000 -0800 +++ arpwatch.xml 2024-12-24 13:07:50.974190000 -0800 @@ -120,7 +120,7 @@ <fielddescr>Update vendors</fielddescr> <fieldname>update_vendors</fieldname> <type>checkbox</type> - <description>Updates the ethernet vendor database, downloaded from http://standards-oui.ieee.org/oui/oui.csv.</description> + <description>Updates the ethernet vendor database, downloaded from https://standards-oui.ieee.org/oui/oui.csv.</description> </field> <field> <fielddescr>Clear database</fielddescr> --- arpwatch.inc.org 2024-11-27 11:19:46.000000000 -0800 +++ arpwatch.inc 2024-12-26 11:01:01.839497000 -0800 @@ -19,7 +19,7 @@ */ define('ARPWATCH_LOCAL_DIR', '/usr/local/arpwatch'); -define('ARPWATCH_ETHERCODES_URL', 'http://standards-oui.ieee.org/oui/oui.csv'); +define('ARPWATCH_ETHERCODES_URL', 'https://standards-oui.ieee.org/oui/oui.csv'); define('ARPWATCH_SENDMAIL_PATH', '/usr/sbin/sendmail'); define('ARPWATCH_SENDMAIL_PROXY', '/usr/local/arpwatch/sendmail_proxy.php'); @@ -128,9 +128,7 @@ } function arpwatch_update_vendors($args) { - exec('/usr/bin/fetch -qo - '.ARPWATCH_ETHERCODES_URL.'|' - .ARPWATCH_LOCAL_DIR.'/massagevendor '.$args.' >' - .ARPWATCH_LOCAL_DIR.'/ethercodes.dat'); + exec('/usr/local/arpwatch/update-ethercodes'); } function arpwatch_clear_database() { @@ -174,7 +172,7 @@ $entry = [ 'ifname' => $ifname, - 'ifdescr' => strtoupper($active_interface), + 'ifdescr' => convert_friendly_interface_to_friendly_descr($active_interface), 'mac' => $mac, 'vendor' => $vendor, 'ip' => $ip, @@ -194,14 +192,6 @@ } } - usort($entries, function($e1, $e2){ - if ($e1['ifdescr'] == $e2['ifdescr']) { - return 0; - } - - return ($e1['ifdescr'] < $e2['ifdescr']) ? -1 : 1; - }); - return $entries; } -
@dennypage no worries. no rush. Just wondering if i could provide a helping hand along the way. Appreciate yah !
-
Took a bit longer than I expected, but hopefully this will reward your patience, and give you a good idea of where I'm headed:
Github: The ANDwatch daemon
There were just too many problems to overcome with arpwatch.
FWIW, it may be two or three weeks before I can do the pfSense package due to travel.
-
@dennypage this looks great! Thank you. Safe travels :-)
-
@dennypage This looks very exiting indeed. Thank you very much for investing your valuable time in creating such a great tool/package for all of us
-
The submission for FreeBSD (upstream) is in. Not sure how long it will take, Usually it's pretty quick.
If anyone would like to give ANDwatch a spin before the pfSense UI is done, please let me know and I will send you a copy of the FreeBSD package.
I'll get to the pfSense UI package written when I return.
-
-
@michmoor said in Arpwatch not downloading vendor ID's:
I can try out the FreeBSD package.
Thanks. I've included the package below. Please let me know if you have any issues.
Here is the notification script if you want it:
#!/usr/bin/env php <?php require_once("notices.inc"); $timestamp=$argv[1]; $ifname=convert_real_interface_to_friendly_descr($argv[2]); $ipaddr=$argv[3]; $old_hwaddr=$argv[4]; $old_hwaddr_org=$argv[5]; $new_hwaddr=$argv[6]; $new_hwaddr_org=$argv[7]; $hostname = gethostbyaddr($ipaddr); $msg = "ANDwatch notificaton\n\n"; $msg .= sprintf("%22s: %s\n", "timestamp", $timestamp); $msg .= sprintf("%22s: %s\n", "interface", $ifname); $msg .= sprintf("%22s: %s\n", "hostname", $hostname); $msg .= sprintf("%22s: %s\n", "ip address", $ipaddr); $msg .= sprintf("%22s: %s %s\n", "old ethernet address", $old_hwaddr, $old_hwaddr_org); $msg .= sprintf("%22s: %s %s\n", "new ethernet address", $new_hwaddr, $new_hwaddr_org); notify_all_remote($msg); ?>I don't have anything to display a status page yet, but you can do a query via the command line like so:
andwatch-query <ifname>That will give you a report of all the latest IP mappings.
[Edit: Updated pkg to v1.0.1 to fix query bug with MAC addresses beginning with '0']
[Edit: Updated pkg to v1.1.0 to change record update / age behavior. Details on GitHub.] -
Just because I hate packing...
Here are a couple of files that will get you going for a database query UI. Note that since there is no configuration you will need to hand edit the list of interfaces you are running ANDwatch on. The list is near the bottom of andwatch.inc.
/usr/local/pkg/andwatch.inc:
<?php /* * andwatch.inc * * part of pfSense (https://www.pfsense.org) * Copyright (c) 2025 Denny Page * All rights reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ require_once("config.inc"); require_once("functions.inc"); require_once("util.inc"); require_once("service-utils.inc"); function andwatch_query_interfaces($ifnames) { $entries = array(); foreach($ifnames as $ifname) { $real_ifname = get_real_interface($ifname); $friendly_ifname = convert_friendly_interface_to_friendly_descr($ifname); $pipe = popen("/usr/local/bin/andwatch-query $real_ifname", 'r'); if ($pipe) { while ($line = fgets($pipe)) { list($date, $time, $age, $ipaddr, $hwaddr, $org) = sscanf(trim($line), '%s %s %s %s %s %[^$]s'); $hostname = gethostbyaddr($ipaddr); if ($hostname == $ipaddr) { $hostname = ""; } $entry = [ 'ifdesc' => $friendly_ifname, 'datetime' => "$date $time", 'age' => $age, 'hostname' => $hostname, 'ipaddr' => $ipaddr, 'hwaddr' => $hwaddr, 'org' => $org ]; $entries[] = $entry; } pclose($pipe); } } return $entries; } function andwatch_query_all() { $ifnames = array("lan"); //$ifnames = array("lan", "opt2", "opt3"); return andwatch_query_interfaces($ifnames); } ?>/usr/local/www/andwatch_database.php:
<?php /* * andwatch_database.php * * Copyright (c) 2025, Denny Page * All rights reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ require_once("guiconfig.inc"); require_once("andwatch.inc"); $pgtitle = array(gettext('Status'), gettext('ANDwatch'), gettext('Database')); include("head.inc"); $entries = andwatch_query_all(); ?> <div class="panel panel-default"> <div class="panel-heading"><h2 class="panel-title"><?=gettext('Database')?></h2></div> <div class="panel-body table-responsive"> <table class="table table-striped table-hover table-condensed sortable-theme-bootstrap" data-sortable> <thead> <tr class="text-nowrap"> <th><?=gettext("Interface")?></th> <th><?=gettext("DateTime")?></th> <th><?=gettext("Hostname")?></th> <th><?=gettext("IP Address")?></th> <th><?=gettext("MAC Address")?></th> <th><?=gettext("MAC Organization")?></th> </tr> </thead> <tbody> <?php if (count($entries)) : ?> <?php foreach ($entries as $entry): ?> <tr class="text-nowrap"> <td><?=htmlspecialchars($entry['ifdesc'])?></td> <td><?=htmlspecialchars($entry['datetime'])?></td> <td><?=htmlspecialchars($entry['hostname'])?></td> <td><?=htmlspecialchars($entry['ipaddr'])?></td> <td><?=htmlspecialchars($entry['hwaddr'])?></td> <td><?=htmlspecialchars($entry['org'])?></td> </tr> <?php endforeach; ?> <?php else: ?> <tr> <td colspan="6"><?=gettext("No entries to display")?></td> </tr> <?php endif; ?> </tbody> </table> </div> <?php include("foot.inc"); ?> -
FYI, I've edited the original post to update the version of the package to 1.1.0. If you've pulled a prior version, please see the post containing the package to get an update.
-
For those that are interested, here is a quick update.
Version 2.0.0 adds hostname for notify/query output, and support for user defined additions to the pcap filter. More detail can be found in the README.
I've included an updated pkg below, plus a tarball that contains these supporting files:
- /usr/local/pkg/andwatch.inc
- /usr/local/pkg/andwatch-notify.php
- /usr/local/www/andwatch_database.php
Here is an example of how I am launching for my LAN interface:
/usr/local/bin/andwatchd -s -F 'not net fe80::0/10 and not net fc00::0/7' -n /usr/local/pkg/andwatch-notify.php ix0I'm beginning work on the pfSense package now.
-
@dennypage
Hey Denny,
Is it possible to add as part of the notification message what ports and/or vlan the arp message pertains to?
I sometimes get arpwatch messages about an IP that someone incorrectly self assigned to a machine and of course there’s a mac/arp conflict but it would be helpful to know which port this was seen on -
@michmoor said in Arpwatch not downloading vendor ID's:
Is it possible to add as part of the notification message what ports and/or vlan the arp message pertains to?
By “ports and/or vlan”, do you mean interface? If so, the interface is already part of the notification. If you mean something other than interface, please explain more.
Edit: This is what notifications look like with ANDwatch:
timestamp: 2025-02-22 16:25:18 interface: DEVICE hostname: myhost.mydomain ip address: 192.168.1.80 new ethernet address: f8:b9:5a:37:1c:1e new ethernet org: LG Innotek old ethernet address: ac:f1:08:5e:85:24 old ethernet org: LG InnotekDEVICEis the interface name as seen in pfSense. -
Hey all, here is a version of the pfSense ANDwatch package for testing.
You'll need to have the andwatch package itself installed (posted a few days ago here) before installing the pfSense package.
Note that you can remove /usr/local/www/andwatch_database.php as it is no longer used.
Feedback welcome. Bonus points for finding bugs.
-
@dennypage
Yes I mean interface.
So it should say “igc3” or if using vlans “igc3.14” -
@michmoor said in Arpwatch not downloading vendor ID's:
So it should say “igc3” or if using vlans “igc3.14”
There are three levels of interface names in pfSense:
Friendly interface name -> Internal interface name -> Real interface name.Some examples:
LAN -> lan -> ix0 GUEST -> opt2 -> igc0.2 DEVICE -> opt3 -> igc0.3You can see the mapping in Status / Interfaces. The Internal and Real names are shown in parens following the Friendly name.
Pretty much everywhere in the GUI the Friendly name is the one used. If you go to Service / DHCP Server for instance, you will see "LAN", GUEST", "DEVICE", etc. Unless you are defining an interface you generally don't see/use the Internal or Real names. Friendly names are what administrators are most familiar with.
ANDwatch uses the Friendly name in the notification for this reason.
-
@dennypage nice!
Thanks for the quick response Denny -
@dennypage I installed the two pkg on a 2.7.2 CE (on Proxmox), not sure that's supported at all.
Great work! Looks cool to me :o) ... some feedback:
- there is no menu entry in the 'Status' menu, the status can only be access through the service 'ANDwatch'
- there are two interface in my installation, LAN and TEST. On the LAN interface it shows a client and pfSense itself, all with the correct IP, MAC and hostname.
For the TEST network the connected client shows up correct but for the pfSense interface for network TEST the hostname is displayed as "(unknown)" (IP 200.1 in the picture). The other client with name "(unknown)" is disconnected. - maybe the icons search and clear icons are taller then they have to be :)
TEST with pfSense name as "unkown"
Missing Status menu entry
-
@patient0 said in Arpwatch not downloading vendor ID's:
there is no menu entry in the 'Status' menu, the status can only be access through the service 'ANDwatch'
Something is really wrong. Did you refresh the pages after the install? There is no way to access Status from the Services / ANDwatch page--did you type in the URL by hand?
If refresh doesn't bring it up, check your config please. Look for the following:
<menu> <name>ANDwatch</name> <tooltiptext>ANDwatch Settings</tooltiptext> <section>Services</section> <url>/andwatch.php</url> </menu> <menu> <name>ANDwatch</name> <tooltiptext>ANDwatch Status</tooltiptext> <section>Status</section> <url>/andwatch_status.php</url> </menu>For the TEST network the connected client shows up correct but for the pfSense interface for network TEST the hostname is displayed as "(unknown)" (IP 200.1 in the picture). The other client with name "(unknown)" is disconnected.
If you log into your pfSense box, and run the command "host 10.99.200.104". This should respond with "Host 104.200.99.10.in-addr.arpa not found: 3(NXDOMAIN)"
maybe the icons search and clear icons are taller then they have to be :)
Very strange. You also don't have the icons. It should look identical to the Search panel on the DHCP Status page. Mine (24.11 & 25.03 beta) look like this:
For
Two questions:
- What browser are you using?
- I don't have a 2.7.2 CE system. Can you run the following command on your system and send me the result please?
grep -i btn /usr/local/www/status_dhcp_leases.phpThanks.
