Arpwatch not downloading vendor ID's

Gertjan

Isn't that what is suggested when you install arpwatch :

....
Executing custom_php_resync_config_command()...done.
Menu items... done.
Services... done.
Writing configuration... done.
=====
Message from arpwatch-3.7:

--
You can create an ethercodes.dat file by running this script:

    /usr/local/arpwatch/update-ethercodes

Here's a example crontab entry to update it every night:

    00 0 * * * root sleep `jot -r 1 0 600` ; /usr/local/arpwatch/update-ethercodes

The -m flag was removed. If you were using the -m watcher flag,
please switch to -w.
>>> Cleaning up cache...done.
Success

dennypage

@Gertjan Those are standard messages from the underlying arpwatch package (arpwatch-3.7), rather than the pfSense arpwatch package (pfSense-pkg-arpwatch-0.2.3).

The pfSense arpwatch package should be handling the update of the vendor database, but it appears to currently be broken. Perhaps a holiday project...

dennypage

It looks like the pfSense arpwatch package as a whole needs an update. A couple of things are out of date, in particular the concept that Update vendors is optional--arpwatch no longer ships a default database.

The database update problem stems from the fact that the package is standing on its head to support the ability to have non zero padded MAC addresses. Without compressed formats, it's an easy fix.

Given that the database itself is inherently zero padded, and that other subsystems such as DHCP use zero padded MAC addresses, does anyone see a reason to maintain the option of compressed MAC addresses?

johnpoz

@dennypage your not the owner of the arpwatch package are you? If you could show it some love that would be fantastic - it needs a little tlc for sure..

dennypage

@johnpoz said in Arpwatch not downloading vendor ID's:

@dennypage your not the owner of the arpwatch package are you? If you could show it some love that would be fantastic - it needs a little tlc for sure..

I'm not, but thinking about picking it up. It definitely needs a bit of TLC.

keyser

@dennypage That would be really excellent if you could find the time. I love what you have done on other packages you created/maintain :-)

darcey

There seems a problem retrieving the oui list with fetch preferred by /usr/local/arpwatch/update-ethercodes.

[2.7.2-RELEASE][root@fw.local.lan]/usr/local/arpwatch: fetch -q -o - https://standards-oui.ieee.org/oui/oui.csv
fetch: https://standards-oui.ieee.org/oui/oui.csv: Unknown HTTP error

Using wget, albeit on another machine, works. Maybe the remote site currently rejects requests with certain UAs? IIRC that was the case with pfBlocker a while back. I scp'd the file to pfsense and cat oui.csv | /usr/local/arpwatch/massagevendor > ethercodes.dat.

EDIT - just checked and it does look like default fetch UA is being blocked.

fireodo

@darcey said in Arpwatch not downloading vendor ID's:

EDIT - just checked and it does look like default fetch UA is being blocked.

Hi,

i do not use arpwatch but I also find out that ieee.org seams to block the user Agent that is provided by fetch (fetch libfetch/2.0). Using a other UA resolves the problem.

Regards,
fireodo

Gertjan

@fireodo
Nice catch.

fetch --user-agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.79 Safari/537.36' -v -o - https://standards-oui.ieee.org/oui/oui.csv

and done.

dennypage

@darcey There are several issues associated with the pfSense package module not using the update script provided by arpwatch itself:

It's using the wrong URL
The user agent (fetch) is blocked
When any form of error occurs, it overwrites the database with any empty file.

The reason behind that the pfSense package attempting its own retrieval is because it's trying to support compressed (non leading zero) MAC formats.

Benjamin 3

@dennypage thanks for the deep dive!!!

Thank you!

keyser

@dennypage Nice detective work!

Here's hoping that you can find the time to "modernize" the package so it becomes proper usable again. Not a critical package, but rather nice to have in smaller in L2 setups.