Getting Zerotier interface up before pfsense looks for interfaces

HoppyChris

I successfully added a zerotier interface to my pfsense box! I can join a network and it presents as an interface that you can then do all of the nice interface-y things with in pfsense. However, on reboot, the zt interface does not show up in time and then I need to assign all of my interfaces again. I've tried starting the zerotier service using a script in /usr/local/etc/rc.d and also using the shellcmd package to try to get it in and up earlier (using both shellcmd and earlyshellcmd).

While the shellcmd options seem to work (the interface comes up), the boot process also seems to hang on the startup, so even though the firewall is passing traffic and I can get to the web interface, the console is stuck.

I'm pretty sure I'm just missing something easy (or I hope I am?) – anyone gotten it up and going successfully? (or starting any other service that creates an interface?)

Chris

For those that are interested --

Go to a command line (option 8, shell)

Make sure "enabled" key is set to "yes" in /etc/pkg/FreeBSD.conf

2. Change "enabled: no" to "enabled: yes" in /usr/local/etc/pkg/repos/FreeBSD.conf

3. Change "enabled: no" to "enabled: yes" in /usr/local/etc/pkg/repos/pfSense.conf for FreeBSD repo

4. Now you're able to install packages from FreeBSD official repo

pkg add zerotier

HoppyChris

I'll add/clarify that the directions there are to get a working-until-reboot ZT interface added, so don't expect it to "work" if you just do the above – I'm looking for the (presumably very few) next steps to get the interface up and going before pfsense checks for them.

ariel

I've been trying to do the same. Right now I have another machine doing the routing to my zerotier vpn network, want to consolidate behind pfsense.

Will do some more test and hope to find something.

ariel

Hey HoppyChris,

I found a way to bring the ZT interfaces just before the configuration loading steps. it's not clean, its ugly but it works.

just edit /etc/rc.bootup and add:

mwexec("/sbin/ifconfig lo0 up");
mwexec("/usr/local/etc/rc.d/zerotier start > /dev/null");
echo "Loading Zerotier INTERFACE !!!! #####################\n";
mwexec("/bin/sleep 5");

just before the:

echo "Loading configuration...";
parse_config_bootup();
echo "done.\n";

This is my entire configuration:

// zerotier
mwexec("/sbin/ifconfig lo0 up");
mwexec("/usr/local/etc/rc.d/zerotier start > /dev/null");
echo "Loading Zerotier INTERFACE !!!! #####################\n";
mwexec("/bin/sleep 5");

echo "Loading configuration...";
parse_config_bootup();
echo "done.\n";

mwexec("/usr/sbin/gnid > {$g['vardb_path']}/uniqueid 2>/dev/null");

This is a temp solution, just waiting for the zerotier package that is being developed.

dangerusty

I had to do this for usb interfaces:

add "ue" to the list of interface types to skip in "is_interface_mismatch()" function of /etc/inc/util.inc.

Maybe do the same for zt?

FearNaBoinne

As there still is no package, and this machine I am using is stuck on older version (32 bit CPU), I was trying the above...

I've used it before, and it kinda worked (at least the zerotier interface was up)

Now the zerotier-one app is running, but zerotier-cli errors out until I restart the service manually...

And whenever I restart it (either manually or by reboot) it "forgets" the network(s) and I have to re-add it, which means the IP changes, and thus the manager route needs to change too...

I cannot see any logging of zerotier anywhere either to give me an idea of what is (not) happening...

Any help appreciated!

FearNaBoinne

Ok, figured out why, and wanted to log it for posterity (in case someone else ends up here on a search):
ZeroTier stores in /var/db/zerotier/*, and the TMP and VAR mounts were set for RAM-disk in the backup I imported...

Now I have a different issue, in that the system won't complete boot (It sits at "trying to mount root UFS blabla <UFS-ID> [rw]"), so I have something else to do (will probably end up rebuilding from scratch!) when I get back from this business trip!