[SOLVED] Still problems with pfsense CARP trigger

DeFix

A slightly better fix for this is possible now with 1.2.3-RC and 2.0, but it's still not ideal.

The CARP interface will now report a transition to MASTER as a "link up" event, and a transition to BACKUP as a "link down" event to the system. These can be caught with devd and used to call scripts on these events – no more need to rely on cron or a delay. This will happen instantaneously once the CARP interface on the backup takes over.

This is more meant for a full install, but I suppose it could be altered to work as the initial solution was for a livecd/embedded platform.

If you are running a recent (as of the date on this post) snapshot of 1.2.3, or 2.0, you can try this.

Edit /etc/devd.conf, and add the following:
Code:
notify 100 {
match "system" "IFNET";
match "type" "LINK_UP";
match "subsystem" "carp";
action "/usr/local/bin/carpup $subsystem";
};
notify 100 {
match "system" "IFNET";
match "type" "LINK_DOWN";
match "subsystem" "carp";
action "/usr/local/bin/carpdown $subsystem";
};

In this instance, you don't really need the $subsystem variable, but it may be useful if you want to perform other actions. It contains the name of the actual carp interface that transitioned. If you want to lock this down to just one carp interface, you could change the subsystem match to "carp0" or "carp1", whichever you like.

Restart devd (or reboot):
Code:
killall -9 devd && /sbin/devd

You can then create the scripts mentioned on the "action" line above. For this case, it would be two different scripts:

/usr/local/bin/carpup
Code:
#!/bin/sh
/sbin/ifconfig bridge0 up

/usr/local/bin/carpdown
Code:
#!/bin/sh
/sbin/ifconfig bridge0 down

Finally, make sure those are executable:
Code:
chmod a+x /usr/local/bin/carpup
chmod a+x /usr/local/bin/carpdown

You could add anything else that you want to these scripts. Calling some sort of notification program would be useful, or whatever else is desired.

I'm trying to come up with some sort of generic detection code that would take the carp interface, and attempt to see if its parent interface is a bridge member, and if so, bring down that bridge

Quoted from the post of Getting pfsense to failover with a bridge using the CD-ROM platform

I did what you said on a other topic (which i could'nt respond in anymore) but still not working for me.

It looks like the event will not trigger my scripts.

Can you tell me why?

Running on: 2.0.1-RELEASE (amd64)

My devd.conf looks like:

$Id$

$FreeBSD: src/etc/devd.conf,v 1.26.2.1 2005/09/03 22:49:22 sam Exp $

options {
directory "/etc/devd";
directory "/usr/local/etc/devd";
pid-file "/var/run/devd.pid";
set scsi-controller-regex
"(aac|adv|adw|aha|ahb|ahc|ahd|aic|amd|amr|asr|bt|ciss|ct|dpt|
esp|ida|iir|ips|isp|mlx|mly|mpt|ncr|ncv|nsp|stg|sym|trm|wds)
[0-9]+";
};

CARP notify hooks. This will call carpup/carpdown with the

interface (carp0, carp1) as the first parameter.

notify 100 {
match "system" "IFNET";
match "type" "LINK_UP";
match "subsystem" "carp";
action "/etc/rc.carpmaster $subsystem";
};

notify 100 {
match "system" "IFNET";
match "type" "LINK_DOWN";
match "subsystem" "carp";
action "/etc/rc.carpbackup $subsystem";
};

When a USB keyboard arrives, attach it as the console keyboard.

attach 100 {
device-name "ukbd0";
action "kbdcontrol -k /dev/ukbd0 < /dev/console 2>/dev/null";
};

detach 100 {
device-name "ukbd0";
action "kbdcontrol -k /dev/kbd0 < /dev/console 2>/dev/null";
};

Signal upper levels that an event happened on ethernet class interface

notify 0 {
match "system" "IFNET";
match "type" "LINK_UP";
media-type "ethernet";
action "/usr/local/sbin/pfSctl -c 'interface linkup start $subsystem'";
};

notify 0 {
match "system" "IFNET";
match "type" "LINK_DOWN";
media-type "ethernet";
action "/usr/local/sbin/pfSctl -c 'interface linkup stop $subsystem'";
};

Notify all users before beginning emergency shutdown when we get

a _CRT or _HOT thermal event and we're going to power down the system

very soon.

notify 10 {
match "system" "ACPI";
match "subsystem" "Thermal";
match "notify" "0xcc";
action "logger -p kern.emerg 'WARNING: system temperature too high, shutting down soon!'";
};

Added by ME, to manage the bridge0 port.

if link_down the bridge0 port goes up, else it goes down.

notify 100 {
match "system" "IFNET";
match "type" "LINK_UP";
match "subsystem" "carp";
action "/usr/local/bin/carpup $subsystem";
};
notify 100 {
match "system" "IFNET";
match "type" "LINK_DOWN";
match "subsystem" "carp";
action "/usr/local/bin/carpdown $subsystem";
};

jimp

Probably because there is already an entry to match those up higher in the file you edited. Just use the scripts already quoted in there rather than adding your own.

Also make sure to reboot or restart devd after editing that file.

DeFix

@jimp:

Probably because there is already an entry to match those up higher in the file you edited. Just use the scripts already quoted in there rather than adding your own.

Ok i'll replace that. But will the pfsense still change from master to backup in CARP after that adjustment?

@jimp:

Also make sure to reboot or restart devd after editing that file.

I did that indeed.

jimp

Sure, the devd hooks have nothing to do with the actual master/backup transition. Those were just added in case any user actions needed to happen during the transition.

Currently all those etc/rc.carpbackup and etc/rc.carpmaster scripts do is issue notifications that the transition happened. Just add your own actions at the end of those files.

DeFix

@jimp:

Sure, the devd hooks have nothing to do with the actual master/backup transition. Those were just added in case any user actions needed to happen during the transition.

Currently all those etc/rc.carpbackup and etc/rc.carpmaster scripts do is issue notifications that the transition happened. Just add your own actions at the end of those files.

I have now

CARP notify hooks. This will call carpup/carpdown with the

interface (carp0, carp1) as the first parameter.

notify 100 {
match "system" "IFNET";
match "type" "LINK_UP";
match "subsystem" "carp";
action "/usr/local/bin/carpup $subsystem";
};

notify 100 {
match "system" "IFNET";
match "type" "LINK_DOWN";
match "subsystem" "carp";
action "/usr/local/bin/carpdown $subsystem";
};

But still when my MASTER is going offline and my other pfsense-server become MASTER (from BACKUP) it doesn't trigger my "carpup"-file.

Do i see something wrong here?

jimp

Does the carpup and carpdown script run if you run it by hand?

It could be a problem with the script (wrong interpreter path, no exec bit, etc)

DeFix

@jimp:

Does the carpup and carpdown script run if you run it by hand?

It could be a problem with the script (wrong interpreter path, no exec bit, etc)

Yes it works when i run it by hand.

Text in carpup

#!/bin/sh
/sbin/ifconfig bridge0 up
logger -t $1 "Set the bridge port to [UP]"

The rights of the file's

[2.0.1-RELEASE][root@fw-tw-main.itnm]/usr/local/bin(4): ls -al | grep carp
-rwxr-xr-x 1 root wheel 83 Feb 7 12:26 carpdown
-rwxr-xr-x 1 root wheel 80 Feb 7 12:27 carpup

Must i first enable something in pfsense to activate the triggering or so?

jimp

No, if devd is running they should get triggered.

Try using the full path to the logger command, too. I'm not sure it would have a PATH set when run from devd.

DeFix

@jimp:

No, if devd is running they should get triggered.

Try using the full path to the logger command, too. I'm not sure it would have a PATH set when run from devd.

Yes devd is running.

Also tried to add the full path to the logger. (also had it fully disabled) but no changes.

Seems like there is no trigger at all. Can you tell me how i can trace that there is a trigger fired at all?

jimp

Not sure off the top of my head. Look at FreeBSD's man page for devd/devd.conf and you'll probably find more answers there.

DeFix

@jimp:

Not sure off the top of my head. Look at FreeBSD's man page for devd/devd.conf and you'll probably find more answers there.

Hold on!! i have the solution!! ;D

Solution:
In the devd.conf file is the subsystem carp. That is the name of the Virtual IP for CARP. On our system the name is vip1.

So it won't match with the name carp.

With that knowledge i changed my devd.conf to:

CARP notify hooks. This will call carpup/carpdown with the

interface (carp0, carp1) as the first parameter.

notify 100 {
match "system" "IFNET";
match "type" "LINK_UP";
match "subsystem" "vip1";
action "/usr/local/bin/carpup $subsystem";
};

notify 100 {
match "system" "IFNET";
match "type" "LINK_DOWN";
match "subsystem" "vip1";
action "/usr/local/bin/carpdown $subsystem";
};

So i changed the carp to vip1. This is now a match for the trigger and it was the solution to our problem.

Thanks for the good support and thinking.

Supermule

But is it implemented by default in PFSense that way? Or can it be changed via the webgui so the trigger wont happen ?

DeFix

@Supermule:

But is it implemented by default in PFSense that way? Or can it be changed via the webgui so the trigger wont happen ?

For me it was a default name of PFSense. Did search to rename it to carp (after i knew what the problem was), but did not found it in the WebGUI.

jimp

Ah, the 'carp' bit was probably left over from 1.2.3 and not updated. If you just use "vip" it may work also.

That file isn't written from the GUI, it's just there on the install. It would be overwritten during an upgrade, but it's left alone otherwise.

DeFix

@jimp:

Ah, the 'carp' bit was probably left over from 1.2.3 and not updated. If you just use "vip" it may work also.

That file isn't written from the GUI, it's just there on the install. It would be overwritten during an upgrade, but it's left alone otherwise.

That explains a lot.

I use explecit vip1 because we also have a vip2 and that may not trigger the bridge port to UP or DOWN.