2.4 snap no default gateway

Guest

Quick note from a VM here, running the latest 2.4 snap (built on Tue Sep 27 06:22:05), it doesn't do the default route. Everything else seems to work fine, but after a reboot I do have to set the default gw from a shell. (just a simple "route add default 85.17.23.62")

jimp

The thread you replied to was on the 2.3.3 board, I split it off and moved this to the 2.4 board. The two issues are unlikely to be related as they are two very different base operating system versions.

jimp

Do you have a static IP address or DHCP?

My 2.4 test box is DHCP but it has a default gateway on the latest snapshot.

JorgeOliveira

Maybe the gateway is not set or outside subnet?
Double check your settings for inconsistencies.

JorgeOliveira

Indeed.

I've done testing on a system with correct IPv4 static configuration (working on 2.3.x) and it would no longer have default gateway on v2.4-dev.

This is a really serious issue as it makes static WAN impossible.

JorgeOliveira

Also, I'm under the impression that the following did work on FreeBSD 10.3 and pfSense 2.3.x but no longer does:

route change default 192.168.111.2

When there is no default route yet, it fails with (on 2.4-dev):

route: route has not been found
change net default: gateway 192.168.111.2 fib 0: not in table

So… The behaviour of the 'change' argument has changed from 10.3 to 11.0 and this is why this happens.

Edit:

System Logs show:

Sep 28 23:27:32 php-cgi rc.bootup: ROUTING: setting default route to 192.168.111.2
Sep 28 23:27:32 php-cgi rc.bootup: The command '/sbin/route change -inet default '192.168.111.2'' returned exit code '1', the output was 'route: route has not been found change net default: gateway 192.168.111.2 fib 0: not in table'

This seems to be reason as I highlighted previously.

Edit 2:

I'm attaching a small and untested patch (as I'm going out) that should aid in working around this issue. There still A LOT of work to be done. Good luck and hoping this is helpful. :)

Edit 3:

A much better solution has been found. See the posts below. :)

route_change_to_delete_add.txt

jimp

I believe we had a patch for FreeBSD 10.x that changed the behavior of "route change" so that it would add if it did not exist. A stock FreeBSD 10.3 produces the same error you see there, but pfSense 2.3 does not, so there must be something that needs carried forward into our FreeBSD 11 code yet.

NOYB

Don't know if this is related or not so just throwing out incase.

Same config as 2.3.3 on my 2.4 VM shows dynamic for gateway and monitor ip for OPT1 interface, and monitoring remains stuck in pending state. Whereas the 2.3.3 VM shows the gateway ip address and monitoring works fine.

Both same config. Except for a few essentials like interface IP addresses of course. But the networks are the same.

The WAN interface with default gateway is working fine though. And the OPT1 gateway exists because it's ping-able from console. So probably not related.

BBcan177

I noticed the following with 2.4 DEV, I was always getting a no route to host (try to ping 8.8.8.8 )

2.4.0-DEVELOPMENT (amd64)
built on Thu Sep 15 21:39:59 CDT 2016
FreeBSD 11.0-RC2

I found that system_gateways.php "Apply" errs when its not in the table… the only way to fix it, is to manually run the route add default command.... Then all worked as expected.

route change -inet default x.x.x.x

route: route has not been found
change net default: gateway x.x.x.x fib 0: not in table

route add default x.x.x.x

add net default: gateway x.x.x.x

JorgeOliveira

@jimp:

I believe we had a patch for FreeBSD 10.x that changed the behavior of "route change" so that it would add if it did not exist. A stock FreeBSD 10.3 produces the same error you see there, but pfSense 2.3 does not, so there must be something that needs carried forward into our FreeBSD 11 code yet.

Ah! This is it:
https://github.com/pfsense/FreeBSD-src/commit/346b289d245fd8fd2013fb5624a9287c9ae4a49d

Needs to be adapted to fit on FreeBSD 11 as code had slight changes.

Maybe something like this could be pushed there:

--- route.c
+++ route.c
@@ -1524,6 +1524,7 @@
 		print_rtmsg(&rtm, l);
 	if (debugonly)
 		return (0);
+testagain:
 	if ((rlen = write(s, (char *)&m_rtmsg, l)) < 0) {
 		switch (errno) {
 		case EPERM:
@@ -1531,6 +1532,10 @@
 			break;
 		case ESRCH:
 			warnx("route has not been found");
+			if (rtm.rtm_type == RTM_CHANGE) {
+				rtm.rtm_type = RTM_ADD;
+				goto testagain;
+			}
 			break;
 		case EEXIST:
 			/* Handled by newroute() */

Please issue new snapshots after a change on this matter is applied, so we can test. Thanks! :)

Edit:
I just rebuilt the "route" on FreeBSD 11 with this patch applied. Replaced the stock version on pfSense and it works! :D

phil.davis

Also, there is a serial number displayed (a UUID), is that new, and if so, what for, and where does it get pulled from? EFI UUID var?

That was added in this commit https://github.com/pfsense/pfsense/commit/e093cb8e75acb6950a89d45c3fcd583b0c8fba2e which uses system_get_serial() from https://github.com/pfsense/pfsense/blob/master/src/etc/inc/system.inc

It tries for the output of:

/bin/kenv smbios.system.serial

and if that does not meet a few criteria, it goes for:

get_single_sysctl('kern.hostuuid')

which gets whatever UUID FreeBSD thinks it is.

jimp

I opened https://redmine.pfsense.org/issues/6828 for the "route change" issue though it looks like we're seeking out an alternate fix. The old patch was not coded in a way that we feel comfortable continuing to include, so we're looking at correcting the behavior in other ways.

jimp

That's what we're thinking, but there are numerous calls that rely on this change behavior (over 20, last I saw), so the current thinking is we'll make a pfSense module function in C that handles the heavy lifting of doing the check and then modify/change as needed where possible, so it will be faster and easier to address if the mechanism has to change in the future.

jimp

It wouldn't do shell/exec style calls it could manipulate the routing table directly (or closer to directly).

And as for removing duplication of route logic, that's the point of adding the function to the module, so other places can use that call and not duplicate the logic.

The module is already there for other functions, this is just likely to be added to it.

JorgeOliveira

In my opinion the following workaround could be applied:
https://forum.pfsense.org/index.php?topic=118859.msg658300#msg658300

And when the improved logic on pfSense module is implemented, the workaround could be removed.
It would be helpful for those running this developement version and "move" testing to other areas as this one is still work in progress.

jimp

That proposal was shot down right away – better to fix it right the first time.

2.4 is still alpha, it's not supposed to be used in production in any capacity yet, there are still missing features. It's not a place that would (or should) require temporary solutions yet. There are other manual workarounds if someone needs connectivity in the meantime.

NOYB

@jimp:

2.4 is still alpha, it's not supposed to be used in production in any capacity yet, there are still missing features. It's not a place that would (or should) require temporary solutions yet. There are other manual workarounds if someone needs connectivity in the meantime.

I'll second that.

jimp

@johnkeates:

PHP module is the way to go. I had a quick browse on GitHub/pfSense but couldn't find it there, is the current module hosted elsewhere?

https://github.com/pfsense/FreeBSD-ports/tree/devel/devel/php56-pfSense-module/files