FRR/OSPF won't distribute static routes from FRR/Zebra

JeGr · May 23, 2019, 10:30 AM

Hi,

we set up FRR/OSPF between two sites. Both sites are connected via an MPLS link and have a static-key-OVPN tunnel (as suggested for FRR) over their WAN link as a failback.

As FRR pushed not only routes from the static routes section from pfSense GUI but also Host /32 routes configured by pfSense itself (e.g. OVPN peer, DNS via WAN1/2 etc.) and as denying those was ignored, we disabled Kernel routing table pushing and only selected "FRR static routes" which - as @jimp showed in the hangout - should push routes configured under FRR/zebra. But although we put those routes in there, the aren't pushed to the OSPF peers.

Any thoughts as to why?

Greets Jens

chrismacmahon · May 23, 2019, 1:23 PM

What status are the other routers showing in status -> frr - OSPF Neighbors

JeGr · May 23, 2019, 1:43 PM

@chrismacmahon

One side is

full/DR
full/DRother

the other side is

full/Backup
full/DRother

DR/Backup state is on the MPLS link which is OK, DRother is on the ovpns/c interfaces.

jimp · May 23, 2019, 4:57 PM

Can you share the contents of /var/etc/frr/ospfd.conf and /var/etc/frr/zebra.conf? I haven't tried doing that specifically in a while, but it worked last time I did. You do have to enable redistribution of static routes in addition to making the routes in Zebra.

JeGr · May 24, 2019, 7:46 AM

@jimp said in FRR/OSPF won't distribute static routes from FRR/Zebra:

I haven't tried doing that specifically in a while, but it worked last time I did.

Me too, that's why I'm completely baffled it won't propagate.

On another thought: Do we have to remove the static routes from "System / Routing" for the static routes of FRR/Zebra to work (so to not define them two times in different locations)?

You do have to enable redistribution of static routes in addition to making the routes in Zebra

Aye, that's what I've done:

Routes set up in FRR/Zebra/General config

Routes from Zebra to propagate as per your Hangout (and my previous setup test) in FRR/OSPF settings

Really baffled.

JeGr · May 24, 2019, 7:50 AM

@jimp

That's from the remote side (e.g. the screenshot above with 192.168.10.0/24 is one of those local nets to push to the other side)

##################### DO NOT EDIT THIS FILE! ######################
###################################################################
# This file was created by an automatic configuration generator.  #
# The contents of this file will be overwritten without warning!  #
###################################################################
password xxx
log syslog

# Null Routes

# Static Routes
ip route 192.168.10.0/24 10.99.9.49
ip route 172.17.41.0/24 10.99.9.49
ip route 172.17.47.0/24 10.99.9.49
ip route 172.17.61.0/24 10.99.9.49
ip route 192.168.71.0/24 10.99.9.49

# Accept Filters
ip prefix-list ACCEPTFILTER deny 10.99.9.16/28
ip prefix-list ACCEPTFILTER deny 10.99.9.19/32
ip prefix-list ACCEPTFILTER deny 10.20.20.0/30
ip prefix-list ACCEPTFILTER deny 10.20.20.2/32
ip prefix-list ACCEPTFILTER deny 172.30.31.0/24
ip prefix-list ACCEPTFILTER deny 172.30.31.201/32
ip prefix-list ACCEPTFILTER deny 10.99.9.32/28
ip prefix-list ACCEPTFILTER deny 10.99.9.33/32
ip prefix-list ACCEPTFILTER deny 10.99.9.48/28
ip prefix-list ACCEPTFILTER deny 10.99.9.50/32
ip prefix-list ACCEPTFILTER permit any
route-map ACCEPTFILTER permit 10
 match ip address prefix-list ACCEPTFILTER
ip protocol ospf route-map ACCEPTFILTER

##################### DO NOT EDIT THIS FILE! ######################
###################################################################
# This file was created by an automatic configuration generator.  #
# The contents of this file will be overwritten without warning!  #
###################################################################
password xxx
log syslog
interface igb3
  ip ospf cost 10
interface ovpnc1
  ip ospf cost 100
interface igb1
interface igb0
interface igb2

router ospf
  ospf router-id 10.99.9.19
  redistribute static route-map DNR
  passive-interface igb1
  passive-interface igb0
  passive-interface igb2
  network 10.99.9.16/28 area 0.0.0.0
  network 10.20.20.0/30 area 0.0.0.0
  network 172.30.31.0/24 area 0.0.0.0
  network 10.99.9.32/28 area 0.0.0.0
  network 10.99.9.48/28 area 0.0.0.0
access-list dnr-list permit any
route-map DNR permit 10
  match ip address dnr-list

jimp · May 24, 2019, 3:33 PM

Is this on 2.5.0 or 2.4.4-p3? I found an issue on 2.5.0 that would prevent static routes in Zebra from working, but I don't have a 2.4.4-p3 setup handy to check right now (and that bug wouldn't be possible there, as it would only affect FRR 6+).

That said, once I got the FRR static route to show in the table, it happily propagated across OSPF to the neighbor.

If you go into vtysh do your static routes show up in show ip route? If so, do they have K> or S>?

If you already have static routes in System > Routing you could just use the option to redistribute kernel routes instead of static routes.

JeGr · May 27, 2019, 8:08 AM

@jimp said in FRR/OSPF won't distribute static routes from FRR/Zebra:

Is this on 2.5.0 or 2.4.4-p3? I found an issue on 2.5.0 that would prevent static routes in Zebra from working, but I don't have a 2.4.4-p3 setup handy to check right now (and that bug wouldn't be possible there, as it would only affect FRR 6+).

Nope, that ist a 2.4.4-p2 stable/production setup. No dev-version anywhere near it ;)

If you already have static routes in System > Routing you could just use the option to redistribute kernel routes instead of static routes.

That was my first intention, but that also triggered distribution of IPs like the WAN IP of the peer as they have an OVPN tunnel between them and that /32 IP is then distributed to the other side (as it's a static host route). Filtering out that entry didn't work either, so I thought about just dropping the hassle with all the unnecessary host routes from the kernel table and instead just configure the 4-5 static routes and be done.

If you go into vtysh do your static routes show up in show ip route? If so, do they have K> or S>?

Which one on which side? I assume you mean the static routes on the same side. Yes they do show.

S 172.16.45.0/24 [1/0] via 10.99.9.6, igb11, 3d18h20m
K>* 172.16.45.0/24 [0/0] via 10.99.9.6, igb11, 3d18h20m

ATM they show up as both S and K. As kernel routes they get pushed (but as described above, a whole lot more, too that I don't want and filtering doesn't seem to work either), as statics they don't :/

All 5 routes defined in the FRR/Zebra/General section show up as "S" routes.

jimp · May 28, 2019, 5:17 PM

I'd use one method or the other then.

Either

Take the routes out of System > Routing and only put them in Zebra, then redistribute static
Leave the routes in System > Routing and remove them from Zebra, then redistribute kernel. If there are routes you don't want to redistribute then in the OSPF main settings add them to the networks list at the bottom with Disable Redistribution checked. You have to match the route exactly, so be sure that it matches what is shown in the table.

JeGr · May 29, 2019, 9:37 AM

@jimp said in FRR/OSPF won't distribute static routes from FRR/Zebra:

Take the routes out of System > Routing and only put them in Zebra, then redistribute static

That's what I did, but they won't distribute to the other side. So I had to re-enter them for the current routing to work before I get FRR to work properly.

Leave the routes in System > Routing and remove them from Zebra, then redistribute kernel. If there are routes you don't want to redistribute then in the OSPF main settings add them to the networks list at the bottom with Disable Redistribution checked. You have to match the route exactly, so be sure that it matches what is shown in the table.

Did that first, too. Problem was:

Static Host Routes from e.g. OpenVPN peer is getting pushed to the other side - which is bad, as it is one of the WAN VIPs there. So I did exactly that - added it with /32 to the OSPF main settings. But it didn't get "deleted" from the pushed kernel routing table.
There were conflicts as both sides use e.g. 1.1.1.1 as Host Route for checking a specific WAN gateway so both have them as static host route. Entering it into OSPF main settings as disabled didn't delete it from the published routes either.

So either I'm missing some switch I never had to play with before, I'm kinda bummed as to where to go now. Kernel routes would be easiest as static routes are "where they are suspected" but the removal of unnecessary routes is not working correctly, even if I try to match the route exactly it will be published nonetheless.

jimp · May 29, 2019, 1:10 PM

Curious. In my lab the only OSPF pair I have setup uses a zebra static route to distribute a route to neighbors and it's working fine there. Nothing too special, it's just in the zebra static routes (not in System > Routing), and I have Redistribute Static set.

JeGr · May 29, 2019, 3:04 PM

As do I in the screens above. Yet nothing happens on the remote site. It's completely bonkers...

nzkiwi68 · Jun 4, 2019, 10:58 PM

I found if the route is for an interface in pfSense that has a CARP address, then, the route will NOT get distributed.

Bug!!!!

JeGr · Jun 5, 2019, 10:16 AM

Hmm in our case one side is a CARP cluster, but the other side (who also doesn't publish) is a single box and that should work, too. Would only make sense (about a possible bug) that it's enough if one side is a CARP cluster, that both sides won't publish correctly.

Anything you could think of @jimp that could be related to that?

jimp · Jun 5, 2019, 1:00 PM

Nothing special about CARP VIPs or them being on interfaces, not that I know of anyhow.

We updated the FRR package for 2.4.4-p3 to FRR 6 the other day, you might try it again with that to see if there is a difference. Static route handling moved to a new daemon, staticd.

And 2.5.0 snapshots now have FRR 7 if you want to try something even newer.

JeGr · Jun 5, 2019, 3:18 PM

@jimp said in FRR/OSPF won't distribute static routes from FRR/Zebra:

And 2.5.0 snapshots now have FRR 7 if you want to try something even newer.

Would like to, but that's a production customer system - no expermiental there ;)

We updated the FRR package for 2.4.4-p3 to FRR 6 the other day, you might try it again with that to see if there is a difference. Static route handling moved to a new daemon, staticd.

Will definetly do and test again, perhaps nearer to Friday evening to see if I can make it work.

smaxwell2 · Feb 20, 2020, 7:24 PM

I've got exactly the same issue here. Got 4 x Static Routes that are programmed into FRR > Global Settings. And no matter what I do, they will not redistribute to OSPF Peers. Using pfSense 2.4.4-RELEASE-p3 which I believe it the latest version. I have "Redistribute FRR static routes ticked" with no Metric or Metric Type Entered. Any pointers on how to resolve this ? Bit of a show stopper for me currently

nzkiwi68 · Feb 20, 2020, 10:49 PM

I had all sorts of issues too, but, my final solution was to distribute kernel routes but use a distribute list to then only send the exact kernel routes I want.

That way, I can have the routes set in pfSense as static routes too.

See my post here;
https://forum.netgate.com/topic/145252/osfp-distributing-routes-just-using-access-lists-not-bothering-with-interfaces-expect-the-vti-ones

Gonzalo Andrade · Sep 15, 2023, 11:52 PM

I hope this is not an issue any more for you @JeGr. I just want to contribute and/or document a minimalistic solution to the original problem: several undesired /32 networks redistributing from the kernel to the routing protocol.
This solution prevent the propagation of any routes to hosts or /32 prefixes or 255.255.255.255 netmasks defined within the GUI, and some routes created automatically by PFSense, such as the Gateway monitoring addresses, VPN remote gateways, etc.

Create a prefix list: permit 0.0.0.0/0 and maximum prefix of 31.
Create a route map: just permit the previous prefix list.
Apply that route map to the kernel redistribution in the routing protocol settings(OSPF in my case).

After making all the changes in the pfsense gui (not in the raw config) the "Running frr.conf" in the Raw config tab should look like this:

router ospf
!..
!..
 redistribute kernel route-map deny_host_routes
!..
!..

ip prefix-list deny_host_routes seq 10 permit 0.0.0.0/0 le 31 
!
route-map deny_host_routes permit 10
 match ip address prefix-list deny_host_routes
!