-
I'm trying to configure FRR OSPF for multi-vlan and multi-site routing with BGP. The challenge I am having is with OSPF neighbors not coming up.
Topology:
+--------------+ +----------------+ | pfsense-1 | | pfsense-2 | | | | | | | | | +-------+------+ +-------+--------+ | .2 | .3 | | | | | | +-------+-----------+-------------+------+ 10.168.20.0/24 | | | .13 +------+------+ | | | ubuntu | | | +-------------+PFsense-1 config:
interface ix1
ip ospf cost 100
ip ospf priority 100
!
interface ix2.20
ip ospf cost 100
ip ospf priority 100
!
router ospf
ospf router-id 10.169.3.2
redistribute kernel route-map DNR
redistribute connected route-map DNR
redistribute bgp route-map DNR
network 10.168.20.0/24 area 0.0.0.0
network 172.17.17.0/24 area 0.0.0.0
!pfsense-1# sh ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
pfsense-1# sh ip ospf interface
ix1 is up
ifindex 2, MTU 1500 bytes, BW 10000 Mbit <UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
This interface is UNNUMBERED, Area 0.0.0.0
MTU mismatch detection: enabled
Router ID 10.169.3.2, Network Type BROADCAST, Cost: 100
Transmit Delay is 1 sec, State DR, Priority 100
No backup designated router on this network
Multicast group memberships: OSPFAllRouters OSPFDesignatedRouters
Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
Hello due in 5.102s
Neighbor Count is 0, Adjacent neighbor count is 0
Internet Address 172.17.17.2/24, Broadcast 172.17.17.255, Area 0.0.0.0
MTU mismatch detection: enabled
Router ID 10.169.3.2, Network Type BROADCAST, Cost: 100
Transmit Delay is 1 sec, State DR, Priority 100
No backup designated router on this network
Multicast group memberships: OSPFAllRouters OSPFDesignatedRouters
Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
Hello due in 5.102s
Neighbor Count is 0, Adjacent neighbor count is 0
ix2.20 is up
ifindex 9, MTU 1500 bytes, BW 10000 Mbit <UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
This interface is UNNUMBERED, Area 0.0.0.0
MTU mismatch detection: enabled
Router ID 10.169.3.2, Network Type BROADCAST, Cost: 100
Transmit Delay is 1 sec, State DR, Priority 100
No backup designated router on this network
Multicast group memberships: OSPFAllRouters OSPFDesignatedRouters
Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
Hello due in 5.102s
Neighbor Count is 0, Adjacent neighbor count is 0
Internet Address 10.168.20.2/24, Broadcast 10.168.20.255, Area 0.0.0.0
MTU mismatch detection: enabled
Router ID 10.169.3.2, Network Type BROADCAST, Cost: 100
Transmit Delay is 1 sec, State DR, Priority 100
No backup designated router on this network
Multicast group memberships: OSPFAllRouters OSPFDesignatedRouters
Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
Hello due in 5.102s
Neighbor Count is 0, Adjacent neighbor count is 0What I find weird int he output for interfaces that there is a unnumbered interface for each.
Now I configured also FRR on a Ubuntu server for load balancing (2nd pfsense) and routing, the neighbors show, but neighbors are not coming up.
Ubuntu FRR:
!
router ospf
ospf router-id 10.169.3.13
passive-interface eth0
passive-interface eth1
passive-interface eth3
network 10.168.20.0/24 area 0.0.0.0
network 10.168.120.0/24 area 0.0.0.0
network 10.169.3.13/32 area 0.0.0.0
network 172.17.17.0/24 area 0.0.0.0
neighbor 10.168.20.2 priority 50
neighbor 10.168.20.3 priority 100
neighbor 172.17.17.2 priority 255
!ubuntu# show ip ospf interface
...
eth4 is up
ifindex 151, MTU 1500 bytes, BW 10000 Mbit <UP,BROADCAST,RUNNING,MULTICAST>
Internet Address 10.168.20.13/24, Area 0.0.0.0
MTU mismatch detection: enabled
Router ID 10.169.3.13, Network Type BROADCAST, Cost: 10
Transmit Delay is 1 sec, State DR, Priority 1
No backup designated router on this network
Multicast group memberships: OSPFAllRouters OSPFDesignatedRouters
Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
Hello due in 6.547s
Neighbor Count is 2, Adjacent neighbor count is 0
...
ubuntu# sh ip ospf neighborNeighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
10.169.3.2 100 Init/DROther 34.503s 10.168.20.2 eth4:10.168.20.13 0 0 0
10.169.3.3 1 Init/DROther 34.672s 10.168.20.3 eth4:10.168.20.13 0 0 0The ubuntu FRR gets the broadcast from both PFsense boxes.
Tcpdump on the pfsense-1 box:
12:31:19.578987 IP 10.168.20.2 > 10.168.20.3: ICMP echo reply, id 4168, seq 16209, length 8
12:31:20.120137 IP 10.168.20.3 > 10.168.20.2: ICMP echo request, id 4168, seq 16210, length 8
12:31:20.120151 IP 10.168.20.2 > 10.168.20.3: ICMP echo reply, id 4168, seq 16210, length 8
12:31:20.381871 IP 10.168.20.13 > 224.0.0.5: OSPFv2, Hello, length 52
12:31:20.382066 IP 10.168.20.2 > 224.0.0.5: OSPFv2, Hello, length 44
12:31:20.382084 IP 10.168.20.1 > 224.0.0.5: OSPFv2, Hello, length 44
12:31:20.387629 IP 10.168.20.13 > 10.168.20.2: ICMP echo request, id 31330, seq 1, length 64
12:31:20.387665 IP 10.168.20.2 > 10.168.20.13: ICMP echo reply, id 31330, seq 1, length 64
12:31:20.638787 IP 10.168.20.3 > 10.168.20.2: ICMP echo request, id 4168, seq 16211, length 8
12:31:20.638802 IP 10.168.20.2 > 10.168.20.3: ICMP echo reply, id 4168, seq 16211, length 8
12:31:20.638816 IP 10.168.20.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 1, authtype none, intvl 1s, length 36
12:31:20.839757 IP 10.168.20.3 > 224.0.0.5: OSPFv2, Hello, length 44
12:31:20.839777 IP 10.168.20.1 > 224.0.0.5: OSPFv2, Hello, length 44
12:31:21.178842 IP 10.168.20.3 > 10.168.20.2: ICMP echo request, id 4168, seq 16212, length 8This interface has one rule to allow both IP4 and IP6 on any any, I've also added the OSPF rule with any any, however to no success for OSPF establishing neighbors. Both pfsense boxes cannot see each other.
There is no issue when I add a second ubuntu box on the same vlan, their adjacencies come right up.
OS versions:
pfsense: 2.4.4-RELEASE
pfsense: frr: 0.2_8 (frr5-5.0.2)Ubuntu FRR: 7.0-1
-
@rfc4711 said in OSPF neighbors not coming up:
OS versions:
pfsense: 2.4.4-RELEASE
pfsense: frr: 0.2_8 (frr5-5.0.2)Installed OPNsense on another VM and OSPF sessions to the ubuntu VM came right up.
This confirms that either FRR ospf on this version is broken or pfsense is blocking the advertisements from being received (even thought each interface was set to IP4 any any) - I suspect the latter.
Hope this post can help someone to prevent wasting their time, I've been pulling our my hair for the past days on this.
-
I have exact the same issue but then on the latest OPNsense 19.1.6-amd64
Which version did you try ? When I turn off pfctl -d then my traffic start floating, look here:
https://forum.opnsense.org/index.php?topic=12413.0
-
OPNsense 19.1.6-amd64
FreeBSD 11.2-RELEASE-p9-HBSD
OpenSSL 1.0.2r 26 Feb 2019I did more work yesterday and got ospf now working. Initially the opnsense fw connected right out from the initial install to my ubuntu frr ospf on the LAN interface. However after a few hours of configuration and setting up rules and interfaces, I noticed OSPF to be down.
by tracing back my steps, on my setup the CARP interface was the issue, apparently you cannot have a virtual carp interface for redundancy and ospf on the same interface. I noticed the "ununumbered" interface on interfaces having CARP enabled:
"This interface is UNNUMBERED, Area 0.0.0.0,No Hellos (Passive interface),No Hellos (Passive interface)"
As soon as I deleted the CARP config, OSPF came up. My solution was to add a new VLAN interface between the firewalls and all servers in need to custom gateways and run OSPF for routing sync there.
last but not least, the OSPF config seems to be very picky, make sure all interfaces are set to broadcast on ethernet connections. In my setup I got a pfsense firewall, one opnsense firewall and several ubuntu VMs connected.