Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    IPSec + GRE + OSPF (FRR) flushing

    FRR
    1
    2
    72
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      Leksandr last edited by

      Hello to all,

      while I was trying out net network protocols and topologies, I run into a problem when if I enabled OSPF on GRE tunnel interfaces, it (GRE tunnel) almost stop working.

      Let me provide a little more information.

      alt text
      Schema of my current topology. Please do not mind a simple setup on Cloud B, it is still a work in progress

      It works perfectly if I set the static route from Cloud A to Cloub B via GRE + IPSec tunnel, but if I enable OSPF on both sides of the GRE tunnel, it (GRE tunnel) start to flush.

      Something similar described here: https://github.com/FRRouting/frr/issues/7213

      OSPFd logs below

      Mar 11 14:15:05	ospfd	17111	nsm_change_state:[192.168.240.6:default], Loading -> Full): scheduling new router-LSA origination
Mar 11 14:15:05	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Loading -> Full (LoadingDone)
Mar 11 14:15:05	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Exchange -> Loading (ExchangeDone)
Mar 11 14:15:05	ospfd	17111	default:Packet[DD]: 192.168.240.6 DB Desc send with seqnum:18fc1ae4 , flags:0
Mar 11 14:15:05	ospfd	17111	default:Packet[DD]: Neighbor 192.168.240.6 state is Exchange, seq_num:0x18fc1ae4, local:0x18fc1ae3
Mar 11 14:15:05	ospfd	17111	default:Packet[DD]: 192.168.240.6 DB Desc send with seqnum:18fc1ae3 , flags:0
Mar 11 14:15:05	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: ExStart -> Exchange (NegotiationDone)
Mar 11 14:15:05	ospfd	17111	Packet[DD]: Neighbor 192.168.240.6 Negotiation done (Slave).
Mar 11 14:15:05	ospfd	17111	default:Packet[DD]: Neighbor 192.168.240.6 state is ExStart, seq_num:0x18fc1ae3, local:0x2996eba0
Mar 11 14:15:05	ospfd	17111	default:Packet[DD]: 192.168.240.6 DB Desc send with seqnum:2996eba0 , flags:7
Mar 11 14:15:05	ospfd	17111	default: Initializing [DD]: 192.168.240.6 with seqnum:2996eba0 , flags:7
Mar 11 14:15:05	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Init -> ExStart (2-WayReceived)
Mar 11 14:15:05	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Down -> Init (PacketReceived)
Mar 11 14:14:50	ospfd	17111	[EC 134217736] ospf_sr_local_block_release_label: Returning label 0 is outside SRLB [15000/15999]
Mar 11 14:14:50	ospfd	17111	[EC 134217736] ospf_sr_local_block_release_label: Returning label 0 is outside SRLB [15000/15999]
Mar 11 14:14:50	ospfd	17111	nsm_change_state:[192.168.240.6:default], Full -> Deleted): scheduling new router-LSA origination
Mar 11 14:14:50	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Full -> Deleted (InactivityTimer)
Mar 11 14:14:10	ospfd	17111	nsm_change_state:[192.168.240.6:default], Loading -> Full): scheduling new router-LSA origination
Mar 11 14:14:10	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Loading -> Full (LoadingDone)
Mar 11 14:14:10	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Exchange -> Loading (ExchangeDone)
Mar 11 14:14:10	ospfd	17111	default:Packet[DD]: 192.168.240.6 DB Desc send with seqnum:18fc1ae1 , flags:0
Mar 11 14:14:10	ospfd	17111	default:Packet[DD]: Neighbor 192.168.240.6 state is Exchange, seq_num:0x18fc1ae1, local:0x18fc1ae0
Mar 11 14:14:05	ospfd	17111	[EC 134217740] interface gre0:10.0.148.2: ospf_check_md5 bad sequence 1678536847 (expect 1678536848)
Mar 11 14:14:05	ospfd	17111	default:Packet[DD]: 192.168.240.6 DB Desc send with seqnum:18fc1ae0 , flags:0
Mar 11 14:14:05	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: ExStart -> Exchange (NegotiationDone)
Mar 11 14:14:05	ospfd	17111	Packet[DD]: Neighbor 192.168.240.6 Negotiation done (Slave).
Mar 11 14:14:05	ospfd	17111	default:Packet[DD]: Neighbor 192.168.240.6 state is ExStart, seq_num:0x18fc1ae0, local:0x2871b6be
Mar 11 14:14:05	ospfd	17111	default:Packet[DD]: 192.168.240.6 DB Desc send with seqnum:2871b6be , flags:7
Mar 11 14:14:05	ospfd	17111	default: Initializing [DD]: 192.168.240.6 with seqnum:2871b6be , flags:7
Mar 11 14:14:05	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Init -> ExStart (2-WayReceived)
Mar 11 14:14:05	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Down -> Init (PacketReceived)
Mar 11 14:13:54	ospfd	17111	[EC 134217736] ospf_sr_local_block_release_label: Returning label 0 is outside SRLB [15000/15999]
Mar 11 14:13:54	ospfd	17111	[EC 134217736] ospf_sr_local_block_release_label: Returning label 0 is outside SRLB [15000/15999]
Mar 11 14:13:54	ospfd	17111	nsm_change_state:[192.168.240.6:default], Full -> Deleted): scheduling new router-LSA origination
Mar 11 14:13:54	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Full -> Deleted (InactivityTimer)
Mar 11 14:13:13	ospfd	17111	nsm_change_state:[192.168.240.6:default], Loading -> Full): scheduling new router-LSA origination
Mar 11 14:13:13	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Loading -> Full (LoadingDone)
Mar 11 14:13:13	ospfd	17111	AdjChg: Nbr 192.168.240.6(default) on gre0:10.0.148.2: Exchange -> Loading (ExchangeDone)
Mar 11 14:13:13	ospfd	17111	default:Packet[DD]: 192.168.240.6 DB Desc send with seqnum:18fc1ade , flags:0
Mar 11 14:13:13	ospfd	17111	default:Packet[DD]: Neighbor 192.168.240.6 state is Exchange, seq_num:0x18fc1ade, local:0x18fc1add

      (192.168.240.6 - ID of VPN router in Cloud A, 10.0.148.2 - ID of VPN router in Cloud B)

      OSPF traffic dump from one side of GRE tunnel

      14:13:38.632299 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:13:48.684087 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:13:58.694570 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 44
14:14:05.294373 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Hello, length 48
14:14:05.294532 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 32
14:14:05.336657 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Database Description, length 32
14:14:05.336864 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 432
14:14:05.379283 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Request, length 36
14:14:05.379291 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Database Description, length 52
14:14:05.379452 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Update, length 52
14:14:05.474227 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:14:08.694587 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:14:10.395615 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Database Description, length 52
14:14:10.395803 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 32
14:14:10.395831 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Request, length 36
14:14:10.438111 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Update, length 88
14:14:10.438235 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Update, length 64
14:14:10.543184 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:14:10.694486 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:14:18.750928 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:14:28.770704 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:14:38.775488 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:14:48.794628 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:14:58.829778 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 44
14:15:05.312770 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Hello, length 48
14:15:05.313073 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 32
14:15:05.355310 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Database Description, length 32
14:15:05.355478 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 432
14:15:05.397477 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Database Description, length 52
14:15:05.397483 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Request, length 36
14:15:05.397620 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 32
14:15:05.397647 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Request, length 36
14:15:05.397659 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Update, length 52
14:15:05.439719 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Update, length 88
14:15:05.439878 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Update, length 64
14:15:05.694601 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:15:06.162408 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:15:08.832008 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:15:15.312957 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Hello, length 48
14:15:15.394638 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Update, length 64
14:15:15.406741 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Update, length 64
14:15:16.036027 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:15:16.416887 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:15:18.844889 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:15:28.876693 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:15:38.881933 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:15:48.894654 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:15:58.923775 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 44
14:16:05.324999 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Hello, length 48
14:16:05.325313 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 32
14:16:05.367551 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Database Description, length 32
14:16:05.367827 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 432
14:16:05.410119 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Request, length 36
14:16:05.410127 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Database Description, length 52
14:16:05.410247 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Update, length 52
14:16:05.862689 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:16:08.965993 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:16:10.398449 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Database Description, length 52
14:16:10.398587 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 32
14:16:10.398616 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Request, length 36
14:16:10.440925 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Update, length 88
14:16:10.441049 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Update, length 64
14:16:11.038724 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:16:11.072448 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:16:18.994440 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:16:29.020137 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:16:39.076450 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:16:49.093976 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:16:59.103866 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 44
14:17:05.341079 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Hello, length 48
14:17:05.341376 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 32
14:17:05.383575 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Database Description, length 32
14:17:05.383863 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 432
14:17:05.425956 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Database Description, length 52
14:17:05.425961 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Request, length 36
14:17:05.426105 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Database Description, length 32
14:17:05.426134 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Request, length 36
14:17:05.426145 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Update, length 52
14:17:05.468458 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Update, length 88
14:17:05.468660 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Update, length 64
14:17:05.880426 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:17:06.154150 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:17:09.137077 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:17:15.363598 IP 10.0.148.1 > 224.0.0.5: OSPFv2, Hello, length 48
14:17:15.404450 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Update, length 64
14:17:15.463821 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Update, length 64
14:17:16.156561 IP 10.0.148.1 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:17:16.392751 IP 10.0.148.2 > 224.0.0.5: OSPFv2, LS-Ack, length 44
14:17:19.190030 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:17:29.224391 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48
14:17:39.226510 IP 10.0.148.2 > 224.0.0.5: OSPFv2, Hello, length 48

      Unfortunately, I do not know why routers on both ends of the GRE tunnel are unable to establish OSPF neighbor relationship.

      Does anyone has any idea how that issue can be solved? I will be grateful for any advice.

      For now I must disable OSPF on GRE tunnel interfaces, effectively disabling OSPF beetwen Cloud A and Cloud B.

      1 Reply Last reply Reply Quote 0
      • L
        Leksandr last edited by

        FRR config for pfsense-vpn (Cloud A)

        ##################### DO NOT EDIT THIS FILE! ######################
###################################################################
# This file was created by an automatic configuration generator.  #
# The contents of this file will be overwritten without warning!  #
###################################################################
!
frr defaults traditional
hostname pfsense-vpn.**hidden**
password **hidden**
log syslog
service integrated-vtysh-config
service password-encryption
!
ip router-id 192.168.240.6
!
interface gre0
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 **hidden**
 ip ospf area 1
interface vtnet0
 ip ospf network broadcast
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 **hidden**
 ip ospf area 0
interface ovpns1
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 **hidden**
 ip ospf area 0
interface ovpns2
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 **hidden**
 ip ospf area 0
!
router ospf
 ospf router-id 192.168.240.6
 log-adjacency-changes detail
 redistribute connected metric 1
 redistribute kernel metric 1
 redistribute static metric 1
 passive-interface ovpns1
 passive-interface ovpns2
 area 0 shortcut default
 area 0 authentication message-digest
 area 1 shortcut default
 area 1 authentication message-digest
!
line vty
!
end

        FRR config for pfsense-main (Cloud B)

        ##################### DO NOT EDIT THIS FILE! ######################
###################################################################
# This file was created by an automatic configuration generator.  #
# The contents of this file will be overwritten without warning!  #
###################################################################
!
frr defaults traditional
hostname pfSense-cloud.**hidden**
password **hidden**
log syslog
service integrated-vtysh-config
service password-encryption
!
ip router-id 10.0.148.2
!
interface gre0
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 **hidden**
 ip ospf area 1
interface ovpns1
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 **hidden**
 ip ospf area 2
!
router ospf
 ospf router-id 10.0.148.2
 log-adjacency-changes detail
 redistribute connected metric 1
 passive-interface ovpns1
 area 1 shortcut default
 area 1 authentication message-digest
 area 2 shortcut default
 area 2 authentication message-digest
 neighbor 10.0.148.1 priority 100 poll-interval 40
!
line vty
!
end
        1 Reply Last reply Reply Quote 0
        • First post
          Last post