Frr with get two BGP full route tables from Neighbors with openvpn Update Source

yon 0

amd64
12.0-RELEASE-p10
FreeBSD 12.0-RELEASE-p10 fe839027797(RELENG_2_5) pfSense

Crash report details:

No PHP errors found.

Filename: /var/crash/info.0
Dump header from device: /dev/label/swap0
Architecture: amd64
Architecture Version: 4
Dump Length: 156160
Blocksize: 512
Compression: none
Dumptime: Wed Nov 6 03:31:05 2019
Hostname:
Magic: FreeBSD Text Dump
Version String: FreeBSD 12.0-RELEASE-p10 fe839027797(RELENG_2_5) pfSense
Panic String:
Dump Parity: 3303730181
Bounds: 0
Dump Status: good

Filename: /var/crash/textdump.tar.0
ddb.txt06000014000013560346771 7106 ustarrootwheeldb:0:kdb.enter.default> run lockinfo
db:1:lockinfo> show locks
No such command; use "help" to list available commands
db:1:lockinfo> show alllocks
No such command; use "help" to list available commands
db:1:lockinfo> show lockedvnods
Locked vnodes
db:0:kdb.enter.default> show pcpu
cpuid = 0
dynamic pcpu = 0xb45ac0
curthread = 0xfffff8002e69a580: pid 76595 tid 101161 "Zebra dplane"
curpcb = 0xfffffe00506d3cc0
fpcurthread = 0xfffff8002e69a580: pid 76595 "Zebra dplane"
idlethread = 0xfffff8000432c000: tid 100003 "idle: cpu0"
curpmap = 0xfffff8002e17e130
tssp = 0xffffffff82db7420
commontssp = 0xffffffff82db7420
rsp0 = 0xfffffe00506d3cc0
gs32p = 0xffffffff82dbe058
ldt = 0xffffffff82dbe098
tss = 0xffffffff82dbe088
curvnet = 0xfffff80004089380
db:0:kdb.enter.default> bt
Tracing pid 76595 tid 101161 td 0xfffff8002e69a580
rtrequest1_fib() at rtrequest1_fib+0x2b1/frame 0xfffffe00506d3660
route_output() at route_output+0xc61/frame 0xfffffe00506d38e0
sosend_generic() at sosend_generic+0x586/frame 0xfffffe00506d3990
sosend() at sosend+0x50/frame 0xfffffe00506d39c0
soo_write() at soo_write+0x33/frame 0xfffffe00506d3a00
dofilewrite() at dofilewrite+0xb2/frame 0xfffffe00506d3a50
sys_write() at sys_write+0xc3/frame 0xfffffe00506d3ac0
amd64_syscall() at amd64_syscall+0x369/frame 0xfffffe00506d3bf0
fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00506d3bf0
--- syscall (4, FreeBSD ELF64, sys_write), rip = 0x8006deb0a, rsp = 0x7fffdfffd788, rbp = 0x7fffdfffd7c0 ---
db:0:kdb.enter.default> ps
pid ppid pgrp uid state wmesg wchan cmd
76846 1 76846 0 Ss (threaded) bgpd
100872 S select 0xfffff8000b3b5340 bgpd
101162 S select 0xfffff8000b2ec4c0 bgpd_io
101163 S uwait 0xfffff8002e3c3800 bgpd_ka
76595 1 76595 0 Rs (threaded) zebra
100169 Run CPU 2 zebra
101161 Run CPU 0 Zebra dplane
101164 S select 0xfffff8000b3c88c0 zebra_apic
101165 S select 0xfffff8004d2de5c0 zebra_apic
53349 1 811 0 S nanslp 0xffffffff82b31d53 php-cgi
35560 6757 811 0 S nanslp 0xffffffff82b31d51 sleep
91986 91512 91512 142 S kqread 0xfffff80006ff2300 ladvd
91512 1 91512 0 Ss kqread 0xfffff8000be81100 ladvd

msgbuf.txt06000027767013560346771 7670 ustarrootwheelet: Network is unreachable
<118>route: route has not been found
<118>route: writing to routing socket: Network is unreachable
<118>route: route has not been found

<118>Configuring LAGG interfaces...
<6>lagg0: IPv6 addresses on igb0 have been removed before adding it as a member to prevent IPv6 address scope violation.
<6>lagg0: link state changed to DOWN
<6>lagg0: IPv6 addresses on igb1 have been removed before adding it as a member to prevent IPv6 address scope violation.

Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address = 0x70
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80e7d451
stack pointer = 0x28:0xfffffe00506d3560
frame pointer = 0x28:0xfffffe00506d3660
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 76595 (Zebra dplane)
version.txt0600007113560346771 7545 ustarrootwheelFreeBSD 12.0-RELEASE-p10 fe839027797(RELENG_2_5) pfSense

yon 0

@jimp when your will fix it?

jimp

There isn't enough information to say what exactly is wrong yet, and we haven't been able to reproduce any crashes here so far. So there is no ETA on a fix.

2.5.0 is under development, it's not expected to be stable at this point.

yon 0

@jimp That could be from the RADIX_MPATH / multipath options we enabled in the kernel and FRR on 2.5.0 to allow for multi-path/ECMP routing. It's entirely possible the code doesn't work as well as it should, since it's not as heavily tested.

maybe should check this.

yon 0

i had try use frr vtysh shell add two openvpn ipv6 address , it still down too. So i think the pf frr with openvpn has bug.

yon 0

@jimp said in Frr with use two ipv6 BGP Neighbors openvpn Update Source:

There isn't enough information to say what exactly is wrong yet, and we haven't been able to reproduce any crashes here so far. So there is no ETA on a fix.

2.5.0 is under development, it's not expected to be stable at this point.

your try frr get two ipv6 full table upstream. why only has one ipv6 upstream is normal work.

frr and system bug

yon 0

@jimp i try setup use openbgpd normarl work ipv6 with openvpn. so i think the frr sure has bugs.

yon 0

frr v7.2 is out, we need for upgrade?

yon 0

<118>Configuring LAGG interfaces...
<6>lagg0: IPv6 addresses on igb0 have been removed before adding it as a member to prevent IPv6 address scope violation.
<6>lagg0: link state changed to DOWN
<6>lagg0: IPv6 addresses on igb1 have been removed before adding it as a member to prevent IPv6 address scope violation.
<118>done.
<118>Configuring VLAN interfaces...
<6>vlan0: changing name to 'igb1.2'
<118>done.
<118>Configuring QinQ interfaces...done.
<118>Configuring IPsec VTI interfaces...done.
<118>Configuring WAN interface...
<6>ng0: changing name to 'pppoe0'
<6>igb2: link state changed to UP
<6>igb0: link state changed to UP
<6>igb1: link state changed to UP
<6>igb1.2: link state changed to UP
<118>done.
<118>Configuring LAN interface...done.
<118>Configuring WAN2 interface...
<6>ng1: changing name to 'pppoe2'
<6>igb3: link state changed to UP
<6>lagg0: link state changed to UP
<118>done.
<118>Configuring CARP settings...done.
<118>Syncing OpenVPN settings...
<6>tun7: changing name to 'ovpnc7'
<6>tun3: changing name to 'ovpnc3'
<6>tun9: changing name to 'ovpnc9'
<6>tun10: changing name to 'ovpnc10'
<6>tun11: changing name to 'ovpnc11'
<118>done.
<6>pflog0: promiscuous mode enabled
<118>Configuring firewall......done.
<118>Starting PFLOG...done.
<118>Setting up gateway monitors...done.
<118>Setting up static routes...done.
<118>Setting up DNSs...
<118>Synchronizing user settings...done.
<118>Starting webConfigurator...done.
<118>Configuring CRON...done.
<118>Starting NTP time client...done.
<118>Starting DHCP service...done.
<118>Starting DHCPv6 service...done.
<118>Configuring firewall......done.
<6>gif4: link state changed to DOWN
<6>gif4: link state changed to UP
<118>Generating RRD graphs...done.
<118>Starting UPnP service... done.
<6>gif0: link state changed to DOWN
<6>gif0: link state changed to UP
<118>Starting syslog...done.
<118>Starting CRON... done.
<118>>>> Removing unnecessary packages... done.
<118>>>> Cleanup pkg cache... done.
<118> Starting package Shellcmd...done.
<118> Starting package Cron...done.
<118> Starting package Backup...done.
<118> Starting package Filer...done.
<118> Starting package LADVD...done.
<118> Starting package iftop...done.
<118> Starting package mtr-nox11...done.
<118> Starting package RRD Summary...done.
<118> Starting package Service Watchdog...done.
<118> Starting package Notes...done.
<118> Starting package OpenBGPD...done.
<118> Starting package acme...done.
<118> Starting package Traffic Totals...done.
<118> Starting package System Patches...done.
<118> Starting package sudo...done.
<118> Starting package OpenVPN Client Export Utility...done.
<118> Starting package iperf...done.
<118> Starting /usr/local/etc/rc.d/bgpd.sh...done.
<118>pfSense 2.5.0-DEVELOPMENT amd64 Thu Nov 21 21:28:02 EST 2019
<118>Bootup complete
<6>ovpnc7: link state changed to UP
<6>ovpnc10: link state changed to UP
<6>ovpnc3: link state changed to UP
<6>ovpnc11: link state changed to UP
<6>ovpnc9: link state changed to UP
<6>ovpnc7: link state changed to DOWN
<6>ovpnc7: link state changed to UP
<6>ovpnc7: link state changed to DOWN
<6>ovpnc7: link state changed to UP
<6>ovpnc7: link state changed to DOWN
<6>ovpnc7: link state changed to UP
<6>ovpnc7: link state changed to DOWN
<6>ovpnc7: link state changed to UP
<6>ovpnc7: link state changed to DOWN
<6>ovpnc7: link state changed to UP
<6>ovpnc7: link state changed to DOWN
<6>ovpnc7: link state changed to UP
<6>ovpnc3: link state changed to DOWN
<6>ovpnc3: link state changed to UP
<6>ovpnc3: link state changed to DOWN
<6>ovpnc9: link state changed to DOWN
<6>ovpnc9: link state changed to UP
<6>ovpnc3: link state changed to UP
<6>re0: link state changed to DOWN
<6>ovpnc7: link state changed to DOWN
<6>ovpnc7: link state changed to UP
<6>ovpnc7: link state changed to DOWN
<6>ovpnc7: link state changed to UP
<6>ovpnc7: link state changed to DOWN
<6>ovpnc7: link state changed to UP
<3>rn_delete: Orphaned Mask 0xfffff801c10ab060 at 0
<3>rn_delete: Orphaned Mask 0xfffff803c6d3b320 at 0
<3>rn_delete: Orphaned Mask 0xfffff8042a4741a0 at 0
<3>rn_delete: Orphaned Mask 0xfffff803c6e202c0 at 0
<3>rn_delete: Orphaned Mask 0xfffff801c0875a80 at 0
<3>rn_delete: couldn't find our annotation

Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 11
fault virtual address = 0x70
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80e7d451
stack pointer = 0x28:0xfffffe0095ab53c0
frame pointer = 0x28:0xfffffe0095ab54c0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 64872 (bgpd)
version.txt0600007113566056012 7535 ustarrootwheelFreeBSD 12.0-RELEASE-p10 fe839027797(RELENG_2_5) pfSense

yon 0

<118> Starting package Service Watchdog...done.
<118> Starting package Notes...done.
<118> Starting package System Patches...done.
<118> Starting package iperf...done.
<118> Starting package FRR...done.
<118>pfSense 2.5.0-DEVELOPMENT amd64 Fri Nov 29 08:22:22 EST 2019
<118>Bootup complete
<6>ovpnc11: link state changed to UP
<6>ovpnc9: link state changed to UP
<6>ovpnc3: link state changed to UP
<6>ovpnc10: link state changed to UP
<6>ovpnc7: link state changed to UP
<3>rn_delete: couldn't find our annotation

Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 13
fault virtual address = 0x70
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80e7d451
stack pointer = 0x28:0xfffffe00944f43a0
frame pointer = 0x28:0xfffffe00944f44a0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 53160 (Zebra dplane)
version.txt0600007113570412703 7533 ustarrootwheelFreeBSD 12.0-RELEASE-p10 fe839027797(RELENG_2_5) pfSense

yon 0

i need edit this, when frr got two or more and ipv4 or ipv6 full route tables from neighbors via openvpn update Source, then frr will down.

yon 0

The results of repeated tests should be that there is a problem with the system, which has an impact on multiple BGP software, but it has the greatest impact on frr, causing the system to be paralyzed.