Upgraded from 2.3.3 to 2.4.0 Ipsec routing error
-
Hi,
Updated yesterday from 2.3.3 to 2.4.0 (I was waiting for the Multi Wan Reply-To bug fix).
But got another problem.Services on this side of IPSec are not reachable from the other side.
Can't find any errors except in dmesg -a
Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-RELEASE-p6 #55 2ede8a24166(RELENG_2_4): Thu Jan 12 07:49:59 CST 2017 root@buildbot2.netgate.com:/builder/ce/tmp/obj/builder/ce/tmp/FreeBSD-src/sys/pfSense amd64 FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0) VT(vga): text 80x25 CPU: Intel(R) Core(TM) i7 CPU 960 @ 3.20GHz (3197.73-MHz K8-class CPU) Origin="GenuineIntel" Id=0x106a5 Family=0x6 Model=0x1a Stepping=5 Features=0x1fa3fbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,dts,mmx,fxsr,sse,sse2,ss,htt>Features2=0x81b82221 <sse3,vmx,ssse3,cx16,sse4.1,sse4.2,x2apic,popcnt,tscdlt,hv>AMD Features=0x28100800 <syscall,nx,rdtscp,lm>AMD Features2=0x1 <lahf>Structured Extended Features=0x2 <tscadj>VT-x: PAT,HLT,MTF,PAUSE,EPT,VPID TSC: P-state invariant Hypervisor: Origin = "VMwareVMware" real memory = 2147483648 (2048 MB) avail memory = 2023337984 (1929 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: <ptltd ="" apic ="">FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 2 package(s) x 2 core(s) MADT: Forcing active-low polarity and level trigger for SCI ioapic0 <version 1.1="">irqs 0-23 on motherboard random: entropy device external interface wlan: mac acl policy registered iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff8069e9d0, 0) error 1 iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff8069ea80, 0) error 1 iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff8069eb30, 0) error 1 kbd1 at kbdmux0 netmap: loaded module module_register_init: MOD_LOAD (vesa, 0xffffffff8122a980, 0) error 19 vtvga0: <vt vga="" driver="">on motherboard cryptosoft0: <software crypto="">on motherboard padlock0: No ACE support. acpi0: <intel 440bx="">on motherboard acpi0: Power Button (fixed) Timecounter "HPET" frequency 14318180 Hz quality 950 cpu0: <acpi cpu="">numa-domain 0 on acpi0 cpu1: <acpi cpu="">numa-domain 0 on acpi0 cpu2: <acpi cpu="">numa-domain 0 on acpi0 cpu3: <acpi cpu="">numa-domain 0 on acpi0 attimer0: <at timer="">port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: <at realtime="" clock="">port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff on acpi0 pci0: <acpi pci="" bus="">on pcib0 pcib1: <acpi pci-pci="" bridge="">at device 1.0 on pci0 pci1: <acpi pci="" bus="">on pcib1 isab0: <pci-isa bridge="">at device 7.0 on pci0 isa0: <isa bus="">on isab0 atapci0: <intel piix4="" udma33="" controller="">port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1060-0x106f at device 7.1 on pci0 ata0: <ata channel="">at channel 0 on atapci0 ata1: <ata channel="">at channel 1 on atapci0 pci0: <bridge>at device 7.3 (no driver attached) vgapci0: <vga-compatible display="">port 0x1070-0x107f mem 0xe8000000-0xefffffff,0xfe000000-0xfe7fffff irq 16 at device 15.0 on pci0 vgapci0: Boot video device mpt0: <lsilogic 1030="" ultra4="" adapter="">port 0x1400-0x14ff mem 0xfeba0000-0xfebbffff,0xfebc0000-0xfebdffff irq 17 at device 16.0 on pci0 mpt0: MPI Version=1.2.0.0 pcib2: <acpi pci-pci="" bridge="">at device 17.0 on pci0 pci2: <acpi pci="" bus="">on pcib2 uhci0: <uhci (generic)="" usb="" controller="">port 0x2000-0x201f irq 19 at device 1.0 on pci2 usbus0 on uhci0 ehci0: <ehci (generic)="" usb="" 2.0="" controller="">mem 0xfd5ff000-0xfd5fffff irq 16 at device 2.0 on pci2 usbus1: EHCI version 1.0 usbus1 on ehci0 pcib3: <acpi pci-pci="" bridge="">at device 21.0 on pci0 pcib3: [GIANT-LOCKED] pci3: <acpi pci="" bus="">on pcib3 vmx0: <vmware vmxnet3="" ethernet="" adapter="">port 0x4000-0x400f mem 0xfd4fc000-0xfd4fcfff,0xfd4fd000-0xfd4fdfff,0xfd4fe000-0xfd4fffff irq 18 at device 0.0 on pci3 vmx0: Ethernet address: 00:50:56:85:19:ce pcib4: <acpi pci-pci="" bridge="">at device 21.1 on pci0 pcib4: [GIANT-LOCKED] pci4: <acpi pci="" bus="">on pcib4 vmx1: <vmware vmxnet3="" ethernet="" adapter="">port 0x8000-0x800f mem 0xfd0fc000-0xfd0fcfff,0xfd0fd000-0xfd0fdfff,0xfd0fe000-0xfd0fffff irq 18 at device 0.0 on pci4 vmx1: Ethernet address: fc:d4:f2:df:00:04 pcib5: <acpi pci-pci="" bridge="">at device 21.2 on pci0 pcib5: [GIANT-LOCKED] pcib6: <acpi pci-pci="" bridge="">at device 21.3 on pci0 pcib6: [GIANT-LOCKED] pcib7: <acpi pci-pci="" bridge="">at device 21.4 on pci0 pcib7: [GIANT-LOCKED] pcib8: <acpi pci-pci="" bridge="">at device 21.5 on pci0 pcib8: [GIANT-LOCKED] pcib9: <acpi pci-pci="" bridge="">at device 21.6 on pci0 pcib9: [GIANT-LOCKED] pcib10: <acpi pci-pci="" bridge="">at device 21.7 on pci0 pcib10: [GIANT-LOCKED] pcib11: <acpi pci-pci="" bridge="">at device 22.0 on pci0 pcib11: [GIANT-LOCKED] pci5: <acpi pci="" bus="">on pcib11 vmx2: <vmware vmxnet3="" ethernet="" adapter="">port 0x5000-0x500f mem 0xfd3fc000-0xfd3fcfff,0xfd3fd000-0xfd3fdfff,0xfd3fe000-0xfd3fffff irq 19 at device 0.0 on pci5 vmx2: Ethernet address: fc:d4:f2:df:00:01 pcib12: <acpi pci-pci="" bridge="">at device 22.1 on pci0 pcib12: [GIANT-LOCKED] pci6: <acpi pci="" bus="">on pcib12 vmx3: <vmware vmxnet3="" ethernet="" adapter="">port 0x9000-0x900f mem 0xfcffc000-0xfcffcfff,0xfcffd000-0xfcffdfff,0xfcffe000-0xfcffffff irq 19 at device 0.0 on pci6 vmx3: Ethernet address: fc:d4:f2:df:00:05 pcib13: <acpi pci-pci="" bridge="">at device 22.2 on pci0 pcib13: [GIANT-LOCKED] pcib14: <acpi pci-pci="" bridge="">at device 22.3 on pci0 pcib14: [GIANT-LOCKED] pcib15: <acpi pci-pci="" bridge="">at device 22.4 on pci0 pcib15: [GIANT-LOCKED] pcib16: <acpi pci-pci="" bridge="">at device 22.5 on pci0 pcib16: [GIANT-LOCKED] pcib17: <acpi pci-pci="" bridge="">at device 22.6 on pci0 pcib17: [GIANT-LOCKED] pcib18: <acpi pci-pci="" bridge="">at device 22.7 on pci0 pcib18: [GIANT-LOCKED] pcib19: <acpi pci-pci="" bridge="">at device 23.0 on pci0 pcib19: [GIANT-LOCKED] pci7: <acpi pci="" bus="">on pcib19 vmx4: <vmware vmxnet3="" ethernet="" adapter="">port 0x6000-0x600f mem 0xfd2fc000-0xfd2fcfff,0xfd2fd000-0xfd2fdfff,0xfd2fe000-0xfd2fffff irq 16 at device 0.0 on pci7 vmx4: Ethernet address: fc:d4:f2:df:00:02 pcib20: <acpi pci-pci="" bridge="">at device 23.1 on pci0 pcib20: [GIANT-LOCKED] pcib21: <acpi pci-pci="" bridge="">at device 23.2 on pci0 pcib21: [GIANT-LOCKED] pcib22: <acpi pci-pci="" bridge="">at device 23.3 on pci0 pcib22: [GIANT-LOCKED] pcib23: <acpi pci-pci="" bridge="">at device 23.4 on pci0 pcib23: [GIANT-LOCKED] pcib24: <acpi pci-pci="" bridge="">at device 23.5 on pci0 pcib24: [GIANT-LOCKED] pcib25: <acpi pci-pci="" bridge="">at device 23.6 on pci0 pcib25: [GIANT-LOCKED] pcib26: <acpi pci-pci="" bridge="">at device 23.7 on pci0 pcib26: [GIANT-LOCKED] pcib27: <acpi pci-pci="" bridge="">at device 24.0 on pci0 pcib27: [GIANT-LOCKED] pci8: <acpi pci="" bus="">on pcib27 vmx5: <vmware vmxnet3="" ethernet="" adapter="">port 0x7000-0x700f mem 0xfd1fc000-0xfd1fcfff,0xfd1fd000-0xfd1fdfff,0xfd1fe000-0xfd1fffff irq 17 at device 0.0 on pci8 vmx5: Ethernet address: fc:d4:f2:df:00:03 pcib28: <acpi pci-pci="" bridge="">at device 24.1 on pci0 pcib28: [GIANT-LOCKED] pcib29: <acpi pci-pci="" bridge="">at device 24.2 on pci0 pcib29: [GIANT-LOCKED] pcib30: <acpi pci-pci="" bridge="">at device 24.3 on pci0 pcib30: [GIANT-LOCKED] pcib31: <acpi pci-pci="" bridge="">at device 24.4 on pci0 pcib31: [GIANT-LOCKED] pcib32: <acpi pci-pci="" bridge="">at device 24.5 on pci0 pcib32: [GIANT-LOCKED] pcib33: <acpi pci-pci="" bridge="">at device 24.6 on pci0 pcib33: [GIANT-LOCKED] pcib34: <acpi pci-pci="" bridge="">at device 24.7 on pci0 pcib34: [GIANT-LOCKED] acpi_acad0: <ac adapter="">on acpi0 atkbdc0: <keyboard controller="" (i8042)="">port 0x60,0x64 irq 1 on acpi0 atkbd0: <at keyboard="">irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: <ps 2="" mouse="">irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse, device ID 3 qpi0: <qpi system="" bus="">on motherboard orm0: <isa option="" roms="">at iomem 0xc0000-0xc7fff,0xc8000-0xc9fff,0xca000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff,0xcd000-0xcdfff,0xce000-0xcefff,0xcf000-0xcffff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0 vga0: <generic isa="" vga="">at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range Timecounters tick every 1.000 msec nvme cam probe device init usbus0: 12Mbps Full Speed USB v1.0 usbus1: 480Mbps High Speed USB v2.0 ugen0.1: <0x15ad> at usbus0 uhub0: <0x15ad UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0 ugen1.1: <0x15ad> at usbus1 uhub1: <0x15ad EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1 uhub0: 2 ports with 2 removable, self powered ugen0.2: <vmware>at usbus0 uhid0: <vmware>on usbus0 uhid1: <vmware>on usbus0 ugen0.3: <vendor 0x0e0f="">at usbus0 uhub2: <vmware virtual="" usb="" hub="">on usbus0 (da0:mpt0:0:0:0): UNMAPPED da0 at mpt0 bus 0 scbus2 target 0 lun 0 da0: <vmware virtual="" disk="" 2.0="">Fixed Direct Access SPC-4 SCSI device SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #1 Launched! da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da0: Command Queueing enabled da0: 8192MB (16777216 512 byte sectors) da0: quirks=0x40 <retry_busy>Trying to mount root from ufs:/dev/ufsid/55e0b2e79f855829 [rw]... Configuring crash dumps... uhub1: 6 ports with 6 removable, self powered Using /dev/label/swap0 for dump device. /dev/ufsid/55e0b2e79f855829: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ufsid/55e0b2e79f855829: clean, 278972 free (10220 frags, 33594 blocks, 1.0% fragmentation) Filesystems are clean, continuing... Mounting filesystems... random: unblocking device. ___ ___/ f \134 / p \134___/ Sense \134___/ \134 \134___/ Welcome to pfSense 2.4.0-BETA... No core dumps found. ...ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/ipsec /usr/local/lib/perl5/5.24/mach/CORE 32-bit compatibility ldconfig path: /usr/lib32 done. uhub2: 7 ports with 7 removable, self powered External config loader 1.0 is now starting... da0s1 da0s1a da0s1b Launching the init system....... done. Initializing.................. done. Starting device manager (devd)...done. Loading configuration......done. Updating configuration...done. Cleaning backup cache.................................done. Setting up extended sysctls...done. Setting timezone...done. Configuring loopback interface...done. Starting syslog...done. Starting Secure Shell Services...done. Setting up interfaces microcode...done. Configuring loopback interface...done. Creating wireless clone interfaces...done. Configuring LAGG interfaces...done. Configuring VLAN interfaces...done. Configuring QinQ interfaces...done. Configuring WAN1 interface... vmx2: link state changed to UP done. Configuring LAN interface... vmx0: link state changed to UP done. Configuring WAN2 interface... vmx4: link state changed to UP done. Configuring WAN3 interface... vmx5: link state changed to UP gif0: link state changed to UP done. Configuring WAN4 interface... vmx1: link state changed to UP gif1: link state changed to UP arp: 0.0.0.0 moved from 20:8c:d3:5f:00:f8 to 00:0b:82:63:de:14 on vmx0 gif2: link state changed to UP done. Configuring MOBILE interface... vmx3: link state changed to UP done. gif0: link state changed to DOWN gif0: link state changed to UP gif1: link state changed to DOWN gif1: link state changed to UP gif2: link state changed to DOWN gif2: link state changed to UP gif3: link state changed to UP Configuring WAN1IPV6 interface...done. Configuring WAN2IPV6 interface...done. Configuring WAN3IPV6 interface...done. Configuring WAN4IPV6 interface...done. Configuring CARP settings...done. Syncing OpenVPN settings...done. pflog0: promiscuous mode enabled Configuring firewall... gif3: link state changed to DOWN gif3: link state changed to UP ...done. Starting PFLOG...done. Setting up gateway monitors...done. Starting DNS Resolver...done. Synchronizing user settings...done. Starting webConfigurator...done. Configuring CRON...done. Starting NTP time client...done. Starting DHCP service...done. Starting DHCPv6 service...done. Configuring firewall......done. Configuring IPsec VPN... route: writing to routing socket: Invalid argument route: writing to routing socket: Invalid argument done Generating RRD graphs...done. Starting UPnP service... done. Starting syslog...done. route: writing to routing socket: Invalid argument route: writing to routing socket: Invalid argument Starting CRON... done. Starting package Open-VM-Tools...done. Starting package squid3...done. Starting package nmap...done. Starting /usr/local/etc/rc.d/c-icap.sh...done. Starting /usr/local/etc/rc.d/clamd.sh...done. Starting /usr/local/etc/rc.d/sqp_monitor.sh...done. Starting /usr/local/etc/rc.d/vmware-guestd.sh...done. pfSense 2.4.0-BETA amd64 Thu Jan 12 07:45:16 CST 2017 Bootup complete cannot forward src fe80:1::20b:82ff:fe63:de15, dst 2001:4998:c:e33::6000, nxt 6, rcvif vmx0, outif gif1 cannot forward src fe80:1::20b:82ff:fe63:de15, dst 2001:4998:c:e33::6000, nxt 6, rcvif vmx0, outif gif1 cannot forward src fe80:1::20b:82ff:fe63:de13, dst 2001:4998:c:e33::6000, nxt 6, rcvif vmx0, outif gif1 cannot forward src fe80:1::20b:82ff:fe63:de13, dst 2001:4998:58:4904::4000, nxt 6, rcvif vmx0, outif gif1 cannot forward src fe80:1::20b:82ff:fe63:de13, dst 2a00:1288:12c:2::4001, nxt 6, rcvif vmx0, outif gif1 cannot forward src fe80:1::20b:82ff:fe63:de13, dst 2001:4998:c:e33::6000, nxt 6, rcvif vmx0, outif gif1</retry_busy></vmware></vmware></vendor></vmware></vmware></vmware></generic></isa></qpi></ps></at></keyboard></ac></acpi></acpi></acpi></acpi></acpi></acpi></acpi></vmware></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></vmware></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></vmware></acpi></acpi></vmware></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></vmware></acpi></acpi></vmware></acpi></acpi></ehci></uhci></acpi></acpi></lsilogic></vga-compatible></bridge></ata></ata></intel></isa></pci-isa></acpi></acpi></acpi></acpi></at></at></acpi></acpi></acpi></acpi></intel></software></vt></version></ptltd ></tscadj></lahf></syscall,nx,rdtscp,lm></sse3,vmx,ssse3,cx16,sse4.1,sse4.2,x2apic,popcnt,tscdlt,hv></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,dts,mmx,fxsr,sse,sse2,ss,htt>The error "route: writing to routing socket: Invalid argument
route: writing to routing socket: Invalid argument" occurs multiple times.Does it come from the IPSec config? where to start searching?
What can I do about "cannot forward src fe80:1::20b:82ff:fe63:de13, dst 2001:4998:c:e33::6000, nxt 6, rcvif vmx0, outif gif1"
I see that same errors also on my FreeBSD VM which is running at the same ESXi host as pfSense does.Any help would be appreciated.
Regards,
Donald. -
Those errors are unrelated to IPsec.
The "cannot forward" message is because your system is attempting to send non-link-local traffic using a link-local source. fe80 addresses cannot talk to anything outside the current L2, they are not routeable, so they can never reach that destination.
-
Related?
https://forum.pfsense.org/index.php?topic=123370.0 -
Maybe, I'll try to disable IPSec and see if the errors disappear.
Can anyone tell me something about the "route: writing to routing socket: Invalid argument" error?
Didn't have that one on 2.3.3 -
This one is related, and exactly the same problem: https://forum.pfsense.org/index.php?topic=117827.0
The error "route: writing to routing socket: Invalid argument" is something else, and disappears when I disable one of my IPSec Tunnels (net2net).
Roadwarrior IPSec (The other tunnel) is not working (tunnel itself does work, bus traffic does not flow), and gives the default deny error in firewall logs, as above topic.
Tried to create the sloppy state floating rule, but is not working for me. -
I see the same message on 2.4.0-BETA on my new sg-1000.
Does anyone have a solution for this?edit: I also created that sloppy rule … does not work here.
IPSEC tunnel(s) up, but traffic doesn't get through.
The imported config works on another pfsense-2.3.3 -
Yeah, I went OpenVPN…
-
Yeah, I went OpenVPN…
not a valid option for everyone. I have customers with IPSEC only.
-
I know but I had no other option.. Migrated all to openvpn.
Was pain in the ass but it was worth it… -
Well, I assume if it works in 2.3.3 it should be solvable in 2.4.x as well.
-
What is recommended? Should I file a ticket for that issue or simply wait … ? ;-)
-
fixed for me, see https://forum.pfsense.org/index.php?topic=126290.0
