Doesn’t start properly



  • I’m running a pfSense 2.3.1-RELEASE-p1 (amd64) on a ESXi 6 host. Today I had a hardware failure but the host is running again. But since then my pfSense doesn’t start as it should. The console stalls and the log file```
    Jun 3 04:19:24 kernel pcib26: <acpi pci-pci="" bridge=""> at device 23.7 on pci0
    Jun 3 04:19:24 kernel pci26: <acpi pci="" bus=""> on pcib26
    Jun 3 04:19:24 kernel pcib27: <acpi pci-pci="" bridge=""> at device 24.0 on pci0
    Jun 3 04:19:24 kernel pci27: <acpi pci="" bus=""> on pcib27
    Jun 3 04:19:24 kernel pcib28: <acpi pci-pci="" bridge=""> at device 24.1 on pci0
    Jun 3 04:19:24 kernel pci28: <acpi pci="" bus=""> on pcib28
    Jun 3 04:19:24 kernel pcib29: <acpi pci-pci="" bridge=""> at device 24.2 on pci0
    Jun 3 04:19:24 kernel pci29: <acpi pci="" bus=""> on pcib29
    Jun 3 04:19:24 kernel pcib30: <acpi pci-pci="" bridge=""> at device 24.3 on pci0
    Jun 3 04:19:24 kernel pci30: <acpi pci="" bus=""> on pcib30
    Jun 3 04:19:24 kernel pcib31: <acpi pci-pci="" bridge=""> at device 24.4 on pci0
    Jun 3 04:19:24 kernel pci31: <acpi pci="" bus=""> on pcib31
    Jun 3 04:19:24 kernel pcib32: <acpi pci-pci="" bridge=""> at device 24.5 on pci0
    Jun 3 04:19:24 kernel pci32: <acpi pci="" bus=""> on pcib32
    Jun 3 04:19:24 kernel pcib33: <acpi pci-pci="" bridge=""> at device 24.6 on pci0
    Jun 3 04:19:24 kernel pci33: <acpi pci="" bus=""> on pcib33
    Jun 3 04:19:24 kernel pcib34: <acpi pci-pci="" bridge=""> at device 24.7 on pci0
    Jun 3 04:19:24 kernel pci34: <acpi pci="" bus=""> on pcib34
    Jun 3 04:19:24 kernel acpi_acad0: <ac adapter=""> on acpi0
    Jun 3 04:19:24 kernel atkbdc0: <keyboard controller="" (i8042)=""> port 0x60,0x64 irq 1 on acpi0
    Jun 3 04:19:24 kernel atkbd0: <at keyboard=""> irq 1 on atkbdc0
    Jun 3 04:19:24 kernel kbd0 at atkbd0
    Jun 3 04:19:24 kernel atkbd0: [GIANT-LOCKED]
    Jun 3 04:19:24 kernel psm0: <ps 2="" mouse=""> irq 12 on atkbdc0
    Jun 3 04:19:24 kernel psm0: [GIANT-LOCKED]
    Jun 3 04:19:24 kernel psm0: model IntelliMouse, device ID 3
    Jun 3 04:19:24 kernel orm0: <isa option="" roms=""> at iomem 0xc0000-0xc7fff,0xc8000-0xc9fff,0xca000-0xcafff,0xcb000-0xcbfff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0
    Jun 3 04:19:24 kernel sc0: <system console=""> at flags 0x100 on isa0
    Jun 3 04:19:24 kernel sc0: VGA <16 virtual consoles, flags=0x300>
    Jun 3 04:19:24 kernel vga0: <generic isa="" vga=""> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
    Jun 3 04:19:24 kernel ppc0: cannot reserve I/O port range
    Jun 3 04:19:24 kernel Timecounters tick every 10.000 msec
    Jun 3 04:19:24 kernel em0: link state changed to UP
    Jun 3 04:19:24 kernel em1: link state changed to UP
    Jun 3 04:19:24 kernel random: unblocking device.
    Jun 3 04:19:24 kernel cd0 at ata1 bus 0 scbus1 target 0 lun 0
    Jun 3 04:19:24 kernel cd0: <necvmwar vmware="" ide="" cdr10="" 1.00=""> Removable CD-ROM SCSI device
    Jun 3 04:19:24 kernel cd0: Serial Number 10000000000000000001
    Jun 3 04:19:24 kernel cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
    Jun 3 04:19:24 kernel cd0: Attempt to query device size failed: NOT READY, Medium not present
    Jun 3 04:19:24 kernel cd0: quirks=0x40 <retry_busy>Jun 3 04:19:24 kernel da0 at mpt0 bus 0 scbus2 target 0 lun 0
    Jun 3 04:19:24 kernel da0: <vmware virtual="" disk="" 1.0=""> Fixed Direct Access SCSI-2 device
    Jun 3 04:19:24 kernel da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)
    Jun 3 04:19:24 kernel da0: Command Queueing enabled
    Jun 3 04:19:24 kernel da0: 12288MB (25165824 512 byte sectors)
    Jun 3 04:19:24 kernel da0: quirks=0x40 <retry_busy>Jun 3 04:19:24 kernel Timecounter "TSC-low" frequency 1546487000 Hz quality 1000
    Jun 3 04:19:24 kernel Trying to mount root from ufs:/dev/ufsid/5710176585c80eae [rw]...
    Jun 3 04:19:24 kernel WARNING: / was not properly dismounted
    Jun 3 04:19:24 kernel WARNING: /: mount pending error: blocks 120 files 0
    Jun 3 04:19:24 kernel vlan0: changing name to 'em1_vlan10'
    Jun 3 04:19:24 kernel vlan1: changing name to 'em1_vlan99'
    Jun 3 04:19:24 check_reload_status rc.newwanip starting em0
    Jun 3 04:19:24 php-cgi rc.bootup: Accept router advertisements on interface em0
    Jun 3 04:19:24 php-cgi rc.bootup: Resyncing OpenVPN instances.
    Jun 3 04:19:24 kernel done.
    Jun 3 04:19:24 kernel done.
    Jun 3 04:19:24 php-cgi rc.bootup: Not installing NAT reflection rules for a port range > 500
    Jun 3 04:19:24 kernel pflog0: promiscuous mode enabled
    Jun 3 04:19:24 php-cgi rc.bootup: Not installing NAT reflection rules for a port range > 500
    Jun 3 04:19:24 xinetd 15338 xinetd Version 2.3.15 started with libwrap loadavg options compiled in.
    Jun 3 04:19:24 kernel ...
    Jun 3 04:19:24 xinetd 15338 Started working: 3 available services
    Jun 3 04:19:24 kernel .done.
    Jun 3 04:19:25 kernel done.
    Jun 3 04:19:25 php-fpm 266 /rc.newwanip: rc.newwanip: Info: starting on em0.
    Jun 3 04:19:25 php-fpm 266 /rc.newwanip: rc.newwanip: on (IP address: 212.51.141.154) (interface: WAN[wan]) (real interface: em0).
    Jun 3 04:19:26 kernel done.
    Jun 3 04:19:26 php-cgi rc.bootup: ROUTING: setting default route to 212.51.141.1
    Jun 3 04:19:26 kernel done.
    Jun 3 04:19:26 kernel done.
    Jun 3 04:19:26 php-cgi rc.bootup: NTPD is starting up.
    Jun 3 04:19:27 rtsold Recieved RA specifying route fe80::224:14ff:fef0:63f for interface wan(em0)
    Jun 3 04:19:27 rtsold Starting dhcp6 client for interface wan(em0)
    Jun 3 04:19:27 kernel done.
    Jun 3 04:19:28 kernel done.
    Jun 3 04:19:28 dhcpleases kqueue error: unkown
    Jun 3 04:19:28 check_reload_status Updating all dyndns
    Jun 3 04:19:28 php-cgi rc.bootup: Not installing NAT reflection rules for a port range > 500
    Jun 3 04:19:28 php-cgi rc.bootup: Not installing NAT reflection rules for a port range > 500
    Jun 3 04:19:28 xinetd 15338 Starting reconfiguration
    Jun 3 04:19:28 xinetd 15338 Swapping defaults
    Jun 3 04:19:28 xinetd 15338 readjusting service 6969-udp
    Jun 3 04:19:28 xinetd 15338 readjusting service 19000-udp
    Jun 3 04:19:28 xinetd 15338 readjusting service 19001-udp
    Jun 3 04:19:28 xinetd 15338 Reconfigured: new=0 old=3 dropped=0 (services)
    Jun 3 04:19:28 kernel ...
    Jun 3 04:19:28 kernel .done.
    Jun 3 04:19:28 php-fpm 266 /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
    Jun 3 04:19:28 php-fpm 266 /rc.newwanipv6: rc.newwanipv6: on (IP address: 2a02:168:2000:10:20c:29ff:fe48:db41) (interface: wan) (real interface: em0).
    Jun 3 04:19:28 php-fpm 266 /rc.newwanipv6: ROUTING: setting default route to 212.51.141.1
    Jun 3 04:19:28 php-fpm 266 /rc.newwanipv6: ROUTING: setting IPv6 default route to fe80::224:14ff:fef0:63f%em0
    Jun 3 04:19:28 php-fpm 266 /rc.newwanipv6: Removing static route for monitor fe80::224:14ff:fef0:63f and adding a new route through fe80::224:14ff:fef0:63f%em0
    Jun 3 04:19:27 php-fpm 266 /rc.dyndns.update: phpDynDNS (quidproquo.myfirewall.org): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 3 04:19:28 php-fpm 266 /rc.dyndns.update: phpDynDNS (quidproquo.myfirewall.org): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 3 04:20:57 php-fpm 42156 /index.php: Successful login for user 'admin' from: 10.0.1.107
    Jun 3 04:20:57 sshlockout 30542 sshlockout/webConfigurator v3.0 starting up
    Jun 3 04:20:58 php-fpm 42156 /index.php: Unable to find IPsec daemon leases file. Could not display mobile user stats!
    Jun 3 04:20:58 php-fpm 31278 /widgets/widgets/ipsec.widget.php: Unable to find IPsec daemon leases file. Could not display mobile user stats!</retry_busy></vmware></retry_busy></necvmwar></generic></system></isa></ps></at></keyboard></ac></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi>

    Neither the ipsec service is starting nor the vmware-guestd
    In the end the GUI is accessible, but I think something isn’t correct.
    ![Bildschirmfoto 2016-06-03 um 04.23.27.png](/public/_imported_attachments_/1/Bildschirmfoto 2016-06-03 um 04.23.27.png)
    ![Bildschirmfoto 2016-06-03 um 04.23.27.png_thumb](/public/_imported_attachments_/1/Bildschirmfoto 2016-06-03 um 04.23.27.png_thumb)


  • What does it show if you hit ctrl-t there?



  • Thank you very much. It showed that the mongod was hanging. It’s part of the UniFi controller running on my pfSense as well. Until now there was no problem with that.
    Now ipsec is running but no IPv6 traffic. Any further advice?