Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    page faults / kernel panics after config change w/23.01

    Scheduled Pinned Locked Moved General pfSense Questions
    10 Posts 3 Posters 927 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • I
      inferno480
      last edited by

      Re: [Source IPv6 Address in Multi-WAN is WAN address](not actual LAN IPv6 address?)

      I recently made a config change to rectify an issue I brought-up in the IPv6 forum, but now I am getting daily kernel panics/page faults. Any suggestions?

      textdump.tar.0

      info.0

      I demD 2 Replies Last reply Reply Quote 0
      • I
        inferno480 @inferno480
        last edited by

        @inferno480

        Crashed again, wish I knew why. Guess I'll re-enable IPv6 NAT? (how that makes sense, I am not sure)

        textdump.tar.0

        info.0

        jimpJ I 2 Replies Last reply Reply Quote 0
        • I
          inferno480
          last edited by

          found this in the log immediately upon reboot after the latest crash:

          Mar 12 00:06:27	kernel		---<<BOOT>>---
Mar 12 00:06:27	kernel		KDB: enter: panic
Mar 12 00:06:27	kernel		time = 1678597461
Mar 12 00:06:27	kernel		cpuid = 13
Mar 12 00:06:27	kernel		panic: page fault
Mar 12 00:06:27	kernel		trap number = 12
Mar 12 00:06:27	kernel		r13: fffff8039830d4d0 r14: 38a r15: fffffffe
Mar 12 00:06:27	kernel		r10: 0 r11: 38a r12: fffff801831afa00
Mar 12 00:06:27	kernel		rax: 38a rbx: 84cd rbp: fffffe01269b2270
Mar 12 00:06:27	kernel		rcx: 2 r8: 200 r9: fffffffe
Mar 12 00:06:27	kernel		rdi: 38a rsi: 0 rdx: a0d
Mar 12 00:06:27	kernel		current process = 57637 (nginx)
Mar 12 00:06:27	kernel		processor eflags = interrupt enabled, resume, IOPL = 0
Mar 12 00:06:27	kernel		= DPL 0, pres 1, long 1, def32 0, gran 1
Mar 12 00:06:27	kernel		code segment = base 0x0, limit 0xfffff, type 0x1b
Mar 12 00:06:27	kernel		frame pointer = 0x28:0xfffffe01269b2270
Mar 12 00:06:27	kernel		stack pointer = 0x28:0xfffffe01269b2240
Mar 12 00:06:27	kernel		instruction pointer = 0x20:0xffffffff80fdc230
Mar 12 00:06:27	kernel		fault code = supervisor read data, page not present
Mar 12 00:06:27	kernel		fault virtual address = 0x0
Mar 12 00:06:27	kernel		cpuid = 13; apic id = 0d
Mar 12 00:06:27	kernel		Fatal trap 12: page fault while in kernel mode
Mar 12 00:06:27	syslogd		kernel boot file is /boot/kernel/kernel
Mar 12 00:06:27	syslogd		sendto: Invalid argument

          Just prior to the latest crash, I don't see anything particularly compelling in the logs, just a few devices trying to communicate to/from Facebook on IPv6 via their link-local addresses (weird).

          Mar 12 00:01:02	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2001:558:fe11:4d::2, nxt 6, rcvif ix0.10, outif ix1
Mar 12 00:00:01	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:58:36	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de84:b717:b0be:760f:27db, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:55:00	sshguard	35612	Now monitoring attacks.
Mar 11 23:55:00	sshguard	29185	Exiting on signal.
Mar 11 23:51:42	kernel		cannot forward src fe80:b::642:1aff:fecb:8eec, dst 2610:20:6f15:15::27, nxt 17, rcvif ix0.100, outif ix1
Mar 11 23:51:03	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2001:558:fe1a:8::1e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:50:30	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2620:108:700f::3270:c478, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:49:30	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2a00:86c0:2041::1, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:46:47	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f134:81:face:b00c:0:38d9, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:46:13	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de85:60a7:6c1e:b894:74ce, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:44:29	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:42:03	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2a04:4e42:400::539, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:41:02	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2001:558:fe1a:8::19, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:39:31	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de84:5701:12b8:1a11:943b, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:37:30	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:31:43	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f134:81:face:b00c:0:38d9, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:31:01	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2001:558:fe1a:8::1b, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:25:28	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:e:face:b00c:0:2, nxt 17, rcvif ix0.10, outif ix1
Mar 11 23:25:14	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:e:face:b00c:0:2, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:21:36	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de84:5229:7a77:935b:82cd, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:19:38	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2a04:4e42:600::539, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:18:52	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f065:2:face:b00c:0:2, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:16:32	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:12:50	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f175:81:face:b00c:0:25de, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:11:44	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:e:face:b00c:0:2, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:11:29	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f003:100:face:b00c:0:3, nxt 17, rcvif ix0.10, outif ix1
Mar 11 23:11:22	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f175:181:face:b00c:0:25de, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:10:48	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2a00:86c0:2041::1, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:10:00	sshguard	29185	Now monitoring attacks.
Mar 11 23:10:00	sshguard	68034	Exiting on signal.
Mar 11 23:08:50	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2a04:4e42:600::539, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:04:59	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:111:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:03:55	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de83:e40a:af1a:866b:9bd0, nxt 6, rcvif ix0.10, outif ix1
Mar 11 23:03:43	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:111:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:58:28	kernel		cannot forward src fe80:a::8e79:f5ff:fe2b:5141, dst 2600:1408:c400:28::17da:d894, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:55:34	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de83:690e:fa3:1d46:abef, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:55:18	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de83:690e:fa3:1d46:abef, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:55:05	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de84:21a3:ee5f:104d:8754, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:54:55	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de84:a214:fe1c:4d84:4a3c, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:54:46	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de84:a214:fe1c:4d84:4a3c, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:54:37	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1f14:62a:de85:4774:a2bd:113f:8054, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:53:10	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:9000:24f7:4c00:9:7244:71c0:93a1, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:52:41	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2a04:4e42:600::539, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:52:08	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2620:108:700f::2ce0:4bc9, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:51:14	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2600:1404:cc00:13::b81c:2914, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:51:01	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2001:558:fe10:14::10, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:50:54	kernel		cannot forward src fe80:a::27c:2dff:fe93:4339, dst 2001:558:fe11:50::2, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:48:10	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:45:37	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f134:81:face:b00c:0:38d9, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:31:03	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f134:81:face:b00c:0:38d9, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:31:00	sshguard	68034	Now monitoring attacks.
Mar 11 22:31:00	sshguard	43608	Exiting on signal.
Mar 11 22:26:17	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 22:17:55	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 21:56:11	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 21:51:26	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 21:51:00	sshguard	43608	Now monitoring attacks.
Mar 11 21:51:00	sshguard	14734	Exiting on signal.
Mar 11 21:49:00	sshguard	14734	Now monitoring attacks.
Mar 11 21:49:00	sshguard	88618	Exiting on signal.
Mar 11 21:47:53	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 21:38:12	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 21:28:59	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 21:24:37	kernel		cannot forward src fe80:a::8e79:f5ff:fe2b:5141, dst 2600:1f14:62a:de84:c768:4983:e89c:3ed9, nxt 6, rcvif ix0.10, outif ix1
Mar 11 21:23:48	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 21:19:03	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:12:face:b00c:0:8e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 21:07:53	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:e:face:b00c:0:2, nxt 17, rcvif ix0.10, outif ix1
Mar 11 21:07:00	sshguard	88618	Now monitoring attacks.
Mar 11 21:07:00	sshguard	14049	Exiting on signal.
Mar 11 20:49:40	kernel		cannot forward src fe80:a::ce6e:a4ff:fe38:99c0, dst 2600:1f14:62a:de84:21a3:ee5f:104d:8754, nxt 6, rcvif ix0.10, outif ix1
Mar 11 20:21:30	kernel		cannot forward src fe80:b::642:1aff:fecb:8eec, dst 2610:20:6f15:15::27, nxt 17, rcvif ix0.100, outif ix1
Mar 11 20:21:00	sshguard	14049	Now monitoring attacks.
Mar 11 20:21:00	sshguard	4643	Exiting on signal.
Mar 11 20:19:59	kernel		cannot forward src fe80:a::ce6e:a4ff:fe38:99c0, dst 2a00:86c0:98:ae::237, nxt 6, rcvif ix0.10, outif ix1
Mar 11 20:19:53	kernel		cannot forward src fe80:a::ce6e:a4ff:fe38:99c0, dst 2600:1f14:62a:de83:690e:fa3:1d46:abef, nxt 6, rcvif ix0.10, outif ix1
Mar 11 20:13:37	kernel		cannot forward src fe80:a::ce6e:a4ff:fe38:99c0, dst 2a00:86c0:98:ae::230, nxt 6, rcvif ix0.10, outif ix1
Mar 11 20:06:29	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f175:181:face:b00c:0:25de, nxt 6, rcvif ix0.10, outif ix1
Mar 11 19:50:20	kernel		cannot forward src fe80:a::ce6e:a4ff:fe38:99c0, dst 2a03:2880:f003:100:face:b00c:0:3, nxt 6, rcvif ix0.10, outif ix1
Mar 11 19:50:14	kernel		cannot forward src fe80:a::ce6e:a4ff:fe38:99c0, dst 2607:f8b0:400a:806::200e, nxt 6, rcvif ix0.10, outif ix1
Mar 11 19:48:11	kernel		cannot forward src fe80:a::1cad:fbff:fe10:d5e1, dst 2a03:2880:f082:e:face:b00c:0:2, nxt 17, rcvif ix0.10, outif ix1
Mar 11 19:38:00	sshguard	4643	Now monitoring attacks.
Mar 11 19:38:00	sshguard	14944	Exiting on signal.
Mar 11 19:05:00	sshguard	14944	Now monitoring attacks.
Mar 11 19:05:00	sshguard	11429	Exiting on signal.
          1 Reply Last reply Reply Quote 0
          • demD
            dem @inferno480
            last edited by dem

            I also had a page fault somewhere in IPv6. I too use Multi-WAN and I use NAT for IPv6.

            At the time of the crash IPv6 was failed over to my backup gateway due to the issue I reported in #12947.

            textdump.tar.0
            info.0

            1 Reply Last reply Reply Quote 0
            • jimpJ
              jimp Rebel Alliance Developer Netgate @inferno480
              last edited by

              @inferno480 said in page faults / kernel panics after config change w/23.01:

              @inferno480

              Crashed again, wish I knew why. Guess I'll re-enable IPv6 NAT? (how that makes sense, I am not sure)

              textdump.tar.0

              info.0

              The backtrace here looks close to https://redmine.pfsense.org/issues/13938 -- close enough I'd try the workaround there to see if it helps.

              @dem said in page faults / kernel panics after config change w/23.01:

              I also had a page fault somewhere in IPv6. I too use Multi-WAN and I use NAT for IPv6.

              At the time of the crash IPv6 was failed over to my backup gateway due to the issue I reported in #12947.

              textdump.tar.0
              info.0

              This looks close to https://redmine.pfsense.org/issues/14077 but there isn't a fix available for that one yet.

              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

              Need help fast? Netgate Global Support!

              Do not Chat/PM for help!

              demD 1 Reply Last reply Reply Quote 0
              • demD
                dem @jimp
                last edited by

                @jimp Thanks Jim, I'll keep an eye on that one.

                1 Reply Last reply Reply Quote 0
                • I
                  inferno480
                  last edited by

                  thanks jimp, I added that tunable... do I need to reboot for it to "take effect" or is it instant?

                  jimpJ 1 Reply Last reply Reply Quote 0
                  • jimpJ
                    jimp Rebel Alliance Developer Netgate @inferno480
                    last edited by

                    @inferno480 said in page faults / kernel panics after config change w/23.01:

                    thanks jimp, I added that tunable... do I need to reboot for it to "take effect" or is it instant?

                    It should be immediate

                    Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                    Need help fast? Netgate Global Support!

                    Do not Chat/PM for help!

                    1 Reply Last reply Reply Quote 0
                    • I
                      inferno480 @inferno480
                      last edited by

                      @inferno480

                      reading through the redmines that were linked... I wonder if there's anything to the fact that I just recently disabled NAT I was inadvertently doing on IPv6, then within a day I started having the crashes. seems almost too coincidental. (I have not re-enabled NAT on v6 yet and probably won't unless I have another crash in which case I may just downgrade temporarily)

                      1 Reply Last reply Reply Quote 0
                      • I
                        inferno480
                        last edited by

                        almost 3 days of uptime, no page faults so far!

                        1 Reply Last reply Reply Quote 1
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.