• Reporting Lock Order Reversal (LOR) Backtraces

    Pinned Moved
    10
    1 Votes
    10 Posts
    2k Views
    P

    FYI, seeing these on the latest 2.7.0-BETA snapshot build from today (06/21/2023).

    This is a google fiber system with no VLANs set using genuine Intel i210 NICs. WAN interface is DHCP/DHCP6 and the LAN interface is tracking DHCP6 on WAN. IPv6 still works great after the message, just wanted to post here as an FYI in case it helps the devs. This same lock order message pops up randomly between 20 minutes and 4 hours after the system has booted.

    Jun 21 10:46:41 kernel #16 0xffffffff8129035e at fork_trampoline+0xe Jun 21 10:46:41 kernel #15 0xffffffff80ca6cf0 at fork_exit+0x80 Jun 21 10:46:41 kernel #14 0xffffffff80d3c7a2 at gtaskqueue_thread_loop+0xc2 Jun 21 10:46:41 kernel #13 0xffffffff80d3c977 at gtaskqueue_run_locked+0xa7 Jun 21 10:46:41 kernel #12 0xffffffff80e55bda at _task_fn_rx+0x7a Jun 21 10:46:41 kernel #11 0xffffffff80e5bbb9 at iflib_rxeof+0xe59 Jun 21 10:46:41 kernel #10 0xffffffff80e3b1c9 at ether_input+0x99 Jun 21 10:46:41 kernel #9 0xffffffff80e6022d at netisr_dispatch_src+0xad Jun 21 10:46:41 kernel #8 0xffffffff80e3c359 at ether_nh_input+0x349 Jun 21 10:46:41 kernel #7 0xffffffff80e3adba at ether_demux+0x17a Jun 21 10:46:41 kernel #6 0xffffffff80e6039d at netisr_dispatch_src+0x21d Jun 21 10:46:41 kernel #5 0xffffffff80f6c227 at ip6_input+0xc77 Jun 21 10:46:41 kernel #4 0xffffffff80f521b6 at icmp6_input+0x9e6 Jun 21 10:46:41 kernel #3 0xffffffff80f81bf1 at nd6_na_input+0x981 Jun 21 10:46:41 kernel #2 0xffffffff80f84fc1 at defrouter_remove+0x41 Jun 21 10:46:41 kernel #1 0xffffffff80cea087 at _rw_wlock_cookie+0x67 Jun 21 10:46:41 kernel #0 0xffffffff80d615e3 at witness_checkorder+0xbb3 Jun 21 10:46:41 kernel lock order lle -> nd6 list attempted at: Jun 21 10:46:41 kernel #17 0xffffffff80d3c7a2 at gtaskqueue_thread_loop+0xc2 Jun 21 10:46:41 kernel #16 0xffffffff80d3c977 at gtaskqueue_run_locked+0xa7 Jun 21 10:46:41 kernel #15 0xffffffff80e55bda at _task_fn_rx+0x7a Jun 21 10:46:41 kernel #14 0xffffffff80e5bbb9 at iflib_rxeof+0xe59 Jun 21 10:46:41 kernel #13 0xffffffff80e3b1c9 at ether_input+0x99 Jun 21 10:46:41 kernel #12 0xffffffff80e6022d at netisr_dispatch_src+0xad Jun 21 10:46:41 kernel #11 0xffffffff80e3c359 at ether_nh_input+0x349 Jun 21 10:46:41 kernel #10 0xffffffff80e3adba at ether_demux+0x17a Jun 21 10:46:41 kernel #9 0xffffffff80e6039d at netisr_dispatch_src+0x21d Jun 21 10:46:41 kernel #8 0xffffffff80f6c227 at ip6_input+0xc77 Jun 21 10:46:41 kernel #7 0xffffffff80f52068 at icmp6_input+0x898 Jun 21 10:46:41 kernel #6 0xffffffff80f84445 at nd6_ra_input+0x1335 Jun 21 10:46:41 kernel #5 0xffffffff80f7d858 at nd6_cache_lladdr+0x858 Jun 21 10:46:41 kernel #4 0xffffffff80f854ac at defrouter_select_fib+0xdc Jun 21 10:46:41 kernel #3 0xffffffff80f7c023 at nd6_lookup+0x83 Jun 21 10:46:41 kernel #2 0xffffffff80f5a29b at in6_lltable_lookup+0x10b Jun 21 10:46:41 kernel #1 0xffffffff80cead92 at __rw_rlock_int+0x82 Jun 21 10:46:41 kernel #0 0xffffffff80d60d2b at witness_checkorder+0x2fb Jun 21 10:46:41 kernel lock order nd6 list -> lle established at: Jun 21 10:46:41 kernel 2nd 0xfffffe0021de8460 nd6 list (nd6 list, rw) @ /var/jenkins/workspace/pfSense-CE-snapshots-2_7_0-main/sources/FreeBSD-src-RELENG_2_7_0/sys/netinet6/nd6_rtr.c:864 Jun 21 10:46:41 kernel 1st 0xfffff8000218c410 lle (lle, rw) @ /var/jenkins/workspace/pfSense-CE-snapshots-2_7_0-main/sources/FreeBSD-src-RELENG_2_7_0/sys/net/if_llatbl.c:370 Jun 21 10:46:41 kernel lock order reversal:
  • Crash error on RC 2.8.0 update

    Moved
    2
    0 Votes
    2 Posts
    102 Views
    stephenw10S

    You may have upgraded just as we were switching the repos to RC and hence could not access the pkgs after upgrade.

    The patches tab is part of the System Patches package, it look like that also failed to reinstall.

    Those PHP errors are normal during the upgrade and are usually removed/hidden. Something during your upgrade prevented that happening. I would assume the same repo access problem.

    Are you still seeing issues? Are you able to retest upgrading?

    Steve

  • No logs of pppoe process

    22
    0 Votes
    22 Posts
    424 Views
    N

    @stephenw10 said in No logs of pppoe process:

    You can check it using: pppcfg pppoe0

    That will return nothing if it's not an if_pppoe interface.

    Based on that I misunderstood that still I should get nothing when running kernel too.
    Thanks for clearing it up

  • PHP Fatal Error Kea2Unbound

    16
    0 Votes
    16 Posts
    463 Views
    GertjanG

    @marcosm said in PHP Fatal Error Kea2Unbound:

    When pfBlockerNG is in unbound mode the records it adds are considered local.

    Thanks for insisting on that one.
    I had to switch back to the old "Unbound mode" = one big DNSBL file, read into unbound during startup, to see what the difference is.
    All these DNSBL entries are now considered local .... and indeed, the file became pretty big.

    I really thought that the 'Unbound mode' wasn't used anymore, as 'way better' = 'Python mode' exists.

    Thanks for the clarification.

  • 0 Votes
    9 Posts
    331 Views
    M

    So, the weird thing is that with the original 2.7.2 pfSense guest, the multicast packets traversed the Proxmox linux bridge fine. In the 2.8.0 pfSense guest, they do not get through the linux bridge.

    My multicast setup has IGMP v3/MLD v2 enabled on all my HP V1910 switches. The central switch runs the IGMP/MLD queriers. This switch is connected to the Proxmox host which runs the standard linux bridge setup. pfSense is plugged into that bridge via virtio network interfaces (one for each VLAN).

    I'm happy to run without snooping on the Proxmox bridge and have set the following in /etc/network/interfaces on the Proxmox host to turn off snooping permanently.

    bridge-mcsnoop 0

    But I'm none the wiser why the 2.7.2 system was fine but the 2.8.0 system is not.

  • Excessive bogon leading to interface down

    Moved
    6
    0 Votes
    6 Posts
    177 Views
    D

    @dennypage

    @dennypage said in Excessive bogon leading to interface down:

    Arpwatch has two ways to deal with this. The first way is the -n flag, which allows adding additional local networks. This would make sense when using a virtual IP addresses, but this option is not exposed in the pfSense Arpwatch package interface. The second way is the -N option, which disables all bogon reporting. This option is exposed in the package as "Disable bogons", and I would always recommend enabling this option.

    Great info, which i wish was in the UI to be honest. It's not clear how "bogons" are interpreted for internal network interfaces.

  • Inconsistent IPv6 gateway behavior after prefix change

    3
    0 Votes
    3 Posts
    208 Views
    M

    Thank you for the assistance with testing. The issue should be addressed as part of https://redmine.pfsense.org/issues/16180.

  • When will 2.8 dev snapshot be available

    38
    0 Votes
    38 Posts
    3k Views
    N

    @louis2 said in When will 2.8 dev snapshot be available:

    I also wonder why there is nu beta download !!

    I planned to install the beta in a VM today so looking for the beta download .... no where ......

    And for what reason ????

    Just download installer, run it, and chose pfsense 2.8 beta from the menu. It is that simple.

  • where is the beta changelog?

    3
    0 Votes
    3 Posts
    174 Views
    stephenw10S

    You can also check the github history: https://github.com/pfsense/pfsense/commits/master/

  • 2.7.2 to 2.8 beta Upgrade Fails [Solved]

    2
    1 Votes
    2 Posts
    190 Views
    A

    Responding to myself here. It was a case of user error - I was simply too impatient.

    I ran the same upgrade on a different system where I could watch the upgrade process on a console (an APU). I saw that after the browser session said "Success" and the system reboots, then starts a complete reinstallation, some steps taking considerable time (like "Skipping untrusted certificates"). At the end of that process, another reboot, and success!

    So lesson learned - have patience. Lots of it.

  • Hetzner KVM missing lan after upgrading to CE 2.8.0.b.20250414.1837

    5
    0 Votes
    5 Posts
    267 Views
    stephenw10S

    I mean the qemu conf file for the VM which I expect to look pretty much the same in any host OS. Like:

    root@pve:~# cat /etc/pve/qemu-server/119.conf args: -global virtio-net-pci.disable-legacy=on -global virtio-scsi-pci.disable-legacy=on -global virtio-ballon-pci.disable-legacy=on balloon: 1024 boot: order=scsi0;ide2;net0 cores: 1 ide2: local:iso/netgate-installer-v1.1-DEVELOPMENT-amd64-20250318-0600.iso,media=cdrom,size=1062414K machine: q35,viommu=virtio memory: 2048 meta: creation-qemu=7.2.0,ctime=1713818950 name: pfSense10 net0: virtio=8A:1C:48:8A:25:E7,bridge=vmbr0,firewall=1 net1: virtio=BC:24:11:CC:61:EA,bridge=vmbr1,firewall=1 numa: 0 ostype: l26 parent: snap3 scsi0: HDIMG:119/vm-119-disk-0.qcow2,iothread=1,size=10G scsihw: virtio-scsi-single serial0: socket smbios1: uuid=ac41daa9-526a-42b9-84d4-0904a44b7468 sockets: 1 vmgenid: 72cd897c-0186-4e1d-aaae-f41810d575d7

    It does look like we might be on to something upstream though. But first we need to replicate the failure which is proving difficult.

  • Connectivity lost after clearing ARP table [Solved]

    13
    0 Votes
    13 Posts
    590 Views
    stephenw10S

    Edited. Thanks for testing!

  • Diagnostics / States

    8
    0 Votes
    8 Posts
    262 Views
    S

    @jimp

    Ok. Yes, i had the patches package installed.

    Thanks for the help.

  • Slow Diagnostics \ DNS Lookup

    4
    0 Votes
    4 Posts
    154 Views
    GertjanG

    @badders

    Try helping your pfSense by informing it :

    66fb872a-8555-4dac-8707-5dd20e8790de-image.png

  • New if_pppoe backend

    11
    1 Votes
    11 Posts
    552 Views
    K

    @TampertK RSS pins flows to specific CPU cores, so if you're testing with a single flow this is what you'd expect to see.
    Repeat your test with more flows and you'll see more of your CPU cores doing work.

  • 0 Votes
    20 Posts
    590 Views
    U

    packetcapture-vtnet0-20250411155701.pcap -> contains a prefix delegation change initiated by a WAN interface save&apply. You should see the request from the pfSense and the answer from the upstream internet router in this capture file.

    IPv6 prefix assigned to upstream internet router by internet provider: 2003:e2:8703:e000::/56
    Delegated to pfSense: 2003:e2:8703:e0f8::/61

  • KEA. Same IP, multiple reservations for IPv6

    5
    0 Votes
    5 Posts
    188 Views
    T

    @marcosm Thanks.
    Yes, I have tried doing an override in the custom config of unbound but the result is as you say. Did a DNS Lookup from the GUI and it showed only a CNAME. From a client the query failed.

    I will look into making a request.

  • "Could not extract fullbogons-ipv6.txt"

    9
    0 Votes
    9 Posts
    248 Views
    A

    @marcosm I manually edited rc.update_bogons.sh to apply the two changesets shown in the redmine. (I first manually applied the first changeset, then I manually applied the one with "additional improvements", which was clearly an edit on top of the first one.)

    Both the bogonsv4 and bogonsv6 now update correctly. Here are my logs:

    Apr 8 17:33:40 root 62500 rc.update_bogons.sh is ending the update cycle. Apr 8 17:33:40 root 61824 Bogons V6 file downloaded: 150898 addresses added. Apr 8 17:33:38 root 55652 Bogons V4 file downloaded: 2807 addresses added. Apr 8 17:33:27 root 42097 rc.update_bogons.sh is beginning the update cycle. Apr 8 17:33:27 root 40810 rc.update_bogons.sh is starting up.

    Thanks!

  • Opensource 2.8 Dead...

    20
    0 Votes
    20 Posts
    6k Views
    JeGrJ

    @bimmerdriver Your whole argument is going in the same circles. You don't see new snapshot -> you say it's dead. You simply ignore that there was and is work done with 2.8. The screen above simply shows that. That public builds of the snapshot stopped because it's no time to test them while being unstable is no proof for your "it's dead" circle. And there were multiple patches and fixes for 2.7.2 afterwards via system patches. Simply repeating stuff over and over doesn't make it more true 🤷

    @Patch said in Opensource 2.8 Dead...:

    I do not doubt Netgate only focus on the close source software branch now. Describing the CE as a split off from Plus is not really what Redmine shows

    It's exactly what they outlined in their blog about Plus as it was introduced. In the past the so-called factory edition (the Plus before there was the new naming) already existed but was dependend on CE being ready, then after it being ready and frozen it could then be fork/merged over to the FE branch and the custom things from the factory edition could be integrated. That intention was changed with Plus and communicated. With faster release cycles, now Plus is the "main" release and CE gets forked when plus is in a specific and stable state and without the additional plus-thingies. If they wouldn't work on a CE anymore you couldn't up/downgrade cleanly between Plus & CE, a thing they just introduced last year and would be stupid to drop already as it's in their own intent to keep that ability. That's why we already had two? three? joint-releases where Plus and CE followed after each other to have the same config revision and allow easy up/downgrades as needed.

    And merging changes from a branch brings merge conflicts to resolve etc. that's just normal so you don't make that even more complicated by actually doing things twice or thrice if you don't have to. So the simples thing is: be done with one release and branch (plus) so the second one (CE) can be merged over, conflicts solved, then you have a state that is actually testable and won't just break by incompatible changes from OS updates, PHP updates etc etc. Simple development logic.

    Cheers

  • When will the CE 2.8.0 Development Snapshot be available?

    12
    0 Votes
    12 Posts
    2k Views
    S

    @Viper_Rus said in When will the CE 2.8.0 Development Snapshot be available?:

    I'll wait for the CE 2.8 snapshot ;)

    Have CE as snapshot again would be awesome!

Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.