Kernel: arpresolve: can't allocate llinfo for [PPPoE DHCP Assigned Gateway]



  • I just upgraded to one of the latest 12/30, 12/31 beta snapshots but ran into an issue where pfsense would not "see" it's dhcp-assigned WAN gateway. This results in routing-based issues like:
    1. Incoming traffic is logged (to port redirected services) on the WAN interface, but traffic never replies,
    2. Under status, gateways, the gateway is listed as down. Fiddling with the settings, as noted in another post, temporarily marks the gateway as up, but checking less than a minute later, it is down again.
    3. The system log contains multiple entries (didn't catch frequency) for "kernel: arpresolve: can't allocate llinfo for [w.x.y.z]" (actual ip address removed, but it's the gateway assigned by pppoe).
    4. This issue exhibits itself right away, one install was a "fresh" install, prior to restoring the 1.2.3 backup.

    I had to revert to a working config (1.2.3) as I ran out of time… on 1.2.3 this issue doesn't present itself.
    Config: running on an Atom M510, used the AMD64 builds from 12/30 & 12/31, DSL with PPPoE (bge0 WAN, re0 LAN). Note that this is a simple two-port config. Traffic shaper is enabled.
    I may try the i386 builds if I have time later.

    Has anyone run into this before?



  • A forum search for llinfo showed a number of threads. Some of them might give some clues as to what is happening. In particular, http://forum.pfsense.org/index.php/topic,29797.0.html might be of interest.



  • This would indicate a driver issue. I dry-ran an upgrade in vmware and it passed traffic right away… I'll look into playing with the setting or replacing the card (the on-board Realtek card in this case is just fine; it's the old broadcom that was added that's being fussy) and post the results. Thanks for the reference.



  • Well, I spent time this afternoon looking at this. I believe the 2.0 build or underlying O/S is not responding properly to the PPPoE config handed to it from the DSL modem (not router). It is failing to see the logical hop presented by the modem to get to the gateway it sends to the pfsense box.

    I began troubleshooting this as a device driver issue, however I ended up switching the interface connections and swapping out the NIC in the expansion slot. I also tried both AMD64/i386 builds. Always with the same result.

    After restoring to 1.2.3 I see that more routes are present. Here's what I see happening in this version:

    1. Modem has ethernet connection to pfsense. It's address is 192.168.1.1.
    2. The pfsense WAN gets 192.168.1.2 from the modem.
    3. PPPoE magic occurs, handing the pfsense box the "real" public IP address. It also hands a "gateway" address to pfsense. I don't know where that gateway is, either the modem or the first hop at the ISP…
    4. All is good. A traceroute shows the first hop after the pfsense box is 192.168.1.1, then the "gateway", and then so on to destination.

    In 2.0, somehow the connection to 192.168.1.1 is lost, resulting in llinfo error and pings to the "gateway" resulting in a "no route to host" error. This appears to be a routing only issue as the firewall logs show the connections being initiated/logged.

    Right now, this is my only connection to the Internet so every time I test this, I'm out of pocket for a while.

    Any suggestions as to what's going on and how to resolve? Thanks again.

    DSL Modem: D-Link 2320B.



  • Suggestion: compare ppp log and output from shell command ifconfig -a; netstat -rn on both pfSense 1.2.3 and 2.0. If you post them here, identify which interface is LAN and which is WAN and clearly identify which pfSense version produced the output then you'll have a few more eyes able to scan them looking for clues.



  • Here's the comparison of routes (I'm booting off CD for the 2.0 release now, so I can quickly go back to 1.2.3…) Note the default rules on 1.2.3 pass traffic right away (after setting the LAN ip...)

    [I've also attached a BeyondCompare generated .html(.txt added) comparison for an easier read]

    Version 1.2.3

    ifconfig -a

    re0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=389b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_ucast,wol_mcast,wol_magic>ether 00:27:0e:09:fb:1d
    inet 192.168.10.1 netmask 0xffffff80 broadcast 192.168.10.127
    inet6 fe80::227:eff:fe09:fb1d%re0 prefixlen 64 scopeid 0x1
    media: Ethernet autoselect (1000baseTX <full-duplex>)
    status: active
    fxp0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=8 <vlan_mtu>ether 00:02:b3:bd:0e:e2
    inet6 fe80::202:b3ff:febd:ee2%fxp0 prefixlen 64 scopeid 0x2
    inet 63.224.3.77 netmask 0xffffff00 broadcast 63.224.3.255
    media: Ethernet autoselect (100baseTX <full-duplex>)
    status: active
    fxp1: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500
    options=8 <vlan_mtu>ether 00:02:b3:bd:0e:e3
    media: Ethernet autoselect (none)
    status: no carrier
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
    inet 127.0.0.1 netmask 0xff000000
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
    enc0: flags=0<> metric 0 mtu 1536
    pfsync0: flags=41 <up,running>metric 0 mtu 1460
    pfsync: syncdev: lo0 syncpeer: 224.0.0.240 maxupd: 128
    pflog0: flags=100 <promisc>metric 0 mtu 33204
    tun0: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
    inet6 fe80::227:eff:fe09:fb1d%tun0 prefixlen 64 scopeid 0x8
    inet 192.168.10.193 –> 192.168.10.194 netmask 0xffffffff
    Opened by PID 459

    netstat -rn

    Routing tables

    Internet:
    Destination        Gateway            Flags    Refs      Use  Netif Expire
    default            link#2            UCS        0      533  fxp0
    8.21.4.203        1c:af:f7:af:61:a7  UHLW        1        6  fxp0  1083
    63.224.3.0/24      link#2            UC          0        0  fxp0
    63.224.3.77        127.0.0.1          UGHS        0        0    lo0
    65.55.17.39        1c:af:f7:af:61:a7  UHLW        1        5  fxp0    345
    65.55.53.190      1c:af:f7:af:61:a7  UHLW        1      11  fxp0  1073
    67.215.65.132      1c:af:f7:af:61:a7  UHLW        1      614  fxp0    301
    69.64.6.21        1c:af:f7:af:61:a7  UHLW        1      261  fxp0  1079
    70.86.250.6        1c:af:f7:af:61:a7  UHLW        1      10  fxp0    117
    72.14.204.100      1c:af:f7:af:61:a7  UHLW        1        7  fxp0  1079
    72.14.204.148      1c:af:f7:af:61:a7  UHLW        1        4  fxp0  1096
    72.14.204.149      1c:af:f7:af:61:a7  UHLW        1        5  fxp0  1096
    72.14.204.155      1c:af:f7:af:61:a7  UHLW        1      28  fxp0  1080
    72.14.204.167      1c:af:f7:af:61:a7  UHLW        1        3  fxp0  1079
    72.246.30.58      1c:af:f7:af:61:a7  UHLW        1      48  fxp0  1084
    72.246.30.65      1c:af:f7:af:61:a7  UHLW        1        4  fxp0  1083
    74.120.140.11      1c:af:f7:af:61:a7  UHLW        1      13  fxp0  1095
    74.202.103.146    1c:af:f7:af:61:a7  UHLW        1    12214  fxp0  1055
    74.202.103.191    1c:af:f7:af:61:a7  UHLW        1        2  fxp0  1031
    96.8.83.121        1c:af:f7:af:61:a7  UHLW        1      69  fxp0    236
    96.8.85.121        1c:af:f7:af:61:a7  UHLW        1      33  fxp0    836
    127.0.0.1          127.0.0.1          UH          1        0    lo0
    174.136.20.151    1c:af:f7:af:61:a7  UHLW        1      14  fxp0    252
    184.84.77.15      1c:af:f7:af:61:a7  UHLW        1        6  fxp0    252
    184.84.247.50      1c:af:f7:af:61:a7  UHLW        1        6  fxp0  1096
    184.84.247.66      1c:af:f7:af:61:a7  UHLW        1        7  fxp0    252
    192.168.1.1        1c:af:f7:af:61:a7  UHLW        1    3151  fxp0  1153
    192.168.10.0/25    link#1            UC          0        0    re0
    192.168.10.2      00:16:cb:c5:94:e7  UHLW        1      205    re0  1003
    192.168.10.10      00:1a:4d:4e:1e:d1  UHLW        1    55494    re0  1199
    192.168.10.11      00:0c:29:dd:71:4c  UHLW        1      377    re0  1184
    192.168.10.12      00:0c:29:12:4a:22  UHLW        1    7771    re0  1199
    192.168.10.13      00:0c:29:9f:78:a1  UHLW        1    3263    re0    351
    192.168.10.20      70:71:bc:bd:c2:d4  UHLW        1    9253    re0  1192
    192.168.10.69      00:16:ea:8e:66:92  UHLW        1    15000    re0  1190
    192.168.10.77      00:e0:91:82:f8:22  UHLW        1      375    re0  1079
    192.168.10.83      00:17:f2:ce:e8:b9  UHLW        1  2209290    re0    397
    192.168.10.192/26  192.168.10.194    UGS        0        0  tun0
    192.168.10.194    192.168.10.193    UH          1        0  tun0
    204.235.61.9      1c:af:f7:af:61:a7  UHLW        1        1  fxp0  1012
    207.46.193.176    1c:af:f7:af:61:a7  UHLW        1      17  fxp0  1081
    207.171.7.151      1c:af:f7:af:61:a7  UHLW        1        6  fxp0  1148
    207.225.140.233    1c:af:f7:af:61:a7  UHLW        1      24  fxp0    80
    207.225.140.239    1c:af:f7:af:61:a7  UHLW        1        0  fxp0    686
    208.67.222.222    1c:af:f7:af:61:a7  UHLW        1    5671  fxp0  1094
    208.69.32.136      1c:af:f7:af:61:a7  UHLW        1      31  fxp0  1020

    Internet6:
    Destination                      Gateway                      Flags      Netif Expire
    ::1                              ::1                          UHL        lo0
    fe80::%re0/64                    link#1                        UC          re0
    fe80::227:eff:fe09:fb1d%re0      00:27:0e:09:fb:1d            UHL        lo0
    fe80::%fxp0/64                    link#2                        UC        fxp0
    fe80::202:b3ff:febd:ee2%fxp0      00:02:b3:bd:0e:e2            UHL        lo0
    fe80::%lo0/64                    fe80::1%lo0                  U          lo0
    fe80::1%lo0                      link#4                        UHL        lo0
    fe80::%tun0/64                    link#8                        UC        tun0
    fe80::227:eff:fe09:fb1d%tun0      link#8                        UHL        lo0
    ff01:1::/32                      link#1                        UC          re0
    ff01:2::/32                      link#2                        UC        fxp0
    ff01:4::/32                      ::1                          UC          lo0
    ff01:8::/32                      link#8                        UC        tun0
    ff02::%re0/32                    link#1                        UC          re0
    ff02::%fxp0/32                    link#2                        UC        fxp0
    ff02::%lo0/32                    ::1                          UC          lo0
    ff02::%tun0/32                    link#8                        UC        tun0

    2.0 Beta-5-2011-01-03-i386

    [2.0-BETA5][admin@pfSense.localdomain]/root(1): ifconfig -a
    re0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=389b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_ucast,wol_mcast,wol_magic>ether 00:27:0e:09:fb:1d
    inet6 fe80::227:eff:fe09:fb1d%re0 prefixlen 64 scopeid 0x1
    inet 192.168.10.1 netmask 0xffffff80 broadcast 192.168.10.127
    nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    fxp0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=8 <vlan_mtu>ether 00:02:b3:bd:0e:e2
    inet6 fe80::202:b3ff:febd:ee2%fxp0 prefixlen 64 scopeid 0x2
    inet 63.224.3.77 netmask 0xffffff00 broadcast 63.224.3.255
    nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (100baseTX <full-duplex>)
    status: active
    fxp1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=8 <vlan_mtu>ether 00:02:b3:bd:0e:e3
    inet6 fe80::202:b3ff:febd:ee3%fxp1 prefixlen 64 scopeid 0x3
    nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (none)
    status: no carrier
    pflog0: flags=100 <promisc>metric 0 mtu 33128
    enc0: flags=0<> metric 0 mtu 1536
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
    options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
    nd6 options=3 <performnud,accept_rtadv>pfsync0: flags=0<> metric 0 mtu 1460
    syncpeer: 224.0.0.240 maxupd: 128
    ue0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=80000 <linkstate>ether 00:00:00:00:2f:9a
    inet6 fe80::200:ff:fe00:2f9a%ue0 prefixlen 64 scopeid 0x8
    nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (none)
    status: no carrier
    [2.0-BETA5][admin@pfSense.localdomain]/root(2): netstat -rn
    Routing tables

    Internet:
    Destination        Gateway            Flags    Refs      Use  Netif Expire
    63.224.3.0/24      link#2            U          0        0  fxp0
    63.224.3.77        link#2            UHS        0        0    lo0
    127.0.0.1          link#6            UH          0      562    lo0
    192.168.10.0/25    link#1            U          0    1010    re0
    192.168.10.1      link#1            UHS        0        0    lo0
    205.171.2.65      00:02:b3:bd:0e:e2  UHS        0        3  fxp0
    205.171.3.65      00:02:b3:bd:0e:e2  UHS        0        4  fxp0

    Internet6:
    Destination                      Gateway                      Flags      Netif Expire
    ::1                              ::1                          UH          lo0
    fe80::%re0/64                    link#1                        U          re0
    fe80::227:eff:fe09:fb1d%re0      link#1                        UHS        lo0
    fe80::%fxp0/64                    link#2                        U          fxp0
    fe80::202:b3ff:febd:ee2%fxp0      link#2                        UHS        lo0
    fe80::%fxp1/64                    link#3                        U          fxp1
    fe80::202:b3ff:febd:ee3%fxp1      link#3                        UHS        lo0
    fe80::%lo0/64                    link#6                        U          lo0
    fe80::1%lo0                      link#6                        UHS        lo0
    fe80::%ue0/64                    link#8                        U          ue0
    fe80::200:ff:fe00:2f9a%ue0        link#8                        UHS        lo0
    ff01:1::/32                      fe80::227:eff:fe09:fb1d%re0  U          re0
    ff01:2::/32                      fe80::202:b3ff:febd:ee2%fxp0  U          fxp0
    ff01:3::/32                      fe80::202:b3ff:febd:ee3%fxp1  U          fxp1
    ff01:6::/32                      ::1                          U          lo0
    ff01:8::/32                      fe80::200:ff:fe00:2f9a%ue0    U          ue0
    ff02::%re0/32                    fe80::227:eff:fe09:fb1d%re0  U          re0
    ff02::%fxp0/32                    fe80::202:b3ff:febd:ee2%fxp0  U          fxp0
    ff02::%fxp1/32                    fe80::202:b3ff:febd:ee3%fxp1  U          fxp1
    ff02::%lo0/32                    ::1                          U          lo0
    ff02::%ue0/32                    fe80::200:ff:fe00:2f9a%ue0    U          ue0

    pfsensecompare.html.txt</performnud,accept_rtadv></linkstate></up,broadcast,running,simplex,multicast></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></promisc></performnud,accept_rtadv></vlan_mtu></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></vlan_mtu></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_ucast,wol_mcast,wol_magic></up,broadcast,running,simplex,multicast></up,pointopoint,running,multicast></promisc></up,running></up,loopback,running,multicast></vlan_mtu></broadcast,simplex,multicast></full-duplex></vlan_mtu></up,broadcast,running,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_ucast,wol_mcast,wol_magic></up,broadcast,running,simplex,multicast>



  • The first thing I saw was that 1.2.3 has a default route but 2.0 doesn't. Hence 2.0 won't know where to forward packets destined to the Internet.

    I reported missing default route in http://forum.pfsense.org/index.php/topic,27636.0.html which suggests a workaround. The consequent discussion suggests you probably shouldn't have seen this problem, depending on where the configuration information came from (e.g. did you start with empty configuration or did you use an "old" configuration file?)

    Shouldn't there be some ppp logs and ppp interfaces shown?



  • Thanks for the reply. The behavior is exhibited using default configurations (other than setting the LAN IP, which is changed right away in each instance). I'll review the link you sent to see if there's anything that rattles lose.

    As for PPP logs, the pfsense box isn't actually doing the PPPoE, as the login-configuration is input on the D-Link DSL modem. So, as far as I can tell, the WAN port is acting like a regular DHCP client rather than anything PPPoE-ish; receiving any configuration data through the standard DHCP mechanism (again, my understanding of this is limited based only on observing the process). Both configs receive the public IP, netmask, two DNS servers, but only 1.2.3 gets/acts upon any routing info. I'm a little fuzzy on how this works exactly since the modem hands the public IP to the WAN port but the modem itself is still an IP hop. So, for example a traceroute will look something like:

    First hop = LAN IP 192.168.10.1
    Second hop = dsl modem 192.168.1.1
    third hop = some provider address 67.215.65.132
    fourth hop, etc…

    Paul



  • Are you in the UK? What you describes like ADSL Modems PPPOA half-bridge mode.

    I have two connected to my pf 2.0 box, one Billion and a BT BusHub they are always contactable via their local 192 ip, but once they sync up they re-issue (very short lease, few mins) and provide the public IP.

    I've used many 2.0 builds and all are fine but whenever i've had the llinfo error on occasion it was because I'd created a switching loop / broadcast storm somewhere. It went over my head and I got confused but when I truely isolated everthing with vlans the errors went and its ok. (Essentially plugging the modems straight into the PF box)



  • In the US. I've been connecting the router via the ethernet cable but will try via the USB connection when I have a chance. …See if pfsense as the pppoe "client" directly makes a difference.



  • I had a little time (not enough, really) do do some quick testing. Again, working from the LiveCD…

    After the system had booted & I changed the LAN IP to the appropriate value, I went into the shell (2.0 1/3/11 build, i386), and ran:

    route add default [public ip]

    I got connectivity outbound.

    So it seems it's not getting the default gateway or handling the info received correctly due to some feature specific to this DSL/PPPoE/DHCP process. In comparison, I tested this in vmware with a regular DHCP server and that config routed correctly…

    Note: I also played with attaching the DLink DSL modem directly via usb. 1.2.3 hated it (recognizing the interface as acde0 [or something, I missed writing it down…]) and 2.0 didn't recognize it.

    I'll play with this more when I have more time.



  • I've had the opportunity to test this again a couple of times. DHCP on the WAN address always receives IP address, DNS Server, but never a default gateway value. In every case manually pointing the default gateway to the WAN IP (along with setting the WAN MTU to 1492) gets traffic flowing every time. The logs I've seen (similar to captured previously) all show the effects of the issue, but not the cause. What logs should I be looking at to see the DHCP/PPPoE process not working properly vs it working correctly in 1.2.3?

    Is this really a pfSense issue or an underlying BSD issue (hardware [in]compatibility)? I suppose I could script the addition of the default gateway at boot (disappears at boot every time) but that seems backwards in light of it working flawlessly in 1.2.3.

    Are there resources someone could point me to to research this further? The PPPoE process seems straightforward (bridge/half-bridge) but there's some of this beyond me at the moment. Thanks for the help.


Locked