Routing traffic through PPPOE interfaces



  • My configuration is based on vlan interfaces:
     WAN (wan)                 -> re0_vlan5  -> 85.xx.xx.xx
     LAN (lan)                 -> re0_vlan10 -> 192.168.100.100
     LANP (opt1)             -> re0_vlan2  -> 192.168.5.1
     WANT (opt2)             -> re0_vlan3  -> 85.xx.xx.xx
     DSL (opt3)                -> pppoe3     -> NONE (PPPoE)

    BTW: status of PPPOE on web interface is CONNECTED with assigned ip address.
    Traffic can't be routed through any pppoe interface.
    Name of dynamic gateways sometimes are shown with GW_ prefix, sometimens only interface name

    I have 3 installations with latest builds and 3 same problems.
    I think this problem is not related with vlans because one installation does not use vlans and still have same problem.
    Maybe user GoldServe has same problem but different manifestation. It seems to me it is related.
    In my logs I can only find few instances :
    php: : The gateway: opt3 is invalid/unkown not using it.

    Routing table that is referencing pppoe link is:
    xx.178.192.1 link#11 UH 0 726 1412        pppoe3
    xx.178.192.220 link#11 UHS 0 0 16384 lo0
    First one is dynamic gateway
    Why is it on lo0 interface. Should it be on re0_vlan4 ?



  • not enough info.



  • Little guidance would be helpful. What to look for?

    This is output of route monitor : if# 11 is single pppoe interface
    got message of size 96 on Wed Jun  9 12:24:33 2010
    RTM_IFINFO: iface status change: len 96, if# 11, link: unknown, flags:<up,ptp,running,noarp,simplex,multicast></up,ptp,running,noarp,simplex,multicast>

    On web interface same interface is marked as "Online"  Status -> Gateways

    Still noting passes through that interface. I have setup filtering rule on LAN interface to use that interface as a gateway for all outgoing traffic.
    If I change gateway (we have 2 other permanent links), everything works fine. It is not DNS related problem because I tried to access IPs too.



  • I think there is some problem with routing through dynamic gateways.
    Here some more info:

    2.0-BETA2
    built on Tue Jun 8 02:45:24 EDT 2010  I386

    *** Welcome to pfSense 2.0-BETA2-pfSense (i386) on ns1 ***

    WAN (wan)                 -> re0_vlan5  -> 88.114.48.110
     LAN (lan)                 -> re0_vlan10 -> 192.168.100.100
     LANPGM (opt1)             -> re0_vlan2  -> 192.168.5.1
     WANTMP (opt2)             -> re0_vlan3  -> 88.114.55.134
     DSL (opt3)                -> pppoe3     -> NONE (PPPoE)

    ifconfig -a

    re0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=389b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_ucast,wol_mcast,wol_magic>ether 00:1c:c0:d0:de:8d
    inet6 fe80::21c:c0ff:fed0:de8d%re0 prefixlen 64 scopeid 0x1
    nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    pfsync0: flags=0<> metric 0 mtu 1460
    syncpeer: 224.0.0.240 maxupd: 128
    enc0: flags=0<> metric 0 mtu 1536
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
    options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
    nd6 options=3 <performnud,accept_rtadv>pflog0: flags=100 <promisc>metric 0 mtu 33200
    re0_vlan2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=3 <rxcsum,txcsum>ether 00:1c:c0:d0:de:8d
    inet6 fe80::21c:c0ff:fed0:de8d%re0_vlan2 prefixlen 64 scopeid 0x6
    inet 192.168.5.1 netmask 0xffffff00 broadcast 192.168.5.255
    nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    vlan: 2 parent interface: re0
    re0_vlan3: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=3 <rxcsum,txcsum>ether 00:1c:c0:d0:de:8d
    inet6 fe80::21c:c0ff:fed0:de8d%re0_vlan3 prefixlen 64 scopeid 0x7
    inet 88.114.55.134 netmask 0xfffffffc broadcast 85.114.55.135
    nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    vlan: 3 parent interface: re0
    re0_vlan4: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=3 <rxcsum,txcsum>ether 00:1c:c0:d0:de:8d
    inet6 fe80::21c:c0ff:fed0:de8d%re0_vlan4 prefixlen 64 scopeid 0x8
    nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    vlan: 4 parent interface: re0
    re0_vlan5: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=3 <rxcsum,txcsum>ether 00:1c:c0:d0:de:8d
    inet6 fe80::21c:c0ff:fed0:de8d%re0_vlan5 prefixlen 64 scopeid 0x9
    inet 88.114.48.110 netmask 0xfffffffc broadcast 85.114.48.111
    nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    vlan: 5 parent interface: re0
    re0_vlan10: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    options=3 <rxcsum,txcsum>ether 00:1c:c0:d0:de:8d
    inet6 fe80::21c:c0ff:fed0:de8d%re0_vlan10 prefixlen 64 scopeid 0xa
    inet 192.168.100.100 netmask 0xffffff00 broadcast 192.168.100.255
    nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    vlan: 10 parent interface: re0
    pppoe3: flags=88d1 <up,pointopoint,running,noarp,simplex,multicast>metric 0 mtu 1412
    inet 99.178.192.220 –> 99.178.192.1 netmask 0xffffffff
    inet6 fe80::21c:c0ff:fed0:de8d%pppoe3 prefixlen 64 scopeid 0xb
    nd6 options=3 <performnud,accept_rtadv># netstat -rn
    Routing tables

    Internet:
    Destination        Gateway            Flags    Refs      Use  Netif Expire
    default            88.114.48.109      UGS         0        6 re0_vl
    88.114.48.108/30   link#9             U           4       53 re0_vl
    88.114.48.110      link#9             UHS         0        0    lo0
    88.114.55.132/30   link#7             U           4       53 re0_vl
    88.114.55.134      link#7             UHS         0        0    lo0
    99.178.192.1       link#11            UH          0     6794 pppoe3
    99.178.192.220     link#11            UHS         0        0    lo0
    127.0.0.1          link#4             UH          0       23    lo0
    127.0.0.2          127.0.0.1          UHS         0        0    lo0
    192.168.5.0/24     link#6             U           0   901976 re0_vl
    192.168.5.1        link#6             UHS         0        0    lo0
    192.168.100.0/24   link#10            U           4   155288 re0_vl
    192.168.100.100    link#10            UHS         0        0    lo0
    216.34.181.60      88.114.55.133      UGHS        2     6439 re0_vl

    I can ping gateway of PPPOE connection:

    ping 99.178.192.1

    PING 99.178.192.1 (99.178.192.1): 56 data bytes
    64 bytes from 99.178.192.1: icmp_seq=0 ttl=254 time=3.379 ms
    64 bytes from 99.178.192.1: icmp_seq=1 ttl=254 time=3.044 ms

    Packet Capture shows gateway pings and TCP initialization in one direction (from my host to web pages). Noting is captured in opposite direction or logged in the firewall logs.

    13:13:03.026818 AF IPv4 (2), length 64: (tos 0x0, ttl 64, id 33784, offset 0, flags [DF], proto TCP (6), length 60)
        192.168.100.50.42413 > 209.62.12.163.80: Flags , cksum 0x16ef (correct), seq 1293308779, win 5840, options [mss 1460,sackOK,TS val 7524499 ecr 0,nop,wscale 7], length 0
    13:13:03.037551 AF IPv4 (2), length 64: (tos 0x0, ttl 64, id 22037, offset 0, flags [DF], proto TCP (6), length 60)
        192.168.100.50.43462 > 140.239.191.10.80: Flags , cksum 0x618a (correct), seq 1245157756, win 5840, options [mss 1460,sackOK,TS val 7524500 ecr 0,nop,wscale 7], length 0
    13:13:03.291553 AF IPv4 (2), length 64: (tos 0x0, ttl 64, id 33911, offset 0, flags [DF], proto TCP (6), length 60)
        192.168.100.50.39476 > 83.138.145.146.80: Flags , cksum 0x198d (correct), seq 1294816474, win 5840, options [mss 1460,sackOK,TS val 7524525 ecr 0,nop,wscale 7], length 0
    </performnud,accept_rtadv></up,pointopoint,running,noarp,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></promisc></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_ucast,wol_mcast,wol_magic></up,broadcast,running,simplex,multicast>



  • Can you show the filter rules?



  • This is setup where my local network (192.168.100.0/24) is router through pppoe connection - not working.
    Just to repeat that I have 2 other instalations (almost same but with different dsl providers) and same problem.

    FILTER RULES:
    scrub in on re0_vlan5 all max-mss 1460 fragment reassemble
    scrub in on re0_vlan10 all max-mss 1460 fragment reassemble
    scrub in on re0_vlan2 all max-mss 1460 fragment reassemble
    scrub in on re0_vlan3 all max-mss 1460 fragment reassemble
    anchor "relayd/" all
    anchor "firewallrules" all
    block drop in log all label "Default deny rule"
    block drop out log all label "Default deny rule"
    block drop in quick inet6 all
    block drop out quick inet6 all
    block drop quick proto tcp from any port = 0 to any
    block drop quick proto tcp from any to any port = 0
    block drop quick proto udp from any port = 0 to any
    block drop quick proto udp from any to any port = 0
    block drop quick from <snort2c>to any label "Block snort2c hosts"
    block drop quick from any to <snort2c>label "Block snort2c hosts"
    anchor "packageearly" all
    anchor "carp" all
    block drop in log quick proto tcp from <sshlockout>to any port = 3322 label "sshlockout"
    block drop in quick from <virusprot>to any label "virusprot overload table"
    anchor "wanbogons" all
    block drop in log quick on re0_vlan5 from <bogons>to any label "block bogon networks from WAN"
    block drop in on ! re0_vlan5 inet from 88.114.48.108/30 to any
    block drop in inet from 88.114.48.110 to any
    block drop in on re0_vlan5 inet6 from fe80::21c:c0ff:fed0:de8d to any
    block drop in log quick on re0_vlan5 inet from 10.0.0.0/8 to any label "block private networks from wan block 10/8"
    block drop in log quick on re0_vlan5 inet from 127.0.0.0/8 to any label "block private networks from wan block 127/8"
    block drop in log quick on re0_vlan5 inet from 172.16.0.0/12 to any label "block private networks from wan block 172.16/12"
    block drop in log quick on re0_vlan5 inet from 192.168.0.0/16 to any label "block private networks from wan block 192.168/16"
    block drop in on ! re0_vlan10 inet from 192.168.100.0/24 to any
    block drop in inet from 192.168.100.100 to any
    block drop in on re0_vlan10 inet6 from fe80::21c:c0ff:fed0:de8d to any
    anchor "dhcpserverLAN" all
    pass in on re0_vlan10 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
    pass in on re0_vlan10 inet proto udp from any port = bootpc to 192.168.100.100 port = bootps keep state label "allow access to DHCP server"
    pass out on re0_vlan10 inet proto udp from 192.168.100.100 port = bootps to any port = bootpc keep state label "allow access to DHCP server"
    block drop in on ! re0_vlan2 inet from 192.168.5.0/24 to any
    block drop in inet from 192.168.5.1 to any
    block drop in on re0_vlan2 inet6 from fe80::21c:c0ff:fed0:de8d to any
    anchor "opt2bogons" all
    block drop in log quick on re0_vlan3 from <bogons>to any label "block bogon networks from WANTMP"
    block drop in on ! re0_vlan3 inet from 88.114.55.132/30 to any
    block drop in inet from 88.114.55.134 to any
    block drop in on re0_vlan3 inet6 from fe80::21c:c0ff:fed0:de8d to any
    block drop in log quick on re0_vlan3 inet from 10.0.0.0/8 to any label "block private networks from wan block 10/8"
    block drop in log quick on re0_vlan3 inet from 127.0.0.0/8 to any label "block private networks from wan block 127/8"
    block drop in log quick on re0_vlan3 inet from 172.16.0.0/12 to any label "block private networks from wan block 172.16/12"
    block drop in log quick on re0_vlan3 inet from 192.168.0.0/16 to any label "block private networks from wan block 192.168/16"
    anchor "spoofing" all
    anchor "loopback" all
    pass in on lo0 all flags S/SA keep state label "pass loopback"
    pass out on lo0 all flags S/SA keep state label "pass loopback"
    anchor "firewallout" all
    pass out all flags S/SA keep state allow-opts label "let out anything from firewall host itself"
    pass out route-to (re0_vlan5 88.114.48.109) inet from 88.114.48.110 to ! 88.114.48.108/30 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
    pass out route-to (re0_vlan3 88.114.55.133) inet from 88.114.55.134 to ! 88.114.55.132/30 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
    anchor "anti-lockout" all
    pass in quick on re0_vlan2 inet from 192.168.5.11 to any flags S/SA keep state label "USER_RULE"
    pass in quick on re0_vlan2 inet from 192.168.5.0/24 to 192.168.100.111 flags S/SA keep state label "USER_RULE"
    pass in quick on re0_vlan2 inet from 192.168.5.0/24 to 192.168.100.110 flags S/SA keep state label "USER_RULE"
    pass in quick on re0_vlan2 route-to (re0_vlan3 88.114.55.133) inet proto udp from 192.168.5.0/24 to any port = domain keep state label "USER_RULE"
    pass in quick on re0_vlan2 route-to (re0_vlan3 88.114.55.133) inet proto udp from 192.168.5.0/24 to any port = ntp keep state label "USER_RULE"
    pass in quick on re0_vlan2 route-to (re0_vlan3 88.114.55.133) inet proto udp from 192.168.5.0/24 to any port = radius keep state label "USER_RULE"
    pass in quick on re0_vlan2 route-to (re0_vlan3 88.114.55.133) inet proto udp from 192.168.5.0/24 to any port = radacct keep state label "USER_RULE"
    pass in quick on re0_vlan10 inet from 192.168.100.0/24 to 192.168.5.0/24 flags S/SA keep state label "USER_RULE"
    pass in quick on re0_vlan10 inet from 192.168.100.0/24 to 192.168.100.100 flags S/SA keep state label "USER_RULE"
    pass in quick on re0_vlan10 inet proto icmp from 192.168.100.0/24 to any keep state label "USER_RULE"
    pass in quick on re0_vlan10 inet proto tcp from 192.168.100.111 to <vpns>flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
    pass in quick on re0_vlan10 route-to (re0_vlan5 88.114.48.109) inet proto tcp from 192.168.100.111 to any port = smtp flags S/SA keep state label "USER_RULE"
    pass in quick on re0_vlan10 inet proto tcp from 192.168.100.111 to <vpns>flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
    pass in quick on re0_vlan10 inet proto udp from 192.168.100.111 to <vpns>keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
    pass in quick on re0_vlan10 route-to (re0_vlan5 88.114.48.109) inet proto tcp from 192.168.100.111 to any port = domain flags S/SA keep state label "USER_RULE"
    pass in quick on re0_vlan10 route-to (re0_vlan5 88.114.48.109) inet proto udp from 192.168.100.111 to any port = domain keep state label "USER_RULE"
    pass in quick on re0_vlan10 inet from 192.168.100.0/24 to <vpns>flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
    pass in quick on re0_vlan10 route-to (pppoe3 99.178.192.1) inet from 192.168.100.0/24 to any flags S/SA keep state label "USER_RULE"
    anchor "packagelate" all
    anchor "tftp-proxy/
    " all
    anchor "limitingesr" all
    anchor "miniupnpd" all
    No queue in use</vpns></vpns></vpns></vpns></bogons></bogons></virusprot></sshlockout></snort2c></snort2c>



  • I can't belive I am the only with this problem. I use VLANs, but configuration is prety ordinary. I can see there are many ongoing changes in recent snapshots on pppoe interfaces…



  • Hi,

    Your problem is solved by this commit.
    https://rcs.pfsense.org/projects/pfsense/repos/mainline/commits/58cbe016de8cd2e88b08529f6272da849251647a

    Your PPPoE interface is last in your interface list and a code change on Jun3 caused the last interface in the list to not be recognized as an interface that really exists.

    Look at the time stamp in the commit and select an update or a new build from the snapshot server that is newer.

    GB



  • This was a big problem for me.
    God bless you :)



  • The code change is small. You can do it yourself if you do want to get a new build.



  • That did a trick even without restart. Thanks


Locked