Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Routing traffic through PPPOE interfaces

    2.0-RC Snapshot Feedback and Problems - RETIRED
    3
    11
    3.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • I
      ivanmar
      last edited by

      My configuration is based on vlan interfaces:
       WAN (wan)                 -> re0_vlan5  -> 85.xx.xx.xx
       LAN (lan)                 -> re0_vlan10 -> 192.168.100.100
       LANP (opt1)             -> re0_vlan2  -> 192.168.5.1
       WANT (opt2)             -> re0_vlan3  -> 85.xx.xx.xx
       DSL (opt3)                -> pppoe3     -> NONE (PPPoE)

      BTW: status of PPPOE on web interface is CONNECTED with assigned ip address.
      Traffic can't be routed through any pppoe interface.
      Name of dynamic gateways sometimes are shown with GW_ prefix, sometimens only interface name

      I have 3 installations with latest builds and 3 same problems.
      I think this problem is not related with vlans because one installation does not use vlans and still have same problem.
      Maybe user GoldServe has same problem but different manifestation. It seems to me it is related.
      In my logs I can only find few instances :
      php: : The gateway: opt3 is invalid/unkown not using it.

      Routing table that is referencing pppoe link is:
      xx.178.192.1 link#11 UH 0 726 1412        pppoe3
      xx.178.192.220 link#11 UHS 0 0 16384 lo0
      First one is dynamic gateway
      Why is it on lo0 interface. Should it be on re0_vlan4 ?

      1 Reply Last reply Reply Quote 0
      • E
        eri--
        last edited by

        not enough info.

        1 Reply Last reply Reply Quote 0
        • I
          ivanmar
          last edited by

          Little guidance would be helpful. What to look for?

          This is output of route monitor : if# 11 is single pppoe interface
          got message of size 96 on Wed Jun  9 12:24:33 2010
          RTM_IFINFO: iface status change: len 96, if# 11, link: unknown, flags:<up,ptp,running,noarp,simplex,multicast></up,ptp,running,noarp,simplex,multicast>

          On web interface same interface is marked as "Online"  Status -> Gateways

          Still noting passes through that interface. I have setup filtering rule on LAN interface to use that interface as a gateway for all outgoing traffic.
          If I change gateway (we have 2 other permanent links), everything works fine. It is not DNS related problem because I tried to access IPs too.

          1 Reply Last reply Reply Quote 0
          • I
            ivanmar
            last edited by

            I think there is some problem with routing through dynamic gateways.
            Here some more info:

            2.0-BETA2
            built on Tue Jun 8 02:45:24 EDT 2010  I386

            *** Welcome to pfSense 2.0-BETA2-pfSense (i386) on ns1 ***

            WAN (wan)                 -> re0_vlan5  -> 88.114.48.110
             LAN (lan)                 -> re0_vlan10 -> 192.168.100.100
             LANPGM (opt1)             -> re0_vlan2  -> 192.168.5.1
             WANTMP (opt2)             -> re0_vlan3  -> 88.114.55.134
             DSL (opt3)                -> pppoe3     -> NONE (PPPoE)

            ifconfig -a

            re0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=389b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_ucast,wol_mcast,wol_magic>ether 00:1c:c0:d0:de:8d
            inet6 fe80::21c:c0ff:fed0:de8d%re0 prefixlen 64 scopeid 0x1
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
            pfsync0: flags=0<> metric 0 mtu 1460
            syncpeer: 224.0.0.240 maxupd: 128
            enc0: flags=0<> metric 0 mtu 1536
            lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
            options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000
            inet6 ::1 prefixlen 128
            inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
            nd6 options=3 <performnud,accept_rtadv>pflog0: flags=100 <promisc>metric 0 mtu 33200
            re0_vlan2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=3 <rxcsum,txcsum>ether 00:1c:c0:d0:de:8d
            inet6 fe80::21c:c0ff:fed0:de8d%re0_vlan2 prefixlen 64 scopeid 0x6
            inet 192.168.5.1 netmask 0xffffff00 broadcast 192.168.5.255
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
            vlan: 2 parent interface: re0
            re0_vlan3: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=3 <rxcsum,txcsum>ether 00:1c:c0:d0:de:8d
            inet6 fe80::21c:c0ff:fed0:de8d%re0_vlan3 prefixlen 64 scopeid 0x7
            inet 88.114.55.134 netmask 0xfffffffc broadcast 85.114.55.135
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
            vlan: 3 parent interface: re0
            re0_vlan4: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=3 <rxcsum,txcsum>ether 00:1c:c0:d0:de:8d
            inet6 fe80::21c:c0ff:fed0:de8d%re0_vlan4 prefixlen 64 scopeid 0x8
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
            vlan: 4 parent interface: re0
            re0_vlan5: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=3 <rxcsum,txcsum>ether 00:1c:c0:d0:de:8d
            inet6 fe80::21c:c0ff:fed0:de8d%re0_vlan5 prefixlen 64 scopeid 0x9
            inet 88.114.48.110 netmask 0xfffffffc broadcast 85.114.48.111
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
            vlan: 5 parent interface: re0
            re0_vlan10: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=3 <rxcsum,txcsum>ether 00:1c:c0:d0:de:8d
            inet6 fe80::21c:c0ff:fed0:de8d%re0_vlan10 prefixlen 64 scopeid 0xa
            inet 192.168.100.100 netmask 0xffffff00 broadcast 192.168.100.255
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
            vlan: 10 parent interface: re0
            pppoe3: flags=88d1 <up,pointopoint,running,noarp,simplex,multicast>metric 0 mtu 1412
            inet 99.178.192.220 –> 99.178.192.1 netmask 0xffffffff
            inet6 fe80::21c:c0ff:fed0:de8d%pppoe3 prefixlen 64 scopeid 0xb
            nd6 options=3 <performnud,accept_rtadv># netstat -rn
            Routing tables

            Internet:
            Destination        Gateway            Flags    Refs      Use  Netif Expire
            default            88.114.48.109      UGS         0        6 re0_vl
            88.114.48.108/30   link#9             U           4       53 re0_vl
            88.114.48.110      link#9             UHS         0        0    lo0
            88.114.55.132/30   link#7             U           4       53 re0_vl
            88.114.55.134      link#7             UHS         0        0    lo0
            99.178.192.1       link#11            UH          0     6794 pppoe3
            99.178.192.220     link#11            UHS         0        0    lo0
            127.0.0.1          link#4             UH          0       23    lo0
            127.0.0.2          127.0.0.1          UHS         0        0    lo0
            192.168.5.0/24     link#6             U           0   901976 re0_vl
            192.168.5.1        link#6             UHS         0        0    lo0
            192.168.100.0/24   link#10            U           4   155288 re0_vl
            192.168.100.100    link#10            UHS         0        0    lo0
            216.34.181.60      88.114.55.133      UGHS        2     6439 re0_vl

            I can ping gateway of PPPOE connection:

            ping 99.178.192.1

            PING 99.178.192.1 (99.178.192.1): 56 data bytes
            64 bytes from 99.178.192.1: icmp_seq=0 ttl=254 time=3.379 ms
            64 bytes from 99.178.192.1: icmp_seq=1 ttl=254 time=3.044 ms

            Packet Capture shows gateway pings and TCP initialization in one direction (from my host to web pages). Noting is captured in opposite direction or logged in the firewall logs.

            13:13:03.026818 AF IPv4 (2), length 64: (tos 0x0, ttl 64, id 33784, offset 0, flags [DF], proto TCP (6), length 60)
                192.168.100.50.42413 > 209.62.12.163.80: Flags , cksum 0x16ef (correct), seq 1293308779, win 5840, options [mss 1460,sackOK,TS val 7524499 ecr 0,nop,wscale 7], length 0
            13:13:03.037551 AF IPv4 (2), length 64: (tos 0x0, ttl 64, id 22037, offset 0, flags [DF], proto TCP (6), length 60)
                192.168.100.50.43462 > 140.239.191.10.80: Flags , cksum 0x618a (correct), seq 1245157756, win 5840, options [mss 1460,sackOK,TS val 7524500 ecr 0,nop,wscale 7], length 0
            13:13:03.291553 AF IPv4 (2), length 64: (tos 0x0, ttl 64, id 33911, offset 0, flags [DF], proto TCP (6), length 60)
                192.168.100.50.39476 > 83.138.145.146.80: Flags , cksum 0x198d (correct), seq 1294816474, win 5840, options [mss 1460,sackOK,TS val 7524525 ecr 0,nop,wscale 7], length 0            </performnud,accept_rtadv></up,pointopoint,running,noarp,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></promisc></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_ucast,wol_mcast,wol_magic></up,broadcast,running,simplex,multicast>

            1 Reply Last reply Reply Quote 0
            • E
              eri--
              last edited by

              Can you show the filter rules?

              1 Reply Last reply Reply Quote 0
              • I
                ivanmar
                last edited by

                This is setup where my local network (192.168.100.0/24) is router through pppoe connection - not working.
                Just to repeat that I have 2 other instalations (almost same but with different dsl providers) and same problem.

                FILTER RULES:
                scrub in on re0_vlan5 all max-mss 1460 fragment reassemble
                scrub in on re0_vlan10 all max-mss 1460 fragment reassemble
                scrub in on re0_vlan2 all max-mss 1460 fragment reassemble
                scrub in on re0_vlan3 all max-mss 1460 fragment reassemble
                anchor "relayd/" all
                anchor "firewallrules" all
                block drop in log all label "Default deny rule"
                block drop out log all label "Default deny rule"
                block drop in quick inet6 all
                block drop out quick inet6 all
                block drop quick proto tcp from any port = 0 to any
                block drop quick proto tcp from any to any port = 0
                block drop quick proto udp from any port = 0 to any
                block drop quick proto udp from any to any port = 0
                block drop quick from <snort2c>to any label "Block snort2c hosts"
                block drop quick from any to <snort2c>label "Block snort2c hosts"
                anchor "packageearly" all
                anchor "carp" all
                block drop in log quick proto tcp from <sshlockout>to any port = 3322 label "sshlockout"
                block drop in quick from <virusprot>to any label "virusprot overload table"
                anchor "wanbogons" all
                block drop in log quick on re0_vlan5 from <bogons>to any label "block bogon networks from WAN"
                block drop in on ! re0_vlan5 inet from 88.114.48.108/30 to any
                block drop in inet from 88.114.48.110 to any
                block drop in on re0_vlan5 inet6 from fe80::21c:c0ff:fed0:de8d to any
                block drop in log quick on re0_vlan5 inet from 10.0.0.0/8 to any label "block private networks from wan block 10/8"
                block drop in log quick on re0_vlan5 inet from 127.0.0.0/8 to any label "block private networks from wan block 127/8"
                block drop in log quick on re0_vlan5 inet from 172.16.0.0/12 to any label "block private networks from wan block 172.16/12"
                block drop in log quick on re0_vlan5 inet from 192.168.0.0/16 to any label "block private networks from wan block 192.168/16"
                block drop in on ! re0_vlan10 inet from 192.168.100.0/24 to any
                block drop in inet from 192.168.100.100 to any
                block drop in on re0_vlan10 inet6 from fe80::21c:c0ff:fed0:de8d to any
                anchor "dhcpserverLAN" all
                pass in on re0_vlan10 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
                pass in on re0_vlan10 inet proto udp from any port = bootpc to 192.168.100.100 port = bootps keep state label "allow access to DHCP server"
                pass out on re0_vlan10 inet proto udp from 192.168.100.100 port = bootps to any port = bootpc keep state label "allow access to DHCP server"
                block drop in on ! re0_vlan2 inet from 192.168.5.0/24 to any
                block drop in inet from 192.168.5.1 to any
                block drop in on re0_vlan2 inet6 from fe80::21c:c0ff:fed0:de8d to any
                anchor "opt2bogons" all
                block drop in log quick on re0_vlan3 from <bogons>to any label "block bogon networks from WANTMP"
                block drop in on ! re0_vlan3 inet from 88.114.55.132/30 to any
                block drop in inet from 88.114.55.134 to any
                block drop in on re0_vlan3 inet6 from fe80::21c:c0ff:fed0:de8d to any
                block drop in log quick on re0_vlan3 inet from 10.0.0.0/8 to any label "block private networks from wan block 10/8"
                block drop in log quick on re0_vlan3 inet from 127.0.0.0/8 to any label "block private networks from wan block 127/8"
                block drop in log quick on re0_vlan3 inet from 172.16.0.0/12 to any label "block private networks from wan block 172.16/12"
                block drop in log quick on re0_vlan3 inet from 192.168.0.0/16 to any label "block private networks from wan block 192.168/16"
                anchor "spoofing" all
                anchor "loopback" all
                pass in on lo0 all flags S/SA keep state label "pass loopback"
                pass out on lo0 all flags S/SA keep state label "pass loopback"
                anchor "firewallout" all
                pass out all flags S/SA keep state allow-opts label "let out anything from firewall host itself"
                pass out route-to (re0_vlan5 88.114.48.109) inet from 88.114.48.110 to ! 88.114.48.108/30 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
                pass out route-to (re0_vlan3 88.114.55.133) inet from 88.114.55.134 to ! 88.114.55.132/30 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
                anchor "anti-lockout" all
                pass in quick on re0_vlan2 inet from 192.168.5.11 to any flags S/SA keep state label "USER_RULE"
                pass in quick on re0_vlan2 inet from 192.168.5.0/24 to 192.168.100.111 flags S/SA keep state label "USER_RULE"
                pass in quick on re0_vlan2 inet from 192.168.5.0/24 to 192.168.100.110 flags S/SA keep state label "USER_RULE"
                pass in quick on re0_vlan2 route-to (re0_vlan3 88.114.55.133) inet proto udp from 192.168.5.0/24 to any port = domain keep state label "USER_RULE"
                pass in quick on re0_vlan2 route-to (re0_vlan3 88.114.55.133) inet proto udp from 192.168.5.0/24 to any port = ntp keep state label "USER_RULE"
                pass in quick on re0_vlan2 route-to (re0_vlan3 88.114.55.133) inet proto udp from 192.168.5.0/24 to any port = radius keep state label "USER_RULE"
                pass in quick on re0_vlan2 route-to (re0_vlan3 88.114.55.133) inet proto udp from 192.168.5.0/24 to any port = radacct keep state label "USER_RULE"
                pass in quick on re0_vlan10 inet from 192.168.100.0/24 to 192.168.5.0/24 flags S/SA keep state label "USER_RULE"
                pass in quick on re0_vlan10 inet from 192.168.100.0/24 to 192.168.100.100 flags S/SA keep state label "USER_RULE"
                pass in quick on re0_vlan10 inet proto icmp from 192.168.100.0/24 to any keep state label "USER_RULE"
                pass in quick on re0_vlan10 inet proto tcp from 192.168.100.111 to <vpns>flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
                pass in quick on re0_vlan10 route-to (re0_vlan5 88.114.48.109) inet proto tcp from 192.168.100.111 to any port = smtp flags S/SA keep state label "USER_RULE"
                pass in quick on re0_vlan10 inet proto tcp from 192.168.100.111 to <vpns>flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
                pass in quick on re0_vlan10 inet proto udp from 192.168.100.111 to <vpns>keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
                pass in quick on re0_vlan10 route-to (re0_vlan5 88.114.48.109) inet proto tcp from 192.168.100.111 to any port = domain flags S/SA keep state label "USER_RULE"
                pass in quick on re0_vlan10 route-to (re0_vlan5 88.114.48.109) inet proto udp from 192.168.100.111 to any port = domain keep state label "USER_RULE"
                pass in quick on re0_vlan10 inet from 192.168.100.0/24 to <vpns>flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
                pass in quick on re0_vlan10 route-to (pppoe3 99.178.192.1) inet from 192.168.100.0/24 to any flags S/SA keep state label "USER_RULE"
                anchor "packagelate" all
                anchor "tftp-proxy/                " all
                anchor "limitingesr" all
                anchor "miniupnpd" all
                No queue in use</vpns></vpns></vpns></vpns></bogons></bogons></virusprot></sshlockout></snort2c></snort2c>

                1 Reply Last reply Reply Quote 0
                • I
                  ivanmar
                  last edited by

                  I can't belive I am the only with this problem. I use VLANs, but configuration is prety ordinary. I can see there are many ongoing changes in recent snapshots on pppoe interfaces…

                  1 Reply Last reply Reply Quote 0
                  • G
                    gnhb
                    last edited by

                    Hi,

                    Your problem is solved by this commit.
                    https://rcs.pfsense.org/projects/pfsense/repos/mainline/commits/58cbe016de8cd2e88b08529f6272da849251647a

                    Your PPPoE interface is last in your interface list and a code change on Jun3 caused the last interface in the list to not be recognized as an interface that really exists.

                    Look at the time stamp in the commit and select an update or a new build from the snapshot server that is newer.

                    GB

                    1 Reply Last reply Reply Quote 0
                    • I
                      ivanmar
                      last edited by

                      This was a big problem for me.
                      God bless you :)

                      1 Reply Last reply Reply Quote 0
                      • G
                        gnhb
                        last edited by

                        The code change is small. You can do it yourself if you do want to get a new build.

                        1 Reply Last reply Reply Quote 0
                        • I
                          ivanmar
                          last edited by

                          That did a trick even without restart. Thanks

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.