Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    VLAN tag on WAN not working

    Scheduled Pinned Locked Moved
    General pfSense Questions
    3
    23
    6.8k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mokozuki
      last edited by mokozuki

      My ISP (Fieber, Netherlands) supplied me with a Draytek router, this has always worked well, but i want better VPN support so am now replacing it with a pfsense box (QOTOM). i have take all the settings from the Draytek and set up the WAN the same way (except for the VLAN bit) but can't get it to work.

      ISP says WAN needs to have VLAN tag ID 128. In the Draytek modem this was just a simple text field to fill in "WAN VLAN SETTING - WAN VLAN ID". I have searched these forums and googled but have not found anything that works for me. MAC address is spoofed with the WAN MAC of the Draytek router. I have tried creating a VLAN with tag ID 128 on igb0 and connected that with WAN, doing nothing else with the igb0 interface. This didn't work either.

      Has anyone with similar conditions from ISP have a working setup with pfsense?

      thanks for any help or nudges in the right direction.

      1 Reply Last reply Reply Quote 0
      • GrimsonG
        Grimson Banned
        last edited by

        Start here: https://www.netgate.com/docs/pfsense/book/vlan/terminology.html

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          If they only need tagged traffic then creating a VLAN on igb0 and assigning that as WAN should work.

          What are you actually seeing? Should it pull an IP via DHCP? What do the logs show?

          Steve

          1 Reply Last reply Reply Quote 0
          • M
            mokozuki
            last edited by

            stephenw, thanks for the reply. i had already tried what you suggested (and just tried again). from what i glanced from other posts that seemed to be what was needed, but i wasn't sure if that was it, since it didn't work for me.

            i did a packet capture on wan to see what the dhcp request packets look like, but don't actually have any idea what they should look like, so i can't see if anything is wrong there. in case someone else does, below is the captured data:

            Frame 13: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
            Encapsulation type: Ethernet (1)
            Arrival Time: Jan 20, 2009 03:37:29.013311000 CET
            [Time shift for this packet: 0.000000000 seconds]
            Epoch Time: 1232419049.013311000 seconds
            [Time delta from previous captured frame: 6.179544000 seconds]
            [Time delta from previous displayed frame: 6.179544000 seconds]
            [Time since reference or first frame: 47.215069000 seconds]
            Frame Number: 13
            Frame Length: 342 bytes (2736 bits)
            Capture Length: 342 bytes (2736 bits)
            [Frame is marked: False]
            [Frame is ignored: False]
            [Protocols in frame: eth:ethertype:ip:udp:bootp]
            [Coloring Rule Name: UDP]
            [Coloring Rule String: udp]
            Ethernet II, Src: Draytek_03:7b:xx (00:1d:aa:03:7b:xx), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
            Destination: Broadcast (ff:ff:ff:ff:ff:ff)
            Source: Draytek_03:7b:xx (00:1d:aa:03:7b:xx)
            Type: IPv4 (0x0800)
            Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
            0100 .... = Version: 4
            .... 0101 = Header Length: 20 bytes (5)
            Differentiated Services Field: 0x10 (DSCP: Unknown, ECN: Not-ECT)
            0001 00.. = Differentiated Services Codepoint: Unknown (4)
            .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
            Total Length: 328
            Identification: 0x0000 (0)
            Flags: 0x00
            0... .... = Reserved bit: Not set
            .0.. .... = Don't fragment: Not set
            ..0. .... = More fragments: Not set
            Fragment offset: 0
            Time to live: 128
            Protocol: UDP (17)
            Header checksum: 0x3996 [validation disabled]
            [Header checksum status: Unverified]
            Source: 0.0.0.0
            Destination: 255.255.255.255
            [Source GeoIP: Unknown]
            [Destination GeoIP: Unknown]
            User Datagram Protocol, Src Port: 68, Dst Port: 67
            Source Port: 68
            Destination Port: 67
            Length: 308
            Checksum: 0xb295 [unverified]
            [Checksum Status: Unverified]
            [Stream index: 0]
            Bootstrap Protocol (Discover)
            Message type: Boot Request (1)
            Hardware type: Ethernet (0x01)
            Hardware address length: 6
            Hops: 0
            Transaction ID: 0x9c4a8303
            Seconds elapsed: 28
            Bootp flags: 0x0000 (Unicast)
            0... .... .... .... = Broadcast flag: Unicast
            .000 0000 0000 0000 = Reserved flags: 0x0000
            Client IP address: 0.0.0.0
            Your (client) IP address: 0.0.0.0
            Next server IP address: 0.0.0.0
            Relay agent IP address: 0.0.0.0
            Client MAC address: Draytek_03:7b:xx (00:1d:aa:03:7b:xx)
            Client hardware address padding: 00000000000000000000
            Server host name not given
            Boot file name not given
            Magic cookie: DHCP
            Option: (53) DHCP Message Type (Discover)
            Length: 1
            DHCP: Discover (1)
            Option: (61) Client identifier
            Length: 7
            Hardware type: Ethernet (0x01)
            Client MAC address: Draytek_03:7b:xx (00:1d:aa:03:7b:xx)
            Option: (12) Host Name
            Length: 9
            Host Name: pfsense
            Option: (55) Parameter Request List
            Length: 9
            Parameter Request List Item: (1) Subnet Mask
            Parameter Request List Item: (28) Broadcast Address
            Parameter Request List Item: (2) Time Offset
            Parameter Request List Item: (121) Classless Static Route
            Parameter Request List Item: (3) Router
            Parameter Request List Item: (15) Domain Name
            Parameter Request List Item: (6) Domain Name Server
            Parameter Request List Item: (12) Host Name
            Parameter Request List Item: (119) Domain Search
            Option: (255) End
            Option End: 255
            Padding: 000000000000000000000000000000000000000000000000...

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              I don't see any VLAN information in that packet. Did you capture on the VLAN interface itself?

              If so try assigning igb0 also and pcap on that in promiscuous mode. You should see the VLAN tagging in the packet you're sending and whatever the ISP is sending back.

              Steve

              1 Reply Last reply Reply Quote 0
              • M
                mokozuki
                last edited by

                thanks for the info. i will try that and hopefully have more info to get it working.

                thanks

                1 Reply Last reply Reply Quote 0
                • M
                  mokozuki
                  last edited by

                  i did the capture in pfsense itself (Diagnostics -> Packet Capture). selected WAN (doesn't allow to select port of virtual port), and WAN is conntected on igb0 on VLAN 128. i redid the capture and it is the same. not sure if pfsense captures before tagging or maybe i set it up wrong. Will try it again tonight when no one should be using anything behind the router. thanks for the help.

                  0_1537779674115_Screenshot_20180924_082725.png

                  0_1537779694988_Screenshot_20180924_082742.png

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Right. You need to assign igb0 as another interface so that it is available to pcap on. That will show the VLAN tagged traffic on it.

                    Steve

                    1 Reply Last reply Reply Quote 0
                    • M
                      mokozuki
                      last edited by

                      thanks. i added igb0 to OPT3, enabled OPT3 and then did the capture on there and i can see the vlan tag. it looks ok. i will contact the isp to release the MAC and try to connect without spoofing the mac and see if that works.

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Sounds like a good plan.
                        Did you see traffic coming back from them at all? On the correct VLAN?

                        Steve

                        1 Reply Last reply Reply Quote 0
                        • M
                          mokozuki
                          last edited by

                          no traffic coming back at all. i am going to try and capture some packets form the draytek and see what is different

                          1 Reply Last reply Reply Quote 0
                          • M
                            mokozuki
                            last edited by

                            managed to capture dhcp discover packets from draytek, there are some differences, will look into trying to make pfsense do the same

                            draytek modem:
                            Option: (55) Parameter Request List
                            Length: 11
                            Parameter Request List Item: (1) Subnet Mask
                            Parameter Request List Item: (3) Router
                            Parameter Request List Item: (6) Domain Name Server
                            Parameter Request List Item: (12) Host Name
                            Parameter Request List Item: (15) Domain Name
                            Parameter Request List Item: (28) Broadcast Address
                            Parameter Request List Item: (42) Network Time Protocol Servers
                            Parameter Request List Item: (121) Classless Static Route
                            Parameter Request List Item: (33) Static Route
                            Parameter Request List Item: (212) 6RD
                            Parameter Request List Item: (150) TFTP Server Address

                            pfsense:
                            Option: (55) Parameter Request List
                            Length: 9
                            Parameter Request List Item: (1) Subnet Mask
                            Parameter Request List Item: (28) Broadcast Address
                            Parameter Request List Item: (2) Time Offset
                            Parameter Request List Item: (121) Classless Static Route
                            Parameter Request List Item: (3) Router
                            Parameter Request List Item: (15) Domain Name
                            Parameter Request List Item: (6) Domain Name Server
                            Parameter Request List Item: (12) Host Name
                            Parameter Request List Item: (119) Domain Search

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              The thing I would be looking for if that doesn't work is packets tagged with a VLAN PRIO number or some other QoS type marking. We seen other ISPs require that.

                              Steve

                              1 Reply Last reply Reply Quote 0
                              • M
                                mokozuki
                                last edited by

                                still stumped. pfsense was missing option-60 vendor-class-identification (googling showed some ISPs require this), so i added this and set it to exactly the same as on the draytek, spoofed the mac, made sure hostname is also spoofed. still not working. vlan id and prio are the same. it seems the packets are not reaching the isp at all or i would at least see some packets coming back.
                                the only difference now is the request list, but so far i can't get it to work. when i put something in pfsense gui request options field, it doesn't send option 55 at all anymore. i still have to try further, maybe i am doing it wrong.
                                but the only difference between pfsense parameter request list en draytek's is that the draytek asks for a few more things: static route, 6rd and tftp server address. the rest is the same as pfsense.

                                i have tried entering "Subnet Mask, Router, Domain Name Server, Host Name, Domain Name, Broadcast Address, Network Time Protocol Servers, Classless Static Route, Static Route, 6RD, TFTP Server Address", "subnet-mask, router, domain-name-server, host-name, domain-name, broadcast-address, network-time-protocol-servers, classless-static-route, static-route, 6rd, tftp-server-address" and just "router" in the "Request Options" field in WAN interface page of pfsense, but each time the packet capture showed de dhcp discover without any Option 55.

                                next test I'll scrounge some hardware and install a squeeky clean pfsense on it and see it it work on that. ISP tech support doesn't know of any special settings that they require, just plain DHCP they told me. But that's tech support, so it could mean anything.

                                thanks for the help and suggestions so far

                                1 Reply Last reply Reply Quote 0
                                • M
                                  mokozuki
                                  last edited by

                                  ok, so more info now after a few more nights/mornings of testing.

                                  i am running pfsense 2.4.4 on this install on a Qotom Q515G6 3865U (6 ports, intel gigabit), which doesn't seem to want to get the ip on wan through dhcp and vlan.
                                  so i installed pfsense 2.4.3 on a PC-Engine APC2C4 (3 ports, intel) and got the same result.
                                  so i thought i'd try opnsense and installed that on a spare Gigabyte BRIX J1900 which only has one ethernet port (realtek) and i hooked up an usb ethernet dongle (also realtek) for LAN. this actually worked, so I assumed there must be something wrong in the way pfsense translates configuration into dhcp packets. Then i went about and installed opsense on the QOTOM and guess what.... doesn't work. Then i installed pfsense 2.3.5 (2.4.4 would not install, it hung at "booting..." trying to install via USB memstick) on the BRIX and wan instantly gets IP from dhcp on WAN.

                                  So it seems there must be some hardware thing with boards that have more than 1 nic that is preventing it from receiving the dhcp offer packets?

                                  hoping someone has run into this kind of problem before and has a solution....

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Hmm, well that's an interesting set of results. You do have a variety of NIC types in there though. It looks like you have not had any success in any OS using an Intel NIC? Are those all igb?

                                    If it appears to hang at "Booting..." you are probably hitting the console error referred to here:
                                    https://www.netgate.com/docs/pfsense/install/upgrade-guide.html?highlight=upgrade#upgrading-from-versions-older-than-pfsense-2-4-4

                                    Booting the installer UEFI is probably easiest on a BRIX to get past that.

                                    Although we tried pcaping in promiscuous mode previously it would be worth trying to set the NIC in promiscuous mode permanently to see it that prevents it dropping packets (if that's what's happening). At the command line:
                                    ifconfig igb0 promisc

                                    If you run ifconfig -v igb0 it would be interesting to see what hardware offloading it had enabled. Similarly with ifconfig -v igb0.128.

                                    Steve

                                    1 Reply Last reply Reply Quote 0
                                    • M
                                      mokozuki
                                      last edited by mokozuki

                                      the PC-Engines APU2C4 has Intel i210AT nics and the Qotom Q515G6 3865U uses Intel I211AT nics.

                                      I tried setting the interfaces to promiscuous mode but that didn't yield any different results. I also tried connecting the usb ethernet dongle to the Qotom and using that for WAN, but that nic didn't support VLANs, somehting pfSense detects and OPNsense doesn't, or OPNsense uses a different driver for the dongle that does support VLANs. Anyways, dongle connected to Qotom with OPNsense didn't get an IP, I was able to set a VLAN on the dongle. On the BRIX i tried swapping WAN and LAN, realtek as LAN and the dongle as WAN, but on the BRIX running pfSense 2.3.5 the dongle ue0 was not selectable.
                                      Now that it seemed more like a driver problem, I did a search and came up with this bug reference https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209581
                                      A bug from May 2016, could what they are describing be the cause of the problem I am having?

                                      flags=UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC
                                      options=VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,TXCSUM_IPV6

                                      1 Reply Last reply Reply Quote 0
                                      • M
                                        mokozuki
                                        last edited by

                                        i tested with disabling -vlanhwfiler, -vlanhwtso and -vlanhwtag on igb0 but none of these made it work.

                                        1 Reply Last reply Reply Quote 0
                                        • stephenw10S
                                          stephenw10 Netgate Administrator
                                          last edited by

                                          Ok then I would try to install 2.4.4 on the BRIX by booting UEFI or upgrading from 2.4.3/2.3.5 and confirm that it works as expected with those interfaces in that version.

                                          That bug appears unrelated as I don't believe we are using that in a bare metal pfSense install. If you were hitting that the packet capture would show the traffic incorrectly tagged. I assume you setup a mirror port to pcap from the Draytek? Did you also use that see actual packets on the wire to/from pfSense?

                                          Steve

                                          1 Reply Last reply Reply Quote 0
                                          • M
                                            mokozuki
                                            last edited by

                                            ok, more testing.

                                            i upgraded the BRIX to 2.4.4 and it stopped getting an IP on WAN. so that felt like good news, so I went about to install 2.3.5 on the Qotom. After getting that done, it still didn't get an IP on WAN, so I thought maybe 2.4.4 didn't work on the BRIX because i had messed up the config before when switching re0 and ue0 for WAN and LAN. So, i created an installer stick with 2.4.4 and installed that on the BRIX which went very smoothly once I disabled CSM support in the BIOS like you suggested. Installed and configured, everything looked ok. Then I took it down to where the ISP line comes in and it didn't seem to boot ok (no screen over there) so wen up again 4 stories and tested it and the damn things boots fine. So i take it down again and same problem !#@##$%%! So I decided to stop for today. Tomorrow I will prepare a small monitor to bring so I don't have to keep climbing stairs. Will also prepare a ddwrt router and see if that works. I did install OpenWRT on the Qotom but did not get it to get an ip on WAN, but not sure if I configured it wrong maybe, I had never used OpenWRT/LuCI before.

                                            I captured the packets from the Draytek by configuring one of the ports on the Draytek as a mirror/monitor port, hooked up my laptop with wireshark.
                                            All captures for pfsense and opnsense were done with the packet capture option under diagnostics.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.