KVM + pfSense + X-WRT-Vortex + VLANs



  • Hi, I've have been using pfSense virtualized on an Ubuntu server with KVM from a couple of years with excellent results and about a month ago I decided to segment my network in a couple of VLANs and that's where I got stucked with an a problem.

    My setup is as follows:

    Cable –-WAN1--
                  |-- Ubuntu KVM(pfSense) -- LAN -- R7000(Xwrt) --
    aDSL ----WAN2--                                              |--ETH--VLAN3--- WDR3600 --- WIFI (guest)
                                                                  |--WIFI
                                                                      |
                                                                      |---VLAN4--- WIFI (kids2 and kids5)
                                                                      |---VLAN5--- WIFI (iot2 and iot5)
                                                                      |------------WIFI---- main wifi2 and main wifi5

    LAN: 192.168.2.0
    vlan3: 192.168.3.0
    vlan4: 192.168.4.0
    vlan5: 192.168.5.0

    So, I have

    • pfSense virtualized on an Ubuntu server with KVM with 2 WANs and 1 LAN
    • LAN interface is connected over ethernet to a Netgear R7000 with Xwrt and R7000 is also connected over ethernet to a WDR3600
    • I have defined 3 VLANs on pfSense, (vlan3, vlan4 and vlan5)
    • WDR3600 is on vlan3 and happily serves my guest wifi, no problem here
    • My trusted wifi devices are connected to my main wifi network on the R7000 (two SSIDS 2.4GHz and 5GHz) and it's over my LAN, so no vlan here
    • vlan4 and vlan5 are on the R7000 with the kids and iot wifi (also both 2.4GHz and 5GHz) and this is where I have problems…

    The devices connnected to either SSIDs from kids and iot networks (vlans 4 and 5) "randomly" gets internet connection and then looses it, wait some minutes and internet comes back. What I mean is that everything seems to work one moment and the other moment both just looses internet connection, but the other SSIDS don't loose internet connection, so devices connected to my main wifi and guest wifi works ok.

    I'm not a networking specialist so I may have probably made something wrong on this setup. For instance I'm no quite sure if it's safe to merge LAN and vlans on the same NIC, perhaps my main LAN should also be a vlan? I mean, if all the networks on this NIC should be vlans or the lan/vlans mix is ok.

    I also don't know if I'm missing some config on the Ubuntu server to support my setup...

    But as this setup "almost work" perhaps I'm only missing some fine tunning.... but I'm not sure if the problem is on the host (Ubuntu), the pfSense VM or the Xwrt config. Although I'm quite sure pfSense config is correct.

    I can share my config screens/files if needed, but didn't want to "overcharge" rhis post from the beginning.



  • Don't know if this helps or if in fact is another problem or the root of the problem I 've mentioned, but when I connect a device to the vlans 4 and 5 I see the assigned IP's on the firewall logs but under the LAN interface, so it appears that the LAN interface can see vlan4 and vlan5 traffic, something that definitely is not right.

    But when some device connects to vlan3 nothing shows up on LAN interface, this is right, and internet access works without problem.

    Also if I use the packet capture, and always capturing on the LAN interface, I can see packets on the LAN interface when some device is connected to vlan4 or vlan5, but I do not see nothing on the LAN interface when the device is connected to vlan3.

    Any ideas?



  • Hi pablot,

    it's really hard to tell, I can only provide best guesses.

    From what you described you can have one or multible of these issues:

    • MultiWAN routing
    • general routing issues
    • VLAN isolation

    From your description, check the last (VLAN isolation) fist:

    • Does your hardware support vlan trunks (that is, sending multible tagged vlans over one physical link)?
    • Did you set up these vlans in KVM / qemu using linux bridging? AFAIK you need to 'untag' and bridge each VLAN as a single network and expose them to your VM as single network. Otherwise the tags might just get discarded. Linux bridging will then do 'the right thing'. Ignore this if you are using openvswitch or similar, more fancy setup on your hypervisor
    • Test the noop vlans using static IPv4. Does this work?
    • Use tcdump and capture for DHCP traffic (67-68/udp). There should only be one server replying with one address. If not, VLANs are not isolated - a very bad thing

    For the rest (routing), sudo traceroute -In google.com is your friend.
    For multi wans, check whatever the you are using round robin interface groups. Some web sites do not like this, at least in my experience.

    Hope this helps,



  • Thanks for your reply.

    From your message the following bumped into my head…

    @helge000:

    • Did you set up these vlans in KVM / qemu using linux bridging? AFAIK you need to 'untag' and bridge each VLAN as a single network and expose them to your VM as single network. Otherwise the tags might just get discarded. Linux bridging will then do 'the right thing'. Ignore this if you are using openvswitch or similar, more fancy setup on your hypervisor

    I do not have openvswitch or nothing similar, I havedone nothing like that, I just have the thee bridges defined on my host as the following:

    The loopback network interface

    auto lo
    iface lo inet loopback

    The primary network interface

    #auto eth0
    #iface eth0 inet manual

    auto br0
    iface br0 inet static
            address 192.168.2.10
            netmask 255.255.255.0
            network 192.168.2.0
            broadcast 192.168.2.255
            gateway 192.168.2.13
            bridge_ports eth0
            bridge_stp off
            bridge_fd 0
            bridge_maxwait 0
            # dns-* options are implemented by the resolvconf package, if installed
            dns-nameservers 192.168.2.13 8.8.8.8
            dns-search localdomain

    auto br1
    iface br1 inet manual
            bridge_ports eth1
            bridge_stp off
            bridge_fd 0
            bridge_maxwait 0

    auto br2
    iface br2 inet manual
            bridge_ports eth2
            bridge_stp off
            bridge_fd 0
            bridge_maxwait 0

    How do I have to configure them so they support vlans?