Multiple interfaces connected to same vlan



  • I have a setup where I'd like to be able to connect multiple interfaces (Opt1, Opt2, Opt3) to the same VLAN using pfSense 1.2 - unless someone has a better idea.

    I have a managed switch and have configured VLANs successfully. In this setup, each opt interfaces is configured as a WAN connection using DHCP from the same cable modem - so each has a different public IP (and I use firewall rules to route/nat traffic). Ideally, in this situation, I'd be using CARP/VIPs on a WAN interface, but getting static IPs is not an option at the moment, and AFAIK, there is no way to use VIPs with DHCP - so I came up with using multiple opt interfaces.

    Right now, I have the setup working on a VMWare machine. The VM has 4 virtual nics, 3 of them bounded to the same VLAN in the VM Host - from pfSense's perspective, it has 4 nics.

    I have setup a proof of concept physical system using an Intel MT Gbe network card with VLANs; with 3 VLANs, and one untagged LAN defined in pfSense. However, the 3 VLANs are connected to the same physical LAN, they're just defined for pfSensen's sake (since pfSense doesn't seem to allow to have more than 1 interface assigned to each VLAN). I'm using the managed switch to untag the packets and then connecting a physical port from each "dummy" VLAN to a hub that connects to the cable modem. It works, but it is a bit messy: it requires 1 physical port for each VLAN, configuring VLANs that have no real reason of being, additional cables, and hub.

    Additional info. I checked "suppress ARP messages" under Advanced/Shared Physical Network. I haven't tried if it'd work w/o it (since it's a shared physical network, I presumed it's better to have it checked).

    Any suggestions? Manually editing the configuration XML file to define other interfaces using the same vlan? In this setup, will each interface generate its own MAC address even if they share the same vlan. Any other ideas?

    Thanks



  • Why dont you change that around?
    multiple different VLANs on a single NIC which go untagged to a single port.

    |–------WAN----------------|
      |            |                      |
      |        pfSense --- CARP_sync
      |            |                      |
      |------VLAN_interface-------|
                      |
                      |
                trunk (tagged)
                      |
                      |
              |-----|----------|
              |  VLAN_switch  |
              |-------|--------|
                        |
                        |
                        |                       
              |--------|--------|             
              |          |          |                 
              |          |          |                   
              |          |    (untagged)             
        (untagged) |          OPT3                  untagged trunk
            OPT1    |            |                      /
              |    (untagged)  |                    /
              |        OPT2      |                  /
              |          |          |                /
              |--------|---------|            /
                        |                      /
                  Cable-modem

    I'm not sure if this is doable with a VM-Ware virtual-switch.
    But on a real VLAN-capable switch i'd just assign multiple different VLAN's tagged to a port (trunk). And the same multiple VLANs untagged to another port which goes to the cable modem.



  • That's what I'm trying to do with the physical installation. There is a single trunk going from pfSense to the VLAN switch.

    My problem is the other side, from the VLAN switch to the cable modem: it's physically "messy" and wasteful - 3+ cables using 3+ ports on the VLAN switch (the 3 untagged legs at the bottom of your diagram) plus introducing another switch (if I were to plug it back into the VLAN switch, it'd be another 3 ports). My ISP allows me to get up to 5 dynamic IPs -I'm using 3 in my tests-, so the number of cables/ports can be higher. I just thought there could be a better way to setup a physical system that could scale better (not that this is a common scenario). Maybe I'm giving too much value to the VLAN ports?: I've become fond of VLANs and when you start trunking and teaming (in other hosts in the LAN), 24 ports get used up pretty quickly…

    The virtual installation is working pretty good. My problems with it are that sporadically there is a spike in disk activity on the host, and it makes most VMs unresponsive on the network (I suspect that the culprit is the 3Ware controller on the host) and it is kind of frustrating that when it happens, I start losing ~half of all internet communications - including voip phones, garbling ongoing phone calls. The other problem is that VMWare Server 1.x only supports 4 virtual NICs on a Windows host (in my case, I'd like to do 6+, having a couple DMZ type interfaces). I might try and install VMWare Server 2.0 Beta on another host (with teaming and trunking) and try the VM there - VMS 2.0 supports 10 interfaces on a Windows host. On the plus side, it's easier, it works ok most of the time, and it brings some redundancy to the pfSense setup -it's on a RAID, and with teamed connections to the switch- with little cost...

    I tried editing the xml file -in a VM setup-, adding an opt1 interface using the same value for <if>as the wan interface, and I set <spoofmac>to a generated MAC address. The VM would lose connectivity altogether - couldn't reach it on the LAN, and it seems it merges the WAN & OPT data into a single interface (the console would show up 2 interfaces instead of the expected 3) and it wasn't able to get an IP on the WAN...</spoofmac></if>



  • My problem is the other side, from the VLAN switch to the cable modem: it's physically "messy" and wasteful - 3+ cables using 3+ ports on the VLAN switch (the 3 untagged legs at the bottom of your diagram) plus introducing another switch (if I were to plug it back into the VLAN switch, it'd be another 3 ports).

    What i mean is: DONT use 3(or more) different ports for the untagged traffic.
    Just use a single one.
    Think of the other side as a kind of "untagged trunk".
    The VLAN switch would be doing not much else than removing the tags of the packets comming from the pfSense, and adding tags to the packets comming from the cable modem.

    I'm not sure if that really works, but i think it should.
    Worth a try at least :)

    (i changed the diagram above)



  • It's my understanding that on each port theres is a "default" vlan for untagged traffic: untagged traffic only participates in a single VLAN.

    While the port would output packets coming from all the VLANs, the cable modem's responses would go back to the port's default VLAN (VLANx). pfSense would see the responses for requests in VLANy & VLANz on VLANx (with the MAC/IP addresses from the other VLANs) and probably ignore them. Unless the "suppress ARP messages" setting affects the way pfSense sees incoming packets…

    I think that for this to work, the switch would have to repeat all traffic it gets in the untagged trunk on each participating VLAN. I'm using a Netgear GSM 7224 managed switch and I haven't found any settings that would suggest it'd repeat the traffic on all VLANs. Yet the config interface is somewhat confusing, maybe I'm missing something. I'll give it a try...

    Thanks



  • I thought about the problem a bit more and configured my Netgear FS726T that was lying around.
    I was thinking i once solved a problem where i had something similar, but we had multiple ports and not a single one.

    Unfortunately i think i know why it doesnt work :(
    The main problem is that you cannot set multiple VID's for a port.
    Meaning the part in my diagram of having an "untagged trunk" is not possible.

    But i dont understand something in your VMWare-approach.
    Isnt it possible to have some kind of "virtual switch"?
    (It's a while since i've played with VMware)
    (just found this link that describes what i mean http://pubs.vmware.com/vi3/serverconfig/wwhelp/wwhimpl/common/html/wwhelp.htm?context=serverconfig&file=sc_networking.5.3.html )

    Now if you could bridge one "port" of this virtual switch to a real NIC, and connect as many virtual NIC's to this virtual switch as you want to get DHCP leases.
    (Does the 4 NIC limit still exist?)



  • Indeed, only 1 VLAN ID can be associated to untagged traffic on each port…

    Yes, with VMWare you have a number of virtual switches and you can add a number of virtual nics to your VM guests. The numbers and features available depend on the edition/version/host type you're using.

    Now if you could bridge one "port" of this virtual switch to a real NIC, and connect as many virtual NIC's to this virtual switch as you want to get DHCP leases.

    Yes, that's what I'm doing. In my setup, on the pfSense VM guest, I bridge 1 virtual nic to my main LAN VLAN, and the other 3 to the VLAN connected to the cable modem (all 3 to the same VLAN) - see attached screenshots. Inside pfSense, I see 4 adapters, everything is working. I was just trying to migrate it to a physical setup due to performance issues on this specific VMWare Host…

    (Does the 4 NIC limit still exist?)

    Yes, but it is going up to 10 on the next version (I think it's already up to 10 in the latest WS). With VMWare Server 1.x on a Windows host, you get 10 "Unmanaged Virtual Switches/Networks" - out of which 3 are used by VMWare, effectively leaving you with 7 in most setups. VMWS 1.x guests are limited to 4 virtual nics, but VMWS 2.0 will allow 10 nics (but I think it's still limited in the number of networks). VMWS under Linux already supports more networks (100?) and I believe more nics.







Locked