Subnet/VLANs with managed and unmanaged switches



  • Hi,

    I'm getting started with pfSense and trying to figure out how I can get things working. I have the following setup:

    pfSense
        WAN - rl0
        LAN - re0 192.168.0.0/24

    plugged into Dell PC5424 managed switch:
         pfSense box
         various servers on 192.168.0.0/24
         6 unmanaged switches without VLAN support for subnets.

    If I want subnets and multiple gateways on re0, is this possible with VLANs? My unmanaged switches don't support VLANs. Would I use Virtual IPs? Which ones? I think the Proxy ARP doesn't work with packages like Squid so is there any option? If VLANs are possible, do I need to configure them for all the servers plugged into the 5424 switch? switch and server interfaces w VLANs? tagged/untagged? How do I CARP all the subnet gateways?

    I can create IP aliases thru the command-line but traffic doesn't route to the WAN interface. I imagine if I did that, I'd also have to manually configure the firewall thru the console?

    My current setup is a Linux box with virtual IPs for each subnet. It's quite an effort to maintain and I'm really hoping to go to pfSense.

    Thanks!

    g



  • If you're using multiple VLANs on a single NIC, you shouldnt use it at the same time as a normal interface.

    1: Go to Interfaces –> assign.
    2: Create on the second tab as many VLAN's as you want on your current LAN-interface.
    3: Create the VLAN's on your switch. Define one of the ports a trunk.
    4: Go to the first tab and set your LAN to a VLAN. (make sure you are on this VLAN on the switch or you will loose access to the webgui).
    5: Assign all your other VLAN's and configure them.

    Each VLAN will appear as new "Interface" (OPT1, OPT2, OPT3, etc.)

    If you have unmanaged switchs that shouldnt matter.
    Just configure your VLAN cappable switch that it sends packets to this switch untagged.



  • Thank you GruensFroeschli. I did loose access to the webgui. I'm not sure how to get it back. I see thru the commandline it has the option VLAN_HWTAGGING. Still trying to figure out how to remove it.

    The default VLAN1 is set to untagged on all ports. Do I need to even use VAN1 anywhere? I also get this message in my switch:

    UnTagged Port g20 can not be added to Unauthenticated Vlan 100.

    I'm looking thru where I can fix that but haven't see it. I'm really just starting out with VLANs. Thanks for all the help.

    g



  • Run "assign interfaces" from the shellmenu to assign the lan interface to a real interface instead of a vlan to get access to the webgui back.



  • Thanks hoba.

    I tried that last night and also reverted the switch back to be trunked but I can't connect for the moment. I might go into the office and check it it out. The setup is like this:

    WAN

    LAN - 192.168.0.x

    VLAN1 on LAN - should be unused?
    VLAN5 on LAN - 192.168.0.x - server network
    VLAN100 on LAN - 192.168.100.x - subnet

    the pfSense LAN interface would be VLAN5 tagged?

    It seems VLAN100 for instance goes to a VLAN unaware switch. That port on the main switch would be VLAN 100 untagged? If I wanted it to be a part of multiple VLANs, it seems possible?

    All the servers are plugged into the main switch. It sounds like would work as untagged or tagged VLAN5?

    I'll try reassigning the LAN interface and should be able to figure this out afterwards.

    The switch was complaining about unauthenticated users for the untagged VLAN100 port. Is EAP/RADIUS used in these cases?



  • @hoba:

    Run "assign interfaces" from the shellmenu to assign the lan interface to a real interface instead of a vlan to get access to the webgui back.

    hoba,

    If I want to bind multiple networks to a single interface, is this how it's done?  My current Watchguard allows for secondary networks on the same LAN.  You just provide a free IP address on that secondary network and it provides the routing between all the other networks.

    thanks



  • @ g: It sounds like you are not really sure how to setup your switch. I would reset it to factory defaults and start simple. Maybe you somehow have enabled 802.1x without setting up a radius server for this and now the ports are locked down.

    @ cirrusflyer: This is bad network design and you usually should not do that. However it will be available in the upcoming version of pfSense 1.3. If you have such a setup now you could "fake" it by using 2 nics (or vlans) that hook up to the same switch/layer2 network unless you want to fiddle around with console commands which will be gone after reboot or filter changes and so on…



  • @hoba:

    @ g: It sounds like you are not really sure how to setup your switch. I would reset it to factory defaults and start simple. Maybe you somehow have enabled 802.1x without setting up a radius server for this and now the ports are locked down.

    Hoba, yes I am not sure about somethings. It is still set to factory defaults. When I tried to change a port from default VLAN1 to  VLAN100 untagged, it complained about an unauthenticated/untagged port. So I thought maybe some authentication method was necessary.



  • Got it all working. Thanks a lot for the help.

    What I was hoping for was a seamless transition between the old and new router. Since the LAN interface only contains VLANs, accessing the rest of the network goes thru WAN. So it's definitely not possible to have the pfSense LAN be re0 AND have VLANs attached to re0? I can't get traffic thru to pfSense that way. Either way, does the switch port need to be trunked or can it be set to general with default VLAN1 untagged and the rest tagged? I sort of undertand why it wouldn't work but not exactly.

    If I wanted to achieve all of this, it sounds like I could use 3 NICs. WAN, LAN, and the all the VLANs on the 3rd. And then a 4th for CARP. That's the only way?



  • only the port connected to pfsense needs to be tagged



  • So it's definitely not possible to have the pfSense LAN be re0 AND have VLANs attached to re0?

    It is possible. But you shouldnt do it.

    But you can define your LAN as a VLAN.

    This way you would need only 3 Interfaces:
    -WAN
    -Interface_with_all_VLAN_inclusive_LAN
    -CARP_sync_interface

    To make it more clear:

    |–------WAN----------------|
      |            |                      |
      |        pfSense --- CARP_sync
      |            |                      |
      |------VLAN_interface-------|
                      |
                      |
                trunk (tagged)
                      |
              |-----|----------|
              |  VLAN_switch  |
              |--|----|-----|--|
                  /      |     
                /        |      (untagged)
      (untagged)  |         
            /      (untagged)   
        LAN            |            OPT2
                        OPT1

    You have NO untagged traffic on the VLAN_interface. You just dont assign the interface directly. Only VLAN's which are on this interface.



  • @GruensFroeschli:

    So it's definitely not possible to have the pfSense LAN be re0 AND have VLANs attached to re0?

    It is possible. But you shouldnt do it.

    I think I tried the right way to do it but it didn't work.

    LAN - re0
    VLANs attached to re0
    switch port trunked

    No traffic gets to the router. I think I also tried general (full 802.1q) with all VLANs on the switch.

    I understand not having a non-VLAN interface is more secure. Any other reason?

    @GruensFroeschli:

    But you can define your LAN as a VLAN.

    I think your original suggestion was to define LAN as a VLAN? That was done and everything works. The problem we're migrating and so many networks will be VLAN unaware until we're finished. Let's say:

    192.168.0.x - server network
    192.168.10.x
    192.168.20.x
    192.168.30.x

    Let's say I move 10 to VLAN10. If I want to route to any other network, since the LAN is a VLAN, it routes to WAN to access any other network. I'm not sure if untagged traffic on the 0 network reaches the interface. I'm not sure how to deal with that. Another NIC would work for sure. But when I try:

    pfSense:
    LAN - 192.168.0.254 - re0 real interface

    re0:
    VLAN10, 20, 30, …

    From the switch, I can't reach any of the networks.

    The main goal is migration without switching everything at once.



  • IMO you should migrate it all together since the firewall will be the backbone. So setup a lab environment where you can test your setup.



  • @Perry:

    There shouldn't be any difference in running the lan assigned to a real nic or a vlan nic.

    The maximum amount of traffic to push between subnets is dependent on this. Don't know his needs so maybe a shared link simply is not enough.



  • 192.168.0.x - server network
    192.168.10.x
    192.168.20.x
    192.168.30.x

    Let's say I move 10 to VLAN10. If I want to route to any other network, since the LAN is a VLAN, it routes to WAN to access any other network. I'm not sure if untagged traffic on the 0 network reaches the interface. I'm not sure how to deal with that. Another NIC would work for sure. But when I try:

    pfSense:
    LAN - 192.168.0.254 - re0 real interface

    re0:
    VLAN10, 20, 30, …

    From the switch, I can't reach any of the networks.

    It wont route to WAN.
    pfSense just routes between it's interfaces directly. And each VLAN is treated as an interface.

    What do you mean from the switch you cannot reach the other networks?
    Did you set the default gateway to pfSense?
    And created rules on the (VLAN)interface that allow traffic?

    I have this exact setup working.
    I'll post screenshots of the config of my switch when i get home.

    The maximum amount of traffic to push between subnets is dependent on this. Don't know his needs so maybe a shared link simply is not enough.

    There are a lot of VLAN capable switches that offer 2 or 4 Gbit port.
    I use such a setup where the trunk interface is Gbit and the normal ports on the switch are 100 Mbit.
    You can avoid this bottleneck.



  • Temporarily. the WAN interface was on 192.168.0.x. So it reaches that network. But it's not the best way.

    With the setup you gave above:

    re0 LAN - VLAN10
    re0 multiple VLANs attached

    I still need to be able to read the 0.x network. Those ports are connected as default VLAN1 on the switch. I also tried making VLAN1 re0 LAN; that didn't work. Maybe I was missing something. Maybe pfSense was properly setup but the switch wasn't. It seemed tagged VLAN traffic went thru but default VLAN1 untagged didn't.



  • Those ports are connected as default VLAN1 on the switch.

    They are not supposed to be the default VLAN1.

    VLAN1 (default) are all the ports that are "not in a VLAN".
    But the point of moving the LAN to a VLAN is: NOT USING VLAN1
    VLAN1 is a reserved VLAN!
    Refer to the 802.1Q specs page 76.
    http://standards.ieee.org/getieee802/download/802.1Q-2005.pdf

    Table 9-2—Reserved VID values

    VID value(hexadecimal) Meaning/Use
    0 The null VLAN ID. Indicates that the tag header contains only priority
    information; no VLAN identifier is present in the frame. This VID value shall not
    be configured as a PVID or a member of a VID Set, or configured in any Filtering
    Database entry, or used in any Management operation.

    1 The default PVID value used for classifying frames on ingress through a Bridge
    Port. The PVID value of a Port can be changed by management.

    FFF Reserved for implementation use. This VID value shall not be configured as a
    PVID or a member of a VID Set, or transmitted in a tag header. This VID value
    may be used to indicate a wildcard match for the VID in management operations
    or Filtering Database entries.

    You wrote that you've set LAN to VLAN10
    –> Just set the PVID for all the ports that should be LAN to 10.

    Then tag packets going to pfSense (on the trunk) and untag packets going to the clients.



  • @GruensFroeschli:

    Those ports are connected as default VLAN1 on the switch.

    They are not supposed to be the default VLAN1.

    VLAN1 (default) are all the ports that are "not in a VLAN".

    But the point of moving the LAN to a VLAN is: NOT USING VLAN1

    :) yes exactly. But since I'm not making the complete switch yet (moving one network at a time), I still need to access VLAN1, everything that is not on a VLAN. So what I was hoping to do is have one interface on pfSense that would have access to everything not on a VLAN and all the VLANs. :)



  • Aha now i get it ;D

    Well i suppose as long as it's only temporarily you could assign the interface directly.
    Of course a second interface would work too.

    It's not like that it wont work, it's just "bad" design.
    "bad" as in mixing tagged and untagged traffic on the same wire.

    But i dont know if you want to add a 4th NIC just to make the transition :)

    When i thinking about it: can you set the trunk on your switch that it eggresses tagged VLAN1 packets?
    Or do you mean with

    I still need to be able to read the 0.x network. Those ports are connected as default VLAN1 on the switch. I also tried making VLAN1 re0 LAN; that didn't work. Maybe I was missing something. Maybe pfSense was properly setup but the switch wasn't. It seemed tagged VLAN traffic went thru but default VLAN1 untagged didn't.

    exactly that?

    Because it should be possible from the pfSense side to accept tagged VLAN1 packets.



  • @GruensFroeschli:

    Aha now i get it ;D
    ..
    But i dont know if you want to add a 4th NIC just to make the transition :)

    I may just do that.

    @GruensFroeschli:

    When i thinking about it: can you set the trunk on your switch that it eggresses tagged VLAN1 packets?
    Or do you mean with

    I still need to be able to read the 0.x network. Those ports are connected as default VLAN1 on the switch. I also tried making VLAN1 re0 LAN; that didn't work. Maybe I was missing something. Maybe pfSense was properly setup but the switch wasn't. It seemed tagged VLAN traffic went thru but default VLAN1 untagged didn't.

    exactly that?

    Because it should be possible from the pfSense side to accept tagged VLAN1 packets.

    I have to try again to see if the Dell switch can tag VLAN1 traffic to the port. I tried setting the Dell to tag traffic to a particular port but it seemed unable to. I tried setting the switch port to something besides trunk (general I think with all ports going tagged) and I think that didn't work.

    I may try again with VLAN1 on the interface. I was curious about a earlier comment about not having interfaces directly assigned.

    Should the LAN interface be assigned anything or can just all the VLANs be attached to interface re0? and LAN have nothing?



  • I may try again with VLAN1 on the interface. I was curious about a earlier comment about not having interfaces directly assigned.

    Should the LAN interface be assigned anything or can just all the VLANs be attached to interface re0? and LAN have nothing?

    That goes into the same as

    It's not like that it wont work, it's just "bad" design.
    "bad" as in mixing tagged and untagged traffic on the same wire.

    You "should" not assign an interface on which VLANs are running.
    Like i said: It will work. It's just not good network design.

    Either Have LAN as VLAN too,
    or have another interface as LAN



  • @GruensFroeschli:

    Either Have LAN as VLAN too,
    or have another interface as LAN

    By LAN you mean the the LAN itself and not the pfSense LAN interface?

    I can go with an additional NIC until we fully switch over but I'm still curious.

    I haven't looked into CARP yet. It seems that I would have a CARP ip for each VLAN and WAN and then use that as the default gateway for clients?

    I'm almost there. :) Really can't wait to start using pfSense.



  • Lets refer to the physical interface as re0.
    I mean: asign the logical LAN-interface either as VLAN on re0, or add another NIC (re1) and assign the LAN-interface directly to re1.

    CARP is not what you are looking for.
    CARP is used for redundant hardware. (Failover on hardware-fail)
    Or to create Virtual IPs to/from which you NAT stuff.

    Each VLAN is a seperate logical interface on pfSense.
    Meaning each interface will have its own IP.



  • Lets refer to the physical interface as re0.
    I mean: asign the logical LAN-interface either as VLAN on re0, or add another NIC (re1) and assign the LAN-interface directly to re1.

    That's the plan. Should be able to get around to it later today or tomorrow.

    @GruensFroeschli:

    CARP is not what you are looking for.
    CARP is used for redundant hardware. (Failover on hardware-fail)
    Or to create Virtual IPs to/from which you NAT stuff.

    I would create a Virtual CARP IP on each VLAN interface and then use that as the default route for each VLAN? The idea would be to avoid routing to any real IPs yes?



  • I would create a Virtual CARP IP on each VLAN interface and then use that as the default route for each VLAN? The idea would be to avoid routing to any real IPs yes?

    I dont follow.
    What is the point of having a router if you dont want to route?



  • @GruensFroeschli:

    I would create a Virtual CARP IP on each VLAN interface and then use that as the default route for each VLAN? The idea would be to avoid routing to any real IPs yes?

    I dont follow.
    What is the point of having a router if you dont want to route?

    It's just that I haven't read the docs yet. For failover to another router, I would want the default gateway interface on each network to move between routers? So if 192.168.[VLAN].1 was the default route for each network, how would this failover to the 2nd router? By using CARP IPs attached to each VLAN interface?



  • @GruensFroeschli:

    You "should" not assign an interface on which VLANs are running.

    You mean for each VLAN, there should be no interface assignments? The following worked before:

    re0
    LAN
      VLAN100 - IP 192.168.100.254

    I could ping 100.254 from a different port on the switch. Now I have the following which doesn't work. Same switch setup:

    sk0 - assigned to 0.254 LAN

    se0
      interfaces assigned to all VLANs with IPs of 192.168.x.254

    se0 is trunked. sk0 is reachable of course but se0 (192.168.100.254) is not. My client is on another port with the gateway set to 100.254. Firewall rules are set to allow everything. The interface status does show IN and OUT packets. Maybe it's the switch? The switch setup is the same as when it did with with LAN assigned to VLAN100.



  • @GruensFroeschli:

    You "should" not assign an interface on which VLANs are running.

    Maybe you mean if multiple VLANs are assigned to a NIC and have IP addresses, that same NIC should not be assigned to WAN or LAN?

    The interface assignments are:

    WAN rl0
    LAN sk0

    All OPT interfaces are assigned to a VLAN on re0:

    OPT1 - VLAN 1 on re0
    OPT2 - VLAN 2 on re0
    OPT3 - VLAN 3 on re0

    WAN and LAN are separate NICs. I got it working. But. All access to WAN works. With LAN (thru sk0), I can only ping or telnet to any of the listening ports. Web/SSH, all traffic shows passing thru the firewall but doesn't come back. Firewall states show:

    192.168.100.81:58487 -> 192.168.0.x:22  CLOSING:CLOSED 
    tcp 192.168.100.81:58574 -> 192.168.0.x:22 SYN_SENT:CLOSED



  • @g:

    Maybe you mean if multiple VLANs are assigned to a NIC and have IP addresses, that same NIC should not be assigned to WAN or LAN?

    I mean about that.
    Only WAN and LAN can be VLAN too.

    Simple: Dont assign a real interfaces if you have VLANs running on them.

    On my WRAP this would look like this:

    availlable interfaces: sis0, sis1, sis2

    LAN:  VLAN 1001 on sis0
    WAN:  sis2
    OPT1: VLAN 1101 on sis0
    OPT2: VLAN 1201 on sis0
    OPT3: VLAN 1301 on sis0
    OPT4: VLAN 1401 on sis0
    OPT5: sis1

    As you can see: i dont mix normal assignments and VLAN assignments on the NICs.
    But still LAN can be a VLAN, even WAN could be a VLAN.

    WAN and LAN are separate NICs. I got it working. But. All access to WAN works. With LAN (thru sk0), I can only ping or telnet to any of the listening ports. Web/SSH, all traffic shows passing thru the firewall but doesn't come back.

    What exatly do you mean with "all traffic shows passing thru the firewall but doesn't come back."
    Where does this traffic go to? Does the destination know the route back to you?
    Did you create rules on all interfaces that allow traffic? (per default everything on a new interface is blocked)


Locked