Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Performance issues while using many vlans

    Scheduled Pinned Locked Moved General pfSense Questions
    14 Posts 9 Posters 5.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      bfr
      last edited by

      Hi,

      we have installed pfSense 2.2.4 on a server with a E3-1220 @3GHz and 16GB memory. After creating 500 VLANs with interfaces for a special setup, pfSense becomes almost unusable. When logging into the webinterface, php-fpm uses one core at 100% for 5-10 minutes.  Every click e.g. on Firewall -> Rules results in a similar behaviour.

      We tried also 2.3-alpha hoping the new webinterface will solve this problem.

      Is it possible to fix this issue?

      Regards
      bfr

      1 Reply Last reply Reply Quote 0
      • johnpozJ
        johnpoz LAYER 8 Global Moderator
        last edited by

        500 vlans???  Over how many physical interfaces?? 1??  Come on

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.8, 24.11

        1 Reply Last reply Reply Quote 0
        • F
          firewalluser
          last edited by

          The use of vlans will increase the cpu demand by a bit, this then has a knock on effect elsewhere in any system.

          My cpu usage is constantly at 50% + since adding 10 vlans to a single nic device running as my firewall.

          Better nics with their own onboard processing help offload the load placed on cpu(s) when using vlans.

          Edit. I should add as a possible workaround to the waiting times, if you have a number of rules which are applied across a number of interface groups https://doc.pfsense.org/index.php/Interface_Groups
          and because the scripts will have to work in some cases checking and applying changes across all vlans which explains the waiting time, a different approach would be to have a number of pfsense instances/devices in a tree like setup, which will break down the waiting time when making rule changes. So the top tier pfsense instance/device could be used just for routing but lower level pfsense instances/devices could actually handle the rules for x number of vlans.
          Ideally separate devices would be best for physical isolation, but easily achievable if running virtual pfsense instances but the risk then shifts to the virtualisation host device itself as bugs are potential backdoors.

          Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

          Asch Conformity, mainly the blind leading the blind.

          1 Reply Last reply Reply Quote 0
          • awebsterA
            awebster
            last edited by

            I think there are two separate issues here.

            First, while, to me, it seems far fetched to create 500 VLANs, the web configurator has to manage all this data for 500+ interfaces.  I'd imagine that unless it is running on a pretty hefty box, this is going to create the problem described, whenever any access is occurring to pages where interfaces are referenced.    php-fpm does not appear to be multi-threaded, so it will impact once CPU core, but should not impact the ability of the box to pass traffic.

            Second, FreeBSD per-se does not have any limit on the number of VLANs, but hardware offload will certainly help, but only in situations where there is traffic. 
            500 VLANs with light traffic should not put any load on the system if there is no web configurator session.

            Of course the project owners could choose to put some arbitrary hard limits to the number of VLANs, interfaces, objects, whatever resource, so that it works well in all situations, but then it would probably be a closed source proprietary project.  One of the nice things about open source projects is that many things are left as open-ended, meaning if you have the resources to run it, then go for it, just don't expect it to work in every extreme situation imaginable.

            –A.

            1 Reply Last reply Reply Quote 0
            • ?
              Guest
              last edited by

              500 vlans???  Over how many physical interfaces?? 1??  Come on

              ??? It takes me real 2 minutes to stop laughing about!  ;D

              pfSense comes with many features, options and a rich feature set, but activating them all together
              means also that you should be know what exactly and how hefty it is narrowing down the entire
              system (hardware).

              1st:
              (More Routing points in the network)
              Why not using Layer3 switches that are stacked together and if needed not the cheap SMB (KMU)
              switches for less money, but more the $2000 - $3000 (cut ´n through) models that are capable to
              handle this with ease and the firewall is then powerful enough to do the WAN - LAN and WAN - DMZ
              routing only? And this might be not related to the number of employees of this company but more
              tended then on the needs this company have.

              After creating 500 VLANs with interfaces for a special setup,

              I hope this are 10 GBit/s interfaces and then more then only one of them.

              Perhaps you might have a look for other network hardware that is also taking many load from the
              pfSense machine and together with stronger based hardware the pfSense will do the job more with ease!?

              2nd:
              (The right hardware playing also right together)
              Taking other hardware together which will have many more benefits as you might be seeing on the
              first look is often the goal that can be reached by assembling it wise together.

              • Chelsio-T520-CR are often able to offload totally the NAT and VLAN process!
                That means two of the greater adapters would be meaning one is offloading the entire NAT load
                and the second one is offloading the entire VLAN part.
              • Lanner FW-889x Series is powerfull enough to handle many more connections as an Intel Xeon E3 system
                The FW-8894, FW-8895 or FW-8896 appliances are very strong and can be sorted right with different
                NIC modules, that will be also a fine choice later to bring Intels QuickAssist in the game!
              • We are using the following switches without any failure and problems!
                – Netgear XSM7224S (M7300) with Layer3 license as Core Switch or with Layer2+ for servers, NAS/SANs, ect..
                -- Netgear XCM Series (M6100) if the installation is small with and mixed set up as Access Switches
                -- Netgear GSM7352S (M5300) Layer3 stacked also as Access Switches

              This is also not Cisco, Avaya, Allied Telesis, Brocade or Juniper for sure and this are also store and
              forward switches, but they are fast enough for all our network needs and not the cheaper $200 SMB ones.

              3rd:
              (Your network topology)
              Bringing it clearly to the point, you are able to stretch your network by inserting more devices
              and setting up more network layers.

              As now or today you have your firewall and then connect directly to this your "Access Switches"
              (Access Layer) but you can also do the following:
              Firewall - Core Layer - Distributed Layer - Access Layer

              This might be more expensive but the whole network load would be spreading over more Switch chips
              by using more switches as well and so you might speed up the entire network to a point where you will
              bring it back to act smooth and liquid likes before. But this could or better should not be only pointed to
              the firewall, as in days before.

              1 Reply Last reply Reply Quote 0
              • B
                bfr
                last edited by

                Hi,

                thanks for all the answers, especially the one which pointed out the php-fpm issue. The "high" number of vlans is related to requirements of a customer setup, so I did not want to start a discussion about the network topology or if an appliance would fit better.

                The main issue is that the webinterface tries to  get all the information of interfaces etc. on every access which only scales when using a few interfaces.

                Regards
                bfr

                1 Reply Last reply Reply Quote 0
                • DerelictD
                  Derelict LAYER 8 Netgate
                  last edited by

                  "A few" being dozens. I have no problem with more than a few VLANs.  < 500.

                  There has to be a better way (Private VLANs in your switches maybe).

                  Chattanooga, Tennessee, USA
                  A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                  DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                  Do Not Chat For Help! NO_WAN_EGRESS(TM)

                  1 Reply Last reply Reply Quote 0
                  • D
                    David_W
                    last edited by

                    @Derelict:

                    There has to be a better way (Private VLANs in your switches maybe).

                    I was thinking the same thing. I doubt that the customer mandates a dedicated VLAN per client device - what they are likely after is client devices isolated from each other. Most modern managed switches allow you to have a VLAN where ports intended for client devices are isolated from each other. Derelict suggests "private VLAN". I have seen the feature called "isolation".

                    If the original poster gives details of the switches in use then another solution might emerge.

                    1 Reply Last reply Reply Quote 0
                    • ?
                      Guest
                      last edited by

                      Derelict suggests "private VLAN". I have seen the feature called "isolation".

                      Mostly together with WiFi clients and guest and private VLANs for them to separate this both
                      groups you can surely build two VLANs one for guest and one for private usage and then activating
                      the "client isolation" so that the smart phones from the guests inside the guests VLANs are not able to
                      see the all the other guests and trying up to connect with them. Also often used in school WLAN that
                      they are not able to share all the crap over the wireless network and this is then slowing down extremely
                      the entire throughput.

                      1 Reply Last reply Reply Quote 0
                      • DerelictD
                        Derelict LAYER 8 Netgate
                        last edited by

                        Isolation across multiple APs especially across multiple switches on the same VLAN is non-trivial. The easiest way is for the feature to be built in to the wireless system or controller. But that doesn't help with guest wired ports.

                        Chattanooga, Tennessee, USA
                        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                        Do Not Chat For Help! NO_WAN_EGRESS(TM)

                        1 Reply Last reply Reply Quote 0
                        • R
                          robi
                          last edited by

                          Well, the Ethernet standard says that abut 4000 VLANs can be configured on a networking device. Low end, cheap VLAN-switches handle up to 500 VLANs. Medium category to enterprise class equipment handle 4000.
                          So expecting pfSense to handle 500 VLANs should not be something outragious.

                          There may be some certain bussiness cases when this is required, that's all. I can imagine pretty complicated industrial environments where traffic itself is very low, but network has to be designed like that. Atfer all, the standard allows 4000 VLANs in a sinlge trunk, having 500 now is only just a bit more than 10%…

                          If pfSense fails to handle this properly because of the user interface, but othewise it could, it's simply a shame.
                          But that doesn't make it a worse product.

                          1 Reply Last reply Reply Quote 0
                          • DerelictD
                            Derelict LAYER 8 Netgate
                            last edited by

                            Switching 500 VLANs in hardware and a router with 500 interfaces are two completely different things.

                            I wouldn't think a decent stack of switches would break a sweat with 500 VLANs.  Though 500 spanning tree instances might cause it some grief.

                            Chattanooga, Tennessee, USA
                            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                            Do Not Chat For Help! NO_WAN_EGRESS(TM)

                            1 Reply Last reply Reply Quote 0
                            • C
                              CaptainElmo
                              last edited by

                              500 VLANS makes perfect sense for something like a WISP, where they have 500 customers each coming in on their own virtual interface.

                              1 Reply Last reply Reply Quote 0
                              • ?
                                Guest
                                last edited by

                                So expecting pfSense to handle 500 VLANs should not be something outragious.

                                Yes for sure you are right, but then please also please on adequate sorted or strong enough hardware
                                that is able to drive this VLANs. And in the last time I see here in the forum more and more peoples
                                they let the router do really the switch jobs on top of all other things. If the need is there and for sure
                                also the traffic it must be a stronger router playing together with more powerful switches and often the
                                SMB (KMU) mid-ranged ones will be in the game, but not the really powerful ones for more money, but
                                pfSense should be then even the evil, has failures, produces problems and so on.

                                I like the way @Firewalluser was suggesting as a fast solution to get more headroom, building
                                Interface groups should be a really good point. And perhaps a Chelsio NIC from the pfSense store
                                that is able to full offload the VLAN part would be also a thing that could help a bit out here.

                                But that doesn't help with guest wired ports.

                                Yes for sure this is right.

                                500 VLANS makes perfect sense for something like a WISP, where they have 500 customers

                                There fore I was thinking perhaps the client isolation would do a good job, to prevent from the many VLANs.

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post
                                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.