How many interfaces can pfsense handle?
-
Hey,
Just checking if anyone has experience... Do not consider hardware performance at all, only talking the software itself here - How many interfaces can pfsense handle before it "bugs out"?
We've been using pfsense as our main router for our datacenter (it's small but still), and the number of interfaces are starting to add up. We've since long separated IPSec to go on another pfsense, just to get away from bulk loads, but still we're still just a tad worried about oversizing the software.
-
https://docs.netgate.com/pfsense/en/latest/interfaces/using-a-large-number-of-interfaces.html
-
@grimson said in How many interfaces can pfsense handle?:
https://docs.netgate.com/pfsense/en/latest/interfaces/using-a-large-number-of-interfaces.html
Thanks mate. I read that too, hence the question. We're waaaaaay past 50 VLAN interfaces... That's why I'd like some comforting words from someone who has a more badass setup than me
-
Like how many?
-
Yes, just how many are you planning to use?
If you have more than, say, 250 interfaces you can expect to see delays in the GUI for example. Depending on what action you are doing.
Steve
-
@derelict
I'd rather not say as there would be competitors interested. But lets just say if we have 200+ small pfsense routers out there, we would have more connections (not always necessarily interfaces) coming in. And as I said, IPSec connections are going through another pfsense router.Honestly I didn't even count. But it's at least 150 interfaces.
-
@stephenw10
Haven't experienced that so much. More like microdelays in connectivity that makes customers complain about being disconnected from RDP sessions. But it might as well be because of really anything else. I'm really just checking out the possibilities. Our ISP delivers (as far as we've measured) 99,997% availability @ 1Gbps so it must really be our problem that these microdelays occur. According to our pfsense hardware it's idling at 5-10% cpu/ram usage, and it's using proper Intel nw chips so like I said, I'm really just grasping here. Might be a switch or something that we didn't see yet. -
Well we tested a 250 VLAN setup a while ago and fixed a bug that existed:
https://redmine.pfsense.org/issues/9115Some parts of the GUI will be slow with that many interfaces (of any type). There are things you can to to mitigate it such as not having the interfaces widget on the dashboard that gets displayed at every login.
If this is likely something you won't be changing often I wouldn't expect an issue. If you will be needing to continually add/remove rules etc it might.
I wouldn't expect issues routing traffic when changes are not being made though.
Steve
-
@stephenw10
That's really what I was looking for, thanks so much! That is what I call tested data. Now I can relax up to 250 interfaces at least
We have like 20 public IP's in our possession so next stage is dividing connections across several HA's. I'm not experienced in running a major datacentre as we grew to this size from basically nothing, and I really don't know how other suppliers do it, but I believe that's kinda secondary as long as you make it work for yourself.
Thanks for your answer!
-
Let me save you some headbanging. I've done some test with many VLANs in pfsense. Device is mostly unusable after you pass 128 interfaces, especially if running with HA. I have one install with 120 interfaces without HA with a C3000 Atom and most interactions with the UI are "traffic affecting". When You don't touch it mostly works, but I have complaints from people there which are weird and hard to troubleshoot.
So, change your network design(like i will) or find another solution. TSNR might be interesting too. Check it out, they launched yesterday.
P.S. - The interface limit may be higher with bigger boxes.
-
Yes, if you're running HA it will definitely take a larger hit on any change as everything is sync'd to the other node.
Packages that load on interface changes will also affect.
I'm surprised you found it 'unusable' after 128 though.
Steve
-
Yeah me too. I'm running vanilla, no packages. In the end I terminated everything on the switch before it(and made it L3 switch of course). Now it's one "WAN" and a couple of "LAN" interfaces and everything is peaches.
Moral of the story is: Always have plan b,c....
-
I have no experience in the datacenter, so take my post with a grain of salt...
But, if you've got the rack space available, why wouldn't you limit your pfsense boxes (real or virtual) to say 100 or less interfaces? Then when you reach a safe limit, add another pfsense box and add new customers to this new machine.
It might make for a small (or large if all instances are HA) pile of pfsense machines, but at least you've spread customers out over firewalls that aren't being crushed under the weight of too many users. I don't know, does that make this project too hard to manage, with multiple pfsense instances?
Just an idea...
Jeff
-
@akuma1x
simply because you don't want to keep in memory what customers is using what addresses. It's a bit difficult when you get up to 400+ customers divided on 4 addresses, it's much easier installing some kind of load balancer dividing all traffic on several setups. but sure enough it would work, if you wouldn't mind a bit of hassle.
