Slow Web GUI with many VLAN Interfaces - 300$
-
Hi,
we have pfSense system with about 350 VLAN interfaces.
As stated here: https://forum.pfsense.org/index.php?topic=102607.0 : and here: https://forum.pfsense.org/index.php?topic=101448 the GUI becomes very slow and assign_interfaces.php becomes unusable.There is already a bug report: https://redmine.pfsense.org/issues/6400 where this problem is described, but the Fix got moved to pfSense version 2.4.1.
We will pay 300$ if this can be fixed within the next two weeks, since we will add another pfSense system with that number of interfaces.
At the moment we add,edit or remove interfaces via the developer shell, but this is very annoying.Some more information about why we need so many interfaces:
We manage a modern student dorm with about 300 apartments. Each student has its own separated ip subnet (via VLAN). The VLAN Tag is either assigned via dynamic RADIUS VLAN assignment (WLAN) or via the Switch (untagged LAN Port at the apartment).
After loggin into the WLAN, students can easily stream content from their smartphones/tablet to a connected AppleTV/Smart TV etc.
There is also KNX Smart Home technology added so that students can control their lighting, heating, jalousie, sound etc. via Smartphone/Tablet/Voice.pfSense hardware:
Xeon E3-1240v6 4x 3.70GHz
16GB DDR4-2400 ECC RAM
MB Supermicro X11SSH-LN4F
SSD 240GB RAID 1
Chelsio 10Gbit T520-SO-CR
WAN: 1 Gbit/s Up/Down Business Fiber ConnectionIf there is more information needed I'm glad to help.
Thanks in advance! -
Hi,
Try these files (back up your existing ones first!) On my testing with 100 VLANs, the page load time is about 6-7 seconds vs 14-15 seconds with the default pages.
All functionality should still be present and working.
Files: /etc/inc/interfaces.inc
/usr/local/www/interfaces_assign.phpEDIT: updated archive with bug fix below.
EDIT2: another bug fix
[slow ui.zip](/public/imported_attachments/1/slow ui.zip)
-
Has this helped at all?
-
I have to do a little bit more testing the next days but on the first look it's great!
I can open interfaces_assign.php again now and it's a lot faster.
After more testing I'll pm you :)EDIT:
When I add a new VLAN it doesn't show up on availible network ports to add a new interface. -
-
Everything working as it should?
-
Another bug squashed, seems ppp stuff got broken, so here's fix:
restore the backed up interfaces.inc in /etc/inc and then use the attached files:
files: /etc/inc/interfaces_fast.inc
/usr/local/www/interfaces_assign.php[slow ui.zip](/public/imported_attachments/1/slow ui.zip)
-
Guys this is great and all but if you're going to post these publicly I think you might as well work these patches via github so they can be diffed and merged along with the rest of the project, gets more eyes on them as well…
-
I don't use git and this particular 'solution' is a placeholder because it's already been fixed officially, the powers that be just aren't ready to release the official fix yet, and probably won't be in the near future.
-
this particular 'solution' is a placeholder because it's already been fixed officially, the powers that be just aren't ready to release the official fix yet, and probably won't be in the near future.
?? could you point us to the redmine that claims it is fixed ?
-
but the Fix got moved to pfSense version 2.4.1.
I read that as a fix exists but isn't going to be made public yet.
-
Bad news: pfSense 2.4 is even worse :(
good news: I fixed it :)
Weird news: Exactly the same edits that shaved 7 seconds off before now shave 2:20 off ::)
Files: /etc/inc/interfaces_fast.inc
/usr/local/www/interfaces_assign.php
/usr/local/www/interfaces_vlan.php[slow ui 2-4.zip](/public/imported_attachments/1/slow ui 2-4.zip)
-
Has this been fixed in 2.4.1?
-
-
Has this been fixed in 2.4.1?
not officially, my fixes should work.
Can you please submit a Pull Request on https://github.com/pfsense/pfsense so we can review and merge the fixes?
-
I did some testing with multiple vlan interfaces using standard pfSense 2.4.1 to get a better view of the problem.
Boot time with 001 vlans: 0 min, 45 seconds
Boot time with 050 vlans: 0 min, 49 seconds
Boot time with 100 vlans: 0 min, 59 seconds
Boot time with 200 vlans: 1 min, 52 seconds
Boot time with 300 vlans: 4 min, 19 seconds
Boot time with 400 vlans: 9 min, 16 seconds
Boot time with 500 vlans: 13 min, 19 secondsAs you can see the boot time is not linear. Maybe this helps with finding and implementing a fix. Because a fix needs to address this non-linear groth as well. When pfSense is finshed with booting with 500 interfaces the web GUI just gives a 504 after some time. Using 400 vlan interfaces and less the web GUI is slow. Then I tried with loonylion patch which he posted October 19 in this tread using 300 vlans.
Boot time with 300 vlans, loonylion path: 3 min, 51 seconds
To GUI is slow with 300 interfaces. Even with the patch from loonylion. I didn't notice a difference.
Test where done using a Intel(R) Xeon(R) CPU E3-1585 v5 @ 3.50GHz, 16GB of memory.
Hope this helps.
-
my patch will not affect boot time in the slightest, it's a GUI modification only. There must be something else at play to get those results.
-
Has this been fixed in 2.4.1?
not officially, my fixes should work.
Can you please submit a Pull Request on https://github.com/pfsense/pfsense so we can review and merge the fixes?
I'll tidy up the code and try to figure out how to do this; I've never used git before.
-
I know this doesn't really solve the issue but isn't having 300+ interfaces off a firewall kind of crazy? I probably would virtualize your PfSense and have several PfSense vm's running in the same box and try to get the job done like that. Are there switches out there that can handle 300 vlans? Again I know this doesn't solve the root issue just seems like a engineering issue.
-
I'm in the process of tidying the code up and making sure it adheres to the pfSense coding guidelines. I also decided to do a bit more profiling of it, and I came up with the attached graph. At 500 VLANs the page load time with my patches is 43 seconds as measured by FireFox. The original code times out as mentioned above. Also noted above, the time doesn't increase linearly with the original code, and my graph shows that.
The graph shows page generation time, actual load time as experienced by the user will be a bit longer. These times are for viewing the interface_assign.php page; for adding an interface add ~3 seconds to page generation and for deleting add ~2 seconds.
