Netgate 3100 + Bridge
-
Hello all.
I have a Netgate 3100, running PFSense+ 21.05.1-RELEASE
I have setup a bridge between the LAN and OPT interfaces.
The WAN port is used for management.
Just running home traffic through this bridge is taxing. Web management of the device is slow and I have not enabled any filtering yet. I think I am having an issue with interrupts.Connectivity is:
Modem -> Router -> (OTP) Netgate 3100
(LAN) Netgate 3100 -> SwitchHere is the output of the commands
[21.05.1-RELEASE][root@hostname.domain.tld]/root: top -aSH last pid: 50196; load averages: 1.57, 1.50, 1.36 up 0+08:51:08 21:28:47 140 threads: 4 running, 115 sleeping, 21 waiting CPU: 0.2% user, 0.0% nice, 0.5% system, 50.0% interrupt, 49.3% idle Mem: 22M Active, 29M Inact, 123M Wired, 58M Buf, 1804M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root -92 - 0B 176K CPU1 1 529:43 100.00% [intr{mpic0: mvneta1}] 10 root 155 ki31 0B 16K RUN 0 521:24 99.15% [idle{idle: cpu0}]1 users Load 1.18 1.42 1.47 Aug 15 21:48 Mem usage: 9%Phy 10%Kmem VN PAGER SWAP PAGER Mem: REAL VIRTUAL in out in out Tot Share Tot Share Free count Act 80276K 25788K 318016K 59128K 1805M pages All 82952K 28416K 330440K 70304K ioflt Interrupts Proc: cow 2503 total r p d s w Csw Trp Sys Int Sof Flt zfod 1145 gic0,p13: 1 44 249 12 233 2K 31 ozfod 662 gic0,p15: %ozfod gic0,s2: t 0.0%Sys 50.0%Intr 0.4%User 0.0%Nice 49.6%Idle daefr 17 gic0,s12: | | | | | | | | | | | prcfr gic0,s25:- +++++++++++++++++++++++++ totfr 8 mpic0: mvn 36 dtbuf react 38 mpic0: mvn Namei Name-cache Dir-cache 31618 desvn pdwak 616 mpic0: mvn Calls hits % hits % 999 numvn 5 pdpgs cpu0:rende 46 46 100 113 frevn intrn cpu1:rende 123M wire 17 cpu0:preem Disks flash mmcsd mmcsd mmcsd md0 21916K act cpu1:preem KB/t 0.00 0.00 0.00 0.00 0.00 29552K inact tps 0 0 0 0 0 0 laund MB/s 0.00 0.00 0.00 0.00 0.00 1805M free %busy 0 0 0 0 0 59430K buf[21.05.1-RELEASE][root@hostname.domain.tld]/root: netstat -m 1627/1673/3300 mbufs in use (current/cache/total) 1216/810/2026/1000000 mbuf clusters in use (current/cache/total/max) 1216/808 mbuf+clusters out of packet secondary zone in use (current/cache) 0/12/12/10035 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/2973 9k jumbo clusters in use (current/cache/total/max) 0/0/0/1672 16k jumbo clusters in use (current/cache/total/max) 2838K/2086K/4925K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/5/6656 sfbufs in use (current/peak/max) 0 sendfile syscalls 0 sendfile syscalls completed without I/O request 0 requests for I/O initiated by sendfile 0 pages read by sendfile as part of a request 0 pages were valid at time of a sendfile request 0 pages were valid and substituted to bogus page 0 pages were requested for read ahead by applications 0 pages were read ahead by sendfile 0 times sendfile encountered an already busy page 0 requests for sfbufs denied 0 requests for sfbufs delayed[21.05.1-RELEASE][root@hostname.domain.tld]/root: vmstat -i interrupt total rate gic0,p13: mp_tmr1 39080137 1174 gic0,p15: mpic0 22936299 689 gic0,s2: twsi0 200 0 gic0,s12: uart0 15514 0 gic0,s25:-dhci_fdt0 51432 2 mpic0: mvneta2 187977 6 mpic0: mvneta0 2189894 66 mpic0: mvneta1 20558464 618 cpu0:rendezvous 506 0 cpu1:rendezvous 2014 0 cpu0:preempt 71036 2 cpu1:preempt 6660 0 Total 85100133 2556Am I wrong about the interrupts? Is this to be expected when running bridge mode?
Thank you very much in advance for any responses!
-
So your bridge is mvneta0 and mvneta1 directly?
How much traffic are you putting through it when you see took that output above?
Steve
-
@stephenw10 Thank you for the response!
Just normal home network traffic. Is there a good way for me to graph this or quantify this? I'm happy to run commands or take screenshots to help. My home internet is only 50 Megs down and 5 megs up.
Should I be using different interfaces for the bridge?
I have moved the bridge from between my router and switch, to between my modem and router. This dropped the CPU usage down to an average of 20%. I'm wondering if my LAN is just too chatty...
-
Bridging interfaces in pfSense requires significantly more CPU than just routing. But to see one CPU pegged at 100% I would expect to see some 100s of Mbps continuously.
Is there some reason you are bridging the connection?
It would likely perform much better if you just replaced the existing router with the SG-3100 and remove the bridge completely.Steve
-
Thank you. It looks like using the bridge internally wont work due to the volume of traffic on our network. Using the bridge externally between the modem and router is working well. I'm going to start adding packages with monitoring to see if it can handle the load.
We do not want to remove the current router, and while the Netgate device is wonderful, we are enjoying the features of the current router (Unifi) . We just want the Netgate to function in transparent mode. Possibly even serving as a DNS filter. Then, if it can handle it, adding on the IPS role.
Thank you very much for the responses! In the future we might look into running a 5100 and see if it can handle the LAN chatter.
-
Ok, if you're doing that I would put the bridge between WAN and OPT and use LAN for management. That removes the switch from the connection.
