Packet loss with 2100 on fibre line
-
I have some odd behaviour with my 2100, any pro-tips appreciated:
I have a Netgate 2100 with mostly a default setup as per Lawrence Systems YouTube video :). Anyway, I'm on a 500Mbps fibre line and works as advertised. When I download a very large data set, say an AI model from Hugging Face (40+ GB) I get about 5-10% packet loss as reported from the monitor. I get this could just be the ISP dropping ICMP because the pfSence dashboard reports ~68 MB/s which seems quite resonable. If I put in pfBlockerNG it jumps to about 10-20% packet loss but the same ~68 MB/s. If I try a long test with fast.com I get ~68 MB/s and 0% packet loss. CPU gets up to about 70-80% mostly interrupt (50%) in all cases.
Normally I wouldn't think this is a problem except packet loss goes up when I make add in pfBlocker. It makes me think the CPU is over taxed and cannot keep up so its dropping packets. The device is advertised at over 900Mbps through the firewall so I would think about 520Mbps wouldn't make it sweat.
Thoughts?
-
@brainbox1100 at 68Mb/sec on a 2100 you should expect a fair amount of packet loss as you are on the absolute interrupt/processing limit of the device. Any service - even the smallest one - will cost on throughtput or by additional packetloss (and throughput) depending on latency and when the CPU cannot keep up.
-
-
@brainbox1100 That’s the theoretical optimal throughput with all packets @ maximum size in a single streaming session. Maximum size matters A LOT because its less interrupts for more data.
In reality your firewall handles lots of packets outside your download (DNS lookups, ICMP, broadcasts, port scans and lots and lots of http keepalives from clients holding sessions for tracking and add services). The number of firewall rules makes very little difference, but the number of active states is what really costs on the CPU.
All those other packets are outside the streaming session and costs continious states lookups - and all those packets are less than one 10th the size of the maximum packet size - causing more interrupts for less data (eats CPU).
When all that is accounted for, you start getting closer to the IMIX throughput specified for the firewall - which is the real number you should be looking at. The SG-2100 is rated at 300Mbps’ish i full Firewall IMIX and about 600Mbps in L3 forwarding IMIX.
So your number is actually very very good - and higher than I can get from my 2100 (probably because of a higher client count = more IMIX like environment here). -
@keyser Thanks for explaining this.
