Netgate 4200 Speed Default Shift Settings Causing Latency Issues?
-
Hello,
I require the lowest latency possible for a critical business application. I've been getting serious lag issues since upgrading from the Netgate 2100 to the 4200 even thought the 4200 = 2x better (2100 x 2 = 4200)! [j/k] Seriously, I certainly didn't expect worse latency.
I'm concerned that the default setting of 50 for Speed Shift is problematic. I've verified that my traffic shaper is optimized for an A+ rating (I check it weekly) on waveform and Great on cloudflare - so it isn't that.
Again, just for clarity, I wasn't having this issue with the 2100 - and everything else in my setup is the same.
I've just set Speed Shift to 20 and rebooted, but I haven't tested it yet (will be able to test it Monday) and I really need the lowest latency possible.
I'd rather not set it to 0 as I'm sure that would unnecessarily use more power and run the router hotter than necessary. Is there an optimal setting for the lowest possible lag and the best efficiency?
Thank you.
-
I just received my 4200 to coincidently replace my 2100. So far, it hasn't been good for me either with latency. When I went from a Ubiquiti edgerotuer 4 to the sg2100 a couple of years ago, my latency went up ~0.2ms which is nothing and I wasn't surprised given the low power ARM processor. When I installed the 4200, my latency went up 1ms and my RTTsd went up almost 1 ms from the 2100. I have a raspberry pi running Zabbix that monitors my ISP's gateway which is how I was measuring this but dpinger also showed an increase.
One thing I noticed and I'm not sure if it started when I imported the config or when I updated it from 23.09 to 24.03 is that the nics seem to only use 1 rx / tx queue insead of 4. I see this in the boot log:
igc3: Using 1 RX queue 1 TX queueWhen I first received it, it did say 4 instead of 1. I'm not sure if that's the problem or if it's a part of the problem. I'm going to do a factory reset to see if it's my config since I did take my 2100 config, edited the interface names and imported it into the 4200.
I tried playing around with speed shift but that didn't make a difference. The odd part is that no matter the settings, the CPU always reported 1.8Ghz on the dashboard.
If I have time tonight to try resetting it, I'll post back.
-
@Doods1130 I didn't have any latency issues yesterday. I'm still using the Speed Shift 20 setting, and will do more testing today.
I'm not sure if the CroudStrikesBlueScreens issues on Friday was part of the inter-web latency issues I was experiencing. To be clear, I do not run any windows, so I mean out there in the inter-tubes.
-
I did the factory reset last night and it didn't make a difference. Starting to wonder if I got a bad unit. It'll stop answering DNS queries once in a while I noticed yesterday. I really don't want to spend much time troubleshooting it.
-
Did another reinstall last night to install a nvme drive. When I did the reinstall, I chose 23.09 instead of 24.03. Using 23.09 seems to have resolved all of my problems. On 23.09, the nics are using all 4 rss queues instead of 1, and my latency is back down to what the sg2100 used to be.
Should I report this somewhere as it may be a regression?
-
@Doods1130 Maybe it has something to do with this:
https://forum.netgate.com/topic/138174/pfsense-vtnet-lack-of-queues/5
"You can have shaping, or you can have queues, but you can't have both on vtnet. Since pfSense uses ALTQ for shaping, that gets defined."
-
@Doods1130 redmine.pfsense.org is the bug report/feature request home.
-
Back to the original topic, anyone have guidance on optimal Speed Shift settings to minimize latency?
-
I cannot find a measurable or noticeable difference between 0, 50, and 80. My 4200 was set at 80 when I got it so I've put it back to that and left it.
-
@RKiFkRyCevGvpLeXMove said in Netgate 4200 Speed Default Shift Settings Causing Latency Issues?:
anyone have guidance on optimal Speed Shift settings to minimize latency?
Speedshift works extremely fast. Far faster that speedstep. But if you want lowest latency just disable it entirely.
The number of receive queues is set to 1 specifically for the 4200 in all versions that support it.
-
@stephenw10 said in Netgate 4200 Speed Default Shift Settings Causing Latency Issues?:
The number of receive queues is set to 1 specifically for the 4200 in all versions that support it.
Pardon my curiosity, but what's special about the 4200 that makes 1 queue the best option?
-
The maximum total throughput is significantly better when a single receive queue per NIC is used. Though if you are passing no more than 2.5Gbps (total) it is not required.
-
For the record, I set speed shift back to 80 last week, and this week has been fine. I think the issue was just a coincidence of my timing with regards to upgrading to the 4200 and the croudstrike issues.
-
This post is deleted!