pfsense latency spikes in ESXi
-
I will try that later if I can, adding / removing and reconfiguring NICS can be done on the fly.
Note though that you are likely referring to DirectPath I/O, which passes through specific functions of the cards, not the cards themselves.
This can of course be disabled if needs be.
If I get the chance later today I will edit the WAN NIC to be E1000e and see how it goes over night.
-
While i will do some testing if time permits, this question was raised back in April, a month after my issues started (March), and the latter post suggests as many of us suspect, something changed, somewhere. It could be driver, ISP, package configuration or PfSense with ESXi and VMXNET3 specifically - since the common factors for people having issues are;
VM in ESXi
VMXNET3 adapter (in most cases, since this is the default)
ISP in the UK is Virgin Media *SH3 and SH4 known to be buggy)
WAN latencies and packet losshttps://forum.netgate.com/topic/152770/is-e1000e-better-supported-than-vmxnet3-in-pfsense/
Within the above it is recommended that VMXNET3 is the used adapter
One poster is having issues LAN side, not WAN side, however this was also posted for 2.4.5 where other known issues were fixed in P1.
My issues seemed to only start in P1
It could be (in my case at least) related to FW in the ISPs modem vs driver support in the FreeBSD OS.
-
@rod-it said in pfsense latency spikes in ESXi:
While i will do some testing if time permits, this question was raised back in April, a month after my issues started (March), and the latter post suggests as many of us suspect, something changed, somewhere. It could be driver, ISP, package configuration or PfSense with ESXi and VMXNET3 specifically - since the common factors for people having issues are;
VM in ESXi
VMXNET3 adapter (in most cases, since this is the default)
ISP in the UK is Virgin Media *SH3 and SH4 known to be buggy)
WAN latencies and packet losshttps://forum.netgate.com/topic/152770/is-e1000e-better-supported-than-vmxnet3-in-pfsense/
Within the above it is recommended that VMXNET3 is the used adapter
One poster is having issues LAN side, not WAN side, however this was also posted for 2.4.5 where other known issues were fixed in P1.
My issues seemed to only start in P1
It could be (in my case at least) related to FW in the ISPs modem vs driver support in the FreeBSD OS.
Honestly, I think in your case its just ISP related. In 2.4.5 I did have issues with larger pfblocker lists and unbound. This was patched on P1.
I'm on the same ISP as you, and I've had major issues myself. It seems they cannot do anything right, crappy modem, crappy network, crappy peering (often loss to certain places like cloud flare). Now speaking to you and the OP seems even the same model of the flawed puma 6 modem doesn't even run the same firmware (guess they're cocking up firmware updates). I've had to get my modem replaced to improve my situation + use things like Fq_codel. Still nowhere near perfect but probably the best you'll get from flawed puma chipset modems + just a overall sketchy ISP.
All in all, it's just a landmine. If possible I recommend you ditch the ISP and move onto something better. With working from home and online learning more prevalent, no one has time to endlessly faff around with Virgin. I've had no choice until recently, I've got community fibre coming to my area which is full fibre to the premises, I'm switching to that right away.
PS: I assumed everyone read the pfsense guides, I've got the bootloader.conf.local options recommended by netgate setup.
-
You're probably right, it likely is just the ISPs modem and/or the ISP themselves, the issue i have is all other ISPs round this way are of a much lesser service and while I'd prefer stability over speed, If i left, I'd also need a TV and telephone package which then come in at different costs, at present i have a fair bundle.
It's all swings and roundabouts, but I've been a VM customer for some 20 years and this is by far the most unreliable service to date, what irks me most is their inability to accept responsibility and actually do something about it - even if it is for a 2% customer base, we're the ones paying the for top tier package, either on broadband only or generically from their services, we should have a little more respect shown.
To throw it in the mix though, and I'm sure you've read this yourself, using the router as a router, does not suffer this issues, so my guess is this is more than just the ISP here, likely a driver or setting not helping somewhere.
-
@rod-it said in pfsense latency spikes in ESXi:
You're probably right, it likely is just the ISPs modem and/or the ISP themselves, the issue i have is all other ISPs round this way are of a much lesser service and while I'd prefer stability over speed, If i left, I'd also need a TV and telephone package which then come in at different costs, at present i have a fair bundle.
It's all swings and roundabouts, but I've been a VM customer for some 20 years and this is by far the most unreliable service to date, what irks me most is their inability to accept responsibility and actually do something about it - even if it is for a 2% customer base, we're the ones paying the for top tier package, either on broadband only or generically from their services, we should have a little more respect shown.
To throw it in the mix though, and I'm sure you've read this yourself, using the router as a router, does not suffer this issues, so my guess is this is more than just the ISP here, likely a driver or setting not helping somewhere.
It's most likely due to the half assed patch, and customers on a bunch of different firmware/hardware revisions. If you haven't already, try push for a Hub 4 and see if you get lucky, if nothing else get your Hub 3 replaced. Hopefully they give you one with a newer firmware which makes it a lesser pain. Do setup some traffic management, it'll help a lot. If you can, I recommend you use an app called WeQ4U to make the call, it'll wait in the phone queue for you. Their customer support wait times are through the roof at the moment...
The truth is, the SoC in their modems are just weak. On the Hub 3, as soon as I push past 100mbps, often times I notice issues even though I'm on the 350 package. These include the sudden spikes here and there, and frequent packet loss. It's improved for me though due to getting my Hub replaced, and my general area being upgraded but it's still an issue. I wouldn't be comfortable with anyone using more than 80mbps bandwidth if I wanted to have a gaming session for example.
All that being said, this ISP isn't one for gamers, enthusiasts or power users until they fix these glaring issues. I'm not 100% sure what your requirements are in terms of TV, but most ISPs do provide some kind of phone service, you can just get that and then sort out TV another way. In my household we mostly use Youtube/Netflix/Plex and then we have a sky box just for Freeview channels, this works just fine in my case but in yours maybe look at IPTV or other services?
-
I will try for a SH4 when i get time to call again, i was keen to go on their gig1 package, not for any reasons specifically, but the price and the current issues have put me in a spot where i may even reduce the package and re-instate my SH2. I'll play on wanting Gig1 though in an attempt to get the SH4 now in preparation.
I do have FQCodel in place for WAN traffic shaping and it does indeed help, but I dont even have to be using my connection for it to spike on the WAN
Weekends ironically are affected less than weekdays.
Yesterday and today I've had less than 1% packet loss with highest peak of 130ms, this is about 1/4 of usual spike.
I know they have recently done upgrades round here too - 9th December to be exact, but it was not clear what they was upgrading, i know it was planned and took out all services for sporadic times during the day.
A combination of over subscription, high overall usage in my area, dodgy boxes (as in not fit for purpose), bad FW and possible driver conflicts in FreeBSD/ESXi all add to the mix.
I appreciate your replies and sticking with this.
-
Ok, I got SH4 and the situation improved significantly right after I replaced my old SH3 with it. Terreble packet drops up to 20% are gone now, from thinkbroadband monitoring I see that overall average latency decreased for about 10%. Latency spikes of 60-140ms are present still but not so frequent that it'd disrupt my work/gaming and happen less frequently (2-4 times a day). Packet drops are happening 3-7 times a day but they're barely noticable (<1%). I've been running it for a week now and opnsense's gateway monitoring hasn't reported any latency issues nor packet drops on it.
-
Can you report back again in a week or so, I've found if i reboot my SH3 it will run for about 7-10 days with much less of an issue, after which time it starts again.
Also, during those first few days i can get to and ping 192.168.100.1 after which it's almost not there and i can no longer see it.
-
@oiyae said in pfsense latency spikes in ESXi:
Ok, I got SH4 and the situation improved significantly right after I replaced my old SH3 with it. Terreble packet drops up to 20% are gone now, from thinkbroadband monitoring I see that overall average latency decreased for about 10%. Latency spikes of 60-140ms are present still but not so frequent that it'd disrupt my work/gaming and happen less frequently (2-4 times a day). Packet drops are happening 3-7 times a day but they're barely noticable (<1%). I've been running it for a week now and opnsense's gateway monitoring hasn't reported any latency issues nor packet drops on it.
Glad its working out better for you, as soon as I saw the hardware revision and software much lower than mine thought it might be a factor here.
-
@rod-it said in pfsense latency spikes in ESXi:
Can you report back again in a week or so, I've found if i reboot my SH3 it will run for about 7-10 days with much less of an issue, after which time it starts again.
Also, during those first few days i can get to and ping 192.168.100.1 after which it's almost not there and i can no longer see it.
Funny you mention that, I had that happen yesterday, couldn't access the management page or ping the router. Restarted it, and all was well and good. Thought it could be just a red herring.
-
How has it been going since?
How did you get them to offer up a SH4 by the way, the people you usually speak to are non-=technical and quite often refuse
-
T tman222 referenced this topic on
-
T tman222 referenced this topic on
-
T tman222 referenced this topic on