Have I set this up right or am I being stupid?
-
As not to reopen an old post but may be related to sendto: 65 with UK ISP and PFsense
So after posting the above the random drops stopped so all was happy until about 3 weeks ago. While the wife was working the net would randomly 'hiccup' meaning webpages would need a refresh to get going and her citrix session would advise of loss of connection.
As part of diagnostics we ran pings to the ESXI host, the PfSense VM and bbc.co.uk and all appear fine, ISP was contacted again and they can't see anything on their network so pushed back to me and my kit. Now before caving in and replacing the PfSense router I would like to check I have done this correctly.
ESXI is 6.7u3, PfSense is 2.5.2 RELEASE with 4 vCPU cores, 8GB Ram, 50GB HDD (ufs).
The ESXI host is a HP Z620 workstation i've repurposed for a Valheim Server, Unifi Controller and PFsense currently (it has a pair of E5-2680 v2 CPUs and 64GB Ram). The NICs are onboard Intel 82579LM and an onboard Intel 82574L but i do have a 4 port Intel I340-T4 if that would be better.
So currently the VM has 3 vNICs:-
1 for WAN
1 for LAN
1 for a Segregated network i play with rules etc.All 3 are set as VMXNET3 adaptors.
Does this seem right?
-
Yes, that should be fine.
I would try to determine what's actually failing when it hiccups.
A DNS issue would be my first suspect. IPv6 can do odd things if it's partially configured.
Gateway flapped and defaulted to something internal?
Steve
-
@stephenw10 Thank you for the reply.
When it hiccups it's like any established connections drop and have to re establish if that makes sense. I.e I'll be dropped from a game, netflix will buffer music streams will stop etc.
My ISP doesn't yet do IPv6 so I think I have that off. As for DNS I thought I had set it to cloudflare DNS where is best to check this? I guess an nslookup? I don't have an internal DNS setup (other than the PFsense box) but that being said I do have pfblocker could it be there?
-
So this morning I added google dns (if i've done it right). Attached i've added my current dashboard.
Some pings i did this morning to the PFsense VM (254 address), The ESXI host (250 address), cloudflare dns and google dns.
My Thinkbroadband quality monitor for yesterday, ignore the red bar as that was me taking the router offline to try literally anything.
-
Ok, if existing streams are cut it's probably not DNS.
I note in your screenshot you have only one gateway and it's showing as offline. Is that because you were doing something at that point that disconnected it?
It also looks like you're monitoring the gateway IP itself (the default) which won't show you any issues upstream of that.
Try this; edit the gateway in System > Routing > Gateways and set an external Monitor IP. I would use 8.8.8.8 there since you've proved it responds. Also setDisable Gateway Monitoring Action. Since you only have one gateway there is no need for it to trigger a reload but you still want to log the gateway status.Steve
-
@stephenw10 The gateway changes between 2, 244 and 243. 244 seems to never respond yet 243 does. My ISP has advised these are gateways to their network not the internet breakout.
I never actually set the gateway it appears to be picked up from PPPoE.
I've added as you suggest and will see what happens.
-
Yeah, the ISPs gateway does not have to respond to ping. My own connections, with BT, do not. Setting an external monitor gives better data anyway.
The gateway should only ever change between PPPoE sessions. If you are seeing it change that implies the PPP session is bouncing which would definitely break open connections.
Steve
-
@stephenw10 Well I'll run it like this and see what happens. The missus plays Sea of Thieves and i don't think i can take anymore "IT'S DONE IT AGAIN!!" conversations lol.
We do have an AltNet threating to install symmetric 1Gbps fibre which would be a godsend for the ISOs i have to send around and means i can drop a point of failure (the Vigor166) out of the network loop.
As an aside this is what my TBB quality check looks like when pfsense shows the 244 gateway.
That to me looks like congestion but my ISP assures me that is just TBB just misreporting. -
Are you on g.fast? Is that an imported V166? When I was looking into this (which was a while back!) Draytek hadn't started selling those in the UK yet.
Yeah that graph looks bad. See what the pfSense monitoring graphs look like against 8.8.8.8 after a while. I wouldn't really expect any packet loss unless you're filling the WAN entirely.
Steve
-
@stephenw10 No not on g.fast and i bought it in the UK
-
Mmm, interesting. Let's see what pfSense shows against 8.8.8.8 then.
This is what my WANs look like over 2 days:
0% loss on both and minimal latency change. WAN is standard FTTC VDSL, WAN2 is g.fast.
That's monitoring against 8.8.8.8 and 8.8.4.4.Steve
-
@stephenw10 Where is that graph hiding? I'm still getting used to the interface
-
In Status > Monitoring. Hit the 'wrench' icon to configure the graph.
-
@stephenw10 Thank you again for your help, i'll report back in a few days :)
-
So ran it with your suggested change for 2 days.
That is just a 80/20 FTTC connection but i think i have been banded as my Draytek show interleaving on the down stream.
Hopefully DLM will kick in after a few days and sort it out.
-
If you enable the WAN traffic graph on the other axis you can check to see if you were moving a lot of traffic when you saw the packet loss and latency. I don't really expect to see any loss.
I still use an old HG612 there and my line looks similar:
# xdslcmd info --state --stats xdslcmd: ADSL driver and PHY status Status: Showtime Retrain Reason: 0 Last initialization procedure status: 0 Max: Upstream rate = 26292 Kbps, Downstream rate = 73580 Kbps Bearer: 0, Upstream rate = 19999 Kbps, Downstream rate = 66999 Kbps Bearer: 1, Upstream rate = 0 Kbps, Downstream rate = 0 Kbps Link Power State: L0 Mode: VDSL2 Annex B VDSL2 Profile: Profile 17a TPS-TC: PTM Mode(0x0) Trellis: U:ON /D:ON Line Status: No Defect Training Status: Showtime Down Up SNR (dB): 8.1 15.3 Attn(dB): 13.6 0.0 Pwr(dBm): 14.0 3.0 -
@stephenw10 I see a few spikes (I expect steam updating)
-
Sorry for the delayed response been away for work.
It appears whatever was the cause has gone. Been over a week and no connection drops.
-
Nice, something upstream then. Good result.
-
@stephenw10 Afraid to say about 10:30am the internet died. This was followed by the usual unimpressed look from the wife as she was knee deep the the latest season of the witcher.
From the pfsense dashboard the WAN interface shows down
I can connect to the web GUI of the modem and that looks good.
From here i've done the usual tests.
VDSL Status
ping and nslookup from desktop
DNS lookup from pfsense
Monitoring from pfsense
Then I tried to even force it
Ultimately I went to the logs.
I rebooted pfsense, nothing. Rebooted the Draytek Vigor 166 modem and it came back to life. I'm kinda stumped at this point so going to ask Draytek support incase i've missed some sort of logging their box does but if you guys see anything i've missed please let me know.
