WAN Package Loss
-
@derx05
I don't really think that suricata can cause this issue, as the gateway monitoring is an outbound connection from pfSense itself. But I'm not experienced with it, since I don't use it or something like that.Yes, when posting log files, you should remove / hide public IPs. But as your pfSense is behind a NAT route, it won't have any anyway.
Since the interface status doesn't show any errors or collisions, I'd also consider to replace the FritzBox for testing.
The whole gateway monitoring happens between pfSense and FB. As you mentioned, you have already replaced the pfSense hardware and the network cable. So the last hardware part is the FritzBox. -
Can you eliminate the Fritz box or put in modem mode.
-
@viragomann
Oh I think I forgot the write in the first statement that I don‘t have these problems with the 3100. If it would be the fritzbox it would also cause these problems with the 3100 I think. I will download the logs files on my pc later and post them here after I looked through them! -
@nogbadthebad Sadly a modern Fritzbox doesn‘t even have a modem mode anymore and cause of our provider I can‘t use another modem. They are a local fiber provider and the Fritzboxes are used for metrics and debugging so I am not allowed to directly plug the WAN Cable from the FritzBox in the pfSense. Thats really sad. Of course I put a static route in the FritzBox and disabled the NAT on pfsense, so PortForwarding works like without a FritzBox in between. Also like I said. With the same settings it worked on the SG 3100… So I am a bit helpless
-
So here are the logs!
I just want to mention, that all logs before 10.02 about 12 o‘clock are useless cause thats where I switched hardware from the ProLiant Server to the 6100. So I think you should look at the logs from this morning, cause we had about 3 packages losses today!
I looked through them and don‘t think that there are any public IPs or else in so should be safe to post them here! -
30% packet loss like that 'feels' like an IP conflict, especially when it's inside the same subnet.
I would expect to see that reported in the logs though.
Steve
-
@stephenw10
You mean an IP conflict in the subnet from the FritzBox and the pfSense? Well in this subnet are only 2 devices! The FritzBox and the pfSense. Directly connected via a LAN cable. So I don‘t know how an ip-conflict can be created here. Also again. With the 3100 it worked. I am worried that this is a driver or software related issue, what shouldn‘t happen cause it’s a netgate device -
Yes, that's what I mean. 30% loss is massive.
Something wifi connected maybe?
It's very unlikely to be a driver issue with the 6100.
Have you actually swapped back in the 3100 since this started to prove it is still unaffected?
So you have the WAN MAC spoofed?
Steve
-
@stephenw10
Sometimes it’s about 50%!
Wifi on the FritzBox is off so no Wifi Connection!
Yes for the last 2 days before the 6100 arrived I switched back to the 3100 and everything worked fine!
No I don‘t have the MAC spoofed. When I switch the firewall I change which MAC is the exposed host in the FritzBox. -
Hmm, well as I say I would expect numerous entries in the system log if it was an IP conflict.
I would be running a packet capture when it happens then. Se if there is anything unusual happening with the traffic.
Steve
-
@stephenw10
Ok i already did a packet capture this morning while we were experiencing the package loss and couldn‘t find any suspicious traffic but also I have to admit I am not very familiar with Wireshark so could also be possible I miss something.I am not sure if its wise to post the data here cause it contains raw network traffic. On the other side all important traffic is encrypted these days and IPs from public servers can be shared?
Shall I just post it here?Also a note to something I said before: Since I disabled promiscious mode in suricata I didn‘t have a loss according to the logs. So maybe it has to something promiscious mode?
-
Hmm, well seems likely though I would not expect it.
You can PM me link the cap if you want.
-
Update: So over the weekend the problem sadly didn't occur again.
@stephenw10 As promised I will send you a much longer packet capture again when it should happen again. I din't change any settings so it should be only a matter of time. Also I switched the LAN cable between the Fritzbox and the pfSense again, just to make sure.
Will keep you updated here. -
So as promised an update:
The error occured again twice today.
Again no useful logs before the loss starts. I just sent @stephenw10 my logs and a 10 minute long packet capture. If we can't solve the problem with these logs and the capture I honestly don't know what else I could try... -
Can you try monitoring a public IP like 1.1.1.1 for the affected WAN gateway?
It could be an upstream interferer on your cable segment. Cable devices can then lose their heads if they lack sync. -
@nocling With an upsteam interferer you mean a problem on the WAN side? Just for your note: We have fiber, not cable. I don't think an upsteam interferer can happen with fiber or am I wrong? But also worth a shot. I think I will change that when I am back at the company next week
-
@nocling
Today I also changed the monitoring IP but still have the package loss. So I will now reset the 6100 and then set ip back up without restoring the config. -
So I hope this will be the last update I will have to write. After a long call with the tech support of our provider I ask for the permission to plug the wan directly in the pfsense. I never liked FritzBoxes and this case gives me one more reason to never buy one again. Since 4 days no more problems! I set the PPPoE connection up in the pfsense, set up the MTU size and vlan and everything just works. I don't know what the FritzBox did, that the connection between the pfSense and the FritzBox got so laggy but I would definetly say it's the FritzBox fault. Someone yesterday also told me it could have been the MTU size. But sadly FritzBoxes don't offer settings for that and I also tried setting the MTU to about 1300 3 weeks ago and it didn't help. So I think this is an issue that will never be solved but honestly I don't care anymore.
I don't expect any more problems the next days so I hope I won't have to write again here.
Also I can't explain why the FritzBox worked with the 3100, which was the reason why we tought the FritzBox couldn't be the cause. Seems I was wrong.
Thanks for all your help! -
Nice result! That's a much nicer setup too.
-
@stephenw10 Yes it is indeed. I hate to do something like double NAT with static routes. The next time a provider tries to force me to use a FritzBox I won't sign a contract with them :-)
