Haproxy - Mobile Networks in UK completely broken
-
@stephenw10 yeah thought so. Just tested it on an android phone. It all seems to be working apart from one of the services I host but no one else is having the issue.
Problem I have right now, is that no that I know can reproduce the problem there end. So I am not sure where to go from here now.
-
I would send some traffic and see if it arrives at your WAN. That will tell you for sure if the mobile network is filtering it.
-
@stephenw10 What would you use for that ? Wire shark ?
Regards
-
Run a pcap in pfSense initially just to see if anything arrives. Filter it by the mobile IP you're testing from. That's easier if you get a real IP. I know 3UK still give you a real address.
-
@stephenw10 Tested with pcap and used Wireshark to read the packet capture, I am seeing a lot of re-transmissions on Public facing WAN, I tested internally to see if i get the re-transmissions and I can say it's only on the Public facing WAN side.
Regards
-
You saw retransmissions from the remote test client coming into your WAN?
That implies haproxy did not reply to them. Or the replies never arrived.
-
@stephenw10 I've had a few friends test on both Celluar and DSL. The capture shows a lot of Re-transmissions going out and In I believe. It is worst on Celluar than DSL though but i guess that makes sense.
217.45 is one of my Static IPs, 82.13 is a friends Virgin Internet connection, You can see there is a couple of Re-Transmissions, now I dunno if this is causing a problem or not.
Regards
-
Ok the duplicated ACKs there imply the remote host is not seeing the packets pfSense is sending, and re-sending. For some reason.
I assume that remote host is seeing problems connecting the site hosted at that static IP.
-
@stephenw10 The client seems to connect to Nextcloud okay, and download files, one thing i have noticed is there are poor speeds out of the WAN too, I have a 18meg up speed but the Client is only seeing 1mbp down when downloading a file.
What would you suggest to do next? because i am complexed by this.
Regards
-
I would test at the client end to confirm that when it's sending ACKs like that it isn't seeing the packets we are sending.
Then I would probably try setting a much lower MTU/MSS on something and see if that makes any difference.
-
@stephenw10 Yeah MTU is set to 1492 and MSS is to 1407 these are what BT Business recommends. I mean you think adjusting will help?
Regards
-
I have seen networks failing to pass packets that large and also failing to send the required ICMP-packet-too-large replies. So yes I would try 1300 to be sure.
You can see the packets pfSense is sending at 1411 but I believe that's the frame size. That's neither 1492 or 1407 though.
But pcap at the remote side first to be sure they are not reaching it.
-
@stephenw10 1411 is odd frame size for a 1500 mtu.. Wireshark would normally show those as 1514.. If it was a full frame
So 1411 is either not a full frame or the mtu is not 1500.
-
Exactly. And it's not anything I'd expected from setting MSS to 1407 either. Implying something in the route the is running a low MTU and sending back ICMP notices. But maybe not low enough.
But also 1407 seems like a weird value for that. I'd double check it.
-
@stephenw10 Changed to 1300, but still the same problem. Not sure where to go from here now. I did check on Client too each side shows both in Wireshark.
Regards
-
So it does show the retransmissions from pfSense arriving at the client?
-
@stephenw10 Good evening, I’ve given up with it for now.
I believe this problem is something to do with Haproxy. All services that is routed through Haproxy keeps dropping out even internally.
So I am not sure whether if it’s the latest version of Haproxy or 2.7.2.
Regards
-
Good afternoon,
Would like to update you, the problem seems to be resolved. I believe the problem was due to Public DNS Servers on Mobile Networks.
Regards
-
Ah, interesting. The first rule of troublshooting: it's always DNS!
Though I wouldn't have guessed that here fro what you were seeing.Glad it was resolved.
-
@stephenw10 I dropped the MTU, MSS on WAN1, but the DNS lookup seems to be fixed with Vodaphone, Three. I do plan on moving Name Servers to my own VMs so i won't have to rely on someone else.
All errors in Wireshark seems to be resolved too so those errors could be related to the ISP. I will keep an eye on it though
Regards
