The strange story of accessing certain websites.
-
Run a pcap, look for differences. I'd check the TTL first. Thought it looks like the loop is exhausting the TTL.
Edit: I see you did!
-
@stephenw10 said in The strange story of accessing certain websites.:
Edit: I see you did!
I don't think its a loop anymore, server is answering with a fin ack directly
dead on arrival, nowhere to be found.
-
Were those pcap filtered?
The non-working one does have the initial syn / syn-ack / ack handshake.
It appears to initially connect to something via http and get's redirected to https. That can see the browser sending the request.
In the failed connection we are not seeing all the traffic in that pcap. Could there be some route symmetry somehow? Loadbalancing between the WANs? You do still occasionally see sites that can't handle that.
-
@mcury said in The strange story of accessing certain websites.:
server is answering with a fin ack directly
The client is sending the fin-ack. But the fin is not shown. The client must have seen it though.
-
@stephenw10 said in The strange story of accessing certain websites.:
@mcury said in The strange story of accessing certain websites.:
server is answering with a fin ack directly
The client is sending the fin-ack. But the fin is not shown. The client must have seen it though.
so, if its not MTU, or a loop, and work with some browsers and not others, it must be something with the website/browser compatibility.
this wouldn't be the first time that this site presents problems with browsers, but the weird part is that in the past, it was Firefox that had problems with it.. at that time, a banner would show up showing "outdated browser detected".
-
@stephenw10 said in The strange story of accessing certain websites.:
Were those pcap filtered?
Yes, but both filtered the same way. ip.addr == 13.0.0.0/8
It was captured on pfSense. I'll try it again on the PC side@stephenw10 said in The strange story of accessing certain websites.:
. Could there be some route symmetry somehow? Loadbalancing between the WANs? You do still occasionally see sites that can't handle that.
No load-balancing currently. I have tried to disable WAN2 interface completely, same behavior exists.
-
I've found just setting MTU can be insufficient. Try also setting MSS to the same value.
-
@stephenw10
edge_less_filtered_upload.pcap
Direct capture from PC, filtered unnecessary LAN traffic only. -
@dem
Can't set it right now, may be tomorrow morning... -
@mcury said in The strange story of accessing certain websites.:
Does this issue only happen with that website ?
Sorry for the late response. I have similar problems with some microsoft pages, like https://answers.microsoft.com/en-us and again, firefox is OK and EDGE is failing to display the page correctly like it is in text mode partially or corrupted, same I see on android phone
-
Ok. Looks like the problem is solved!
I just went to the WAN1 and removed MTU and MSS completely!
No
It was just some cached result, whatever... not working again
Setting MSS also have no effect -
The problem is isolated, and it seems that some network setting or equipment of my network is to blame because when connecting the laptop or this PC directly with a cable to the ISP and establishing a PPPoE connection, all websites are accessible and everything works fine.
Edit1:
I have CARP and now switched to the secondary firewall and looks like the problem is gone, this time it is isolated to the next step — only when primary firewall is active.after a while, this is not true anymore, because EDGE refuses to access the site again.
Tried playing with TSO/LRO settings, compared routes, don't see much difference. Yes, the hardware on the firewalls is different, but by adding LAGG interfaces, they are brought to a "compatible" state for synchronization. The rules and settings are almost identical.
I got lost... what's next?
Edit2: I just went a bit crazy and set the MTU to 1200 on the primary firewall... The result is that I can now access and ping micron.com
Edit3: after a 20 min. i can still ping micron.com but the site is no more accessible in EDGE, but firefox works.
WTF, magic?
Edit4: Same story on the secondary firewall after a while. Will re-test on the second ISP (WAN2) and will re-test on direct connection to the WAN1 ISP... -
@w0w your pcap for edge is showing errors for K-Meleon.
Check packet 15 of edge_less_filtered_upload.pcap<html><head><title>Encountered a 404 error</title> <body> <p>An error has been encountered in accessing this page. <p>1. <b>Server:</b> kmeleon.sourceforge.net <br />2. <b>URL path:</b> /update.php <br />3. <b>Error notes:</b> NONE <br />4. <b>Error type:</b> 404 <br />5. <b>Request method:</b> GET <br />6. <b>Request query string:</b> NONE <br />7. <b>Time:</b> 2024-04-06 17:30:09 UTC (1712424609) <p><b>Reporting this problem:</b> The problem you have encountered is with a project web site hosted by SourceForge.net. This issue should be reported to the SourceForge.net-hosted project (not to SourceForge.net). <p><i>If this is a severe or recurring/persistent problem,</i> please do one of the following, and provide the error text (numbered 1 through 7, above): <ol><li>Contact the project via their <a href="https://sourceforge.net/support/prweb-lookup.php?host=kmeleon.sourceforge.net&support=1">designated support resources</a>. <li>Contact the project administrators of this project via email (see the upper right-hand corner of the <a href="https://sourceforge.net/support/prweb-lookup.php?host=kmele -
Since making changes there appear to allow it to work initially did you test the direct PPPoE connection from a laptop for long enough to sure that too wouldn;t fail after some time?
-
@stephenw10
I just re-tested on PC, long enough, no problem.@mcury said in The strange story of accessing certain websites.:
your pcap for edge is showing errors for K-Meleon.
I think we can ignore it. K-meleon was running in the background.
-
@w0w said in The strange story of accessing certain websites.:
Edit2: I just went a bit crazy and set the MTU to 1200 on the primary firewall... The result is that I can now access and ping micron.com
Your pcaps don't point to any MTU problems, I don't see packet out of order, retransmissions..
-
It seems the issue is somehow related to IPv6. WAN1 is using IPv6, and I'm getting all the addresses normally, and even online IPv6 tests are passing without any problems. But the fact is, I've disabled the use of IPv6 now, and it seems like everything is working.
Now I don't really know what to do about it and which direction to investigate. Similarly, when connecting directly, IPv6 works perfectly fine in Windows. -
@w0w check if these can help you.
Note: I just disable IPv6.https://docs.netgate.com/pfsense/en/latest/recipes/multiwan-ipv6.html
https://docs.netgate.com/pfsense/en/latest/multiwan/considerations.html -
Aha, nice catch. That would explain the hosted VMs I guess. And you just don't see the v6 traffic in the filtered pcap.
-
@stephenw10
Yes, hosted VM does not use IPv6
Only WAN1 utilizes IPv6. In principle, Multi-WAN is used in failover mode only for IPv4 (Tier 1 for PPPoE-WAN1 and Tier 2 for WAN2). At the moment, I don't understand what impact IPv6 has in this case.
AFAIK micron.com does not use IPv6
also
this was checked next to allow IPv6
So what can be wrong with IPv6, any thoughts?
