Able to ping, nslookup and curl in pfSense box but curl failed in clients
-
@rcoleman-netgate This is the result of packet capture (time in pfsense box slightly different with my laptop)
-
@mltobing What's between your pfSense and the world?
This suggests the issue does not lie in your pfSense but on the next step out - it goes in the pfSense and out the WAN port but nothing is coming back. -
@rcoleman-netgate That really strange. There is ISP modem after pfSense box. I removed pfSense box and change it with OpenWRT AP all working fine.
If you think like that, now I am not sure if this issue related to Intel Atom box or the NICs
-
@mltobing Are you running any type of VPN?
-
@rcoleman-netgate I am not running any type of VPN
I forgot if we cannot curl, because of that we didn't get any reply. But we have no issue with ping, so I tried to capture and we got reply.
Strange. Why pfSense box allowed ICMP and nslookup but blocked other traffic. On my laptop I checked the network status "internet access" but I cannot browsing the internet.
Thanks for your fast response. I will go out and reply you later
-
@rcoleman-netgate I tried this scenario to get packets from OpenWRT. I ran curl on my laptop first then pfsense box
pfSense WAN captured packets from both of them but didn't forward requests from my laptop to OpenWRT. We can see OpenWRT captured packets after 20:55:48 only. Do you know why pfSense WAN didn't forward packets from my laptop?
-
It's not that pfSense is not forwarding the responses it's that it never gets any responses to forward. For some reason.
There must be some difference between the packets from the client and those from pfSense. The TTL would be different for example.
The pcap on openwrt doesn't show any of the traffic from the laptop behind pfSense. Was is started after that had failed?
Steve
-
My number one suspect here would be the USB NIC you're using except you have that as LAN and it appears to be passing inbound there.
What is the WAN NIC in that device? What hardware off-loading do you have enabled?Steve
-
@stephenw10 The NIC assignment were correct. If I set them wrong, WAN interface didn't get any IP Address.
Because this is initial setup I didn't change any hardware offloading setting.I tested again and this time I captured OpenWRT LAN interface. pfSense WAN directly connected to OpenWRT LAN.
Traffics forwarded from my laptop don't have TS val, but from pfSense have.
-
Ok so pfSense is sending that traffic as expected and OpenWRT sees it on the LAN but no replies.
Are there any replies on the OpenWRT WAN?You're probably going to need to open that in Wireshark and look at it more closely.
You might also connect the laptop to OpenWRT directly and pcap the same traffic there when it succeeds.
https://en.wikipedia.org/wiki/Transmission_Control_Protocol#TCP_timestamps
TCP timestamps are enabled by default In Linux kernel.,[27] and disabled by default in Windows Server 2008, 2012 and 2016.[28]
Steve
-
@stephenw10 I am not familiar with packet inspection. Is there something I need to check ?
This is the packets capture result on OpenWRT LAN interface
Top = my laptop => pfSense => OpenWRT (unable to connect to internet, no reply from OpenWRT WAN)
Bottom = my laptop => OpenWRT (able to connect to internet)
-
OK, it looks like you may have some asymmetric routing somewhere.
In the working pcap you can see there is traffic in that TCP session that did not pass the interface.
By far the most likely is that the laptop you're testing from has some other connection. Like maybe it has wifi directly to the ISP "modem".
pfSense will block out of state TCP traffic like that. Lesser firewalls may not.
Do you see any blocked TCP traffic in the pfSense firewall log?
It's still hard to see how that could happen though because each device is source NATing on the way out...
But that fact the ping works also points to that.
Steve
-
@stephenw10 I did so many tests but traffics filtered by destination only these
-
Hmm 192.168.1.101 should not be blocked inbound on LAN like that. Assuming that's still the LAN subnet?
But if it was that traffic would not appear on the pfSense WAN
Are you able to upload those pcaps so I can review them?
https://nc.netgate.com/nextcloud/s/9jFb69WaCHzfSRBSteve
-
@stephenw10 That is a SYN packet - it could be because of a closed state or a bad state reference (like you commented about asymmetry).
-
SYN should be passed. Any other flag combo might be dropped. Hard to see where another route might exist though since the laptop is already connected via wifi.
-
Files uploaded. I thought I disabled the wifi during the test
-
@rcoleman-netgate @stephenw10 @viragomann and the others on this thread. Thank you very much for your help.
This issue resolved after I checked "Disable hardware checksum offload" on System / Advanced / Networking (default uncheck).
I can access pfsense.org and other websites now
-
Ah, that will do it. I should have pressed that question when I asked it earlier. Lesson for today.
Good result.
Steve
