Software or Hardware issue?

MyxomopeC

Greetings all,

Lately I have very strange issue.
My pfSense have several networks ( 1 for my home LAN ).

At that particular network I`m not able to access reddit or stackoverflow for example. And before you say you have DNS issue ... I have ping to the site (

ping www.reddit.com

Pinging reddit.map.fastly.net [151.101.129.140] with 32 bytes of data:
Reply from 151.101.129.140: bytes=32 time=2ms TTL=59
Reply from 151.101.129.140: bytes=32 time=2ms TTL=59
Reply from 151.101.129.140: bytes=32 time=2ms TTL=59
Reply from 151.101.129.140: bytes=32 time=2ms TTL=59

)
You will say 443 is closed ... Nope:

telnet www.redit.com 443
Trying 192.254.236.136...
Connected to www.redit.com.
Escape character is '^]'.

tcp dump shows that I got acknowledged by the endpoint (reddit in this example).
No Firewall rules forbidding traffic for my home network.

Currently I`m using version.2.6.0 ( with RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller)

Another thing. When I test connectivity with "curl -iL https://redd......." I`m getting response from other networks. They are on a different lan port. Only my home network is using the realtek card.

Please help me with some suggestions?

Regards!!!

stephenw10

You see any errors logged from the Realtek NIC?

Anything blocked in the firewall logs?

Jarhead

@myxomopec Just pointing out, your telnet went to the wrong site. You missed a d.

MyxomopeC

@jarhead Ups, My bad :)

 telnet reddit.com 443
Trying 151.101.65.140...
Connected to reddit.com.
Escape character is '^]'.

MyxomopeC

@stephenw10 Hello :)
Here is the relevant log from Firewall:

 	Mar 2 08:51:25 	HOMELAN 	USER_RULE (1613119901) 	192.168.2.247:64955		151.101.65.140:443		TCP:S
	Mar 2 08:51:24 	HOMELAN 	USER_RULE (1613119901) 	192.168.2.247:64953		151.101.65.140:443		TCP:S

Could you please tell me where I can find the NIC log ?
Regards!

Gertjan

@myxomopec said in Software or Hardware issue?:

At that particular network I`m not able to access reddit

What is the pfSense IP of that network ?

What are the IP settings of the device that is using that LAN network : run ipconfig or comparable :

ipconfig /all

What is the pfSense WAN IP ?
If it is not a RFC1918, you can hide one or two numbers.

MyxomopeC

@gertjan Greetings,

Here is the output of ipconfig from my machine (where I`m trying to access reddit(example))

Windows IP Configuration


Unknown adapter OpenVPN Wintun:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter Ethernet:

   Connection-specific DNS Suffix  . : homenet.pri
   IPv4 Address. . . . . . . . . . . : 192.168.2.247
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 192.168.2.1

Unknown adapter Local Area Connection:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter Wi-Fi:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : bundlearrows.com

Wireless LAN adapter Local Area Connection* 1:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter Local Area Connection* 10:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter Bluetooth Network Connection:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Here is the output from ifconfig from pfSense:

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=81009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER>
	ether 00:15:17:cb:21:88
	inet6 fe80::215:17ff:fecb:2188%em0 prefixlen 64 scopeid 0x1
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: em1
	options=81009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER>
	ether 00:15:17:cb:21:89
	inet6 fe80::215:17ff:fecb:2189%em1 prefixlen 64 scopeid 0x2
	inet 192.168.1.5 netmask 0xffffff00 broadcast 192.168.1.255
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=81209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER>
	ether 18:03:73:d0:5c:b2
	inet6 fe80::1a03:73ff:fed0:5cb2%em2 prefixlen 64 scopeid 0x3
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1480
	description: HOMELAN
	options=201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
	ether e0:8f:ec:00:39:2d
	inet6 fe80::e28f:ecff:fe00:392d%re0 prefixlen 64 scopeid 0x4
	inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=41<UP,RUNNING> metric 0 mtu 1536
	groups: enc
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
	inet 127.0.0.1 netmask 0xff000000
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pfsync0: flags=0<> metric 0 mtu 1500
	groups: pfsync
pflog0: flags=100<PROMISC> metric 0 mtu 33160
	groups: pflog
em1.1000: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: IIVLAN
	options=3<RXCSUM,TXCSUM>
	ether 00:15:17:cb:21:89
	inet6 fe80::215:17ff:fecb:2189%em1.1000 prefixlen 64 scopeid 0x9
	inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
	groups: vlan
	vlan: 1000 vlanpcp: 0 parent interface: em1
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em0.1001: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: ServerVLAN
	options=3<RXCSUM,TXCSUM>
	ether 00:15:17:cb:21:88
	inet6 fe80::215:17ff:fecb:2188%em0.1001 prefixlen 64 scopeid 0xa
	inet 10.1.0.1 netmask 0xffff0000 broadcast 10.1.255.255
	groups: vlan
	vlan: 1001 vlanpcp: 0 parent interface: em0
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em1.1002: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: ProxmoxClusterVLAN
	options=3<RXCSUM,TXCSUM>
	ether 00:15:17:cb:21:89
	inet6 fe80::215:17ff:fecb:2189%em1.1002 prefixlen 64 scopeid 0xb
	inet 10.0.10.1 netmask 0xffffff00 broadcast 10.0.10.255
	groups: vlan
	vlan: 1002 vlanpcp: 0 parent interface: em1
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em0.1003: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: LVLAN
	options=3<RXCSUM,TXCSUM>
	ether 00:15:17:cb:21:88
	inet6 fe80::215:17ff:fecb:2188%em0.1003 prefixlen 64 scopeid 0xc
	inet 10.2.0.1 netmask 0xffffff00 broadcast 10.2.0.255
	groups: vlan
	vlan: 1003 vlanpcp: 0 parent interface: em0
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em0.1004: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: BA
	options=3<RXCSUM,TXCSUM>
	ether 00:15:17:cb:21:88
	inet6 fe80::215:17ff:fecb:2188%em0.1004 prefixlen 64 scopeid 0xd
	inet 10.2.1.1 netmask 0xffffff00 broadcast 10.2.1.255
	groups: vlan
	vlan: 1004 vlanpcp: 0 parent interface: em0
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em0.1005: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: DDHTLXX
	options=3<RXCSUM,TXCSUM>
	ether 00:15:17:cb:21:88
	inet6 fe80::215:17ff:fecb:2188%em0.1005 prefixlen 64 scopeid 0xe
	inet 10.2.2.1 netmask 0xffffff00 broadcast 10.2.2.255
	groups: vlan
	vlan: 1005 vlanpcp: 0 parent interface: em0
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em0.1006: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: DDHTLXVI
	options=3<RXCSUM,TXCSUM>
	ether 00:15:17:cb:21:88
	inet6 fe80::215:17ff:fecb:2188%em0.1006 prefixlen 64 scopeid 0xf
	inet 10.2.3.1 netmask 0xffffff00 broadcast 10.2.3.255
	groups: vlan
	vlan: 1006 vlanpcp: 0 parent interface: em0
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pppoe0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
	description: WAN
	inet 195.yyy.yyy.yyy --> 195.xxx.xxx.xxx netmask 0xffffffff
	inet6 fe80::215:17ff:fecb:2188%pppoe0 prefixlen 64 scopeid 0x10
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ovpns1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
	options=80000<LINKSTATE>
	inet6 fe80::215:17ff:fecb:2188%ovpns1 prefixlen 64 scopeid 0x11
	inet 10.4.0.1 --> 10.4.0.2 netmask 0xffffff00
	groups: tun openvpn
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	Opened by PID 14530
ovpns2: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
	options=80000<LINKSTATE>
	inet6 fe80::215:17ff:fecb:2188%ovpns2 prefixlen 64 scopeid 0x12
	inet 10.4.1.1 --> 10.4.1.2 netmask 0xffffff00
	groups: tun openvpn
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	Opened by PID 53796
ovpns3: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
	options=80000<LINKSTATE>
	inet6 fe80::215:17ff:fecb:2188%ovpns3 prefixlen 64 scopeid 0x13
	inet 10.4.2.1 --> 10.4.2.2 netmask 0xffffff00
	groups: tun openvpn
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	Opened by PID 89803
ovpns4: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
	options=80000<LINKSTATE>
	inet6 fe80::215:17ff:fecb:2188%ovpns4 prefixlen 64 scopeid 0x14
	inet 10.4.3.1 --> 10.4.3.2 netmask 0xffffffff
	groups: tun openvpn
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	Opened by PID 73903
ovpns5: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
	options=80000<LINKSTATE>
	inet6 fe80::215:17ff:fecb:2188%ovpns5 prefixlen 64 scopeid 0x15
	inet 10.4.4.1 --> 10.4.4.2 netmask 0xffffff00
	groups: tun openvpn
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	Opened by PID 16490
ovpns6: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
	options=80000<LINKSTATE>
	inet6 fe80::215:17ff:fecb:2188%ovpns6 prefixlen 64 scopeid 0x16
	inet 10.4.5.1 --> 10.4.5.2 netmask 0xffffff00
	groups: tun openvpn
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	Opened by PID 46386

heper

@myxomopec you are using pppoe on wan - so it might be an mss issue.

you could try setting mss clamping on your wan to 1492 (or lower)

MyxomopeC

@heper I`ve tried but that does not change the behavior.
Also, from other networks I can still access reddit.

stephenw10

What's the user rule that's passing that traffic? Also I assume those are pass logs? The default pass rule would not normally log.

If it was a packet size issue you would still see the initial TCP handshake succeed. Those packets are tiny.

MyxomopeC

@stephenw10 Yes, We`ve dump the tcp traffic and we can see that we have response from the target IP.

I hope this rules you are referring to:

stephenw10

Ok so you do see a TCP handshake complete in a pcap?

If the first packet that fails it a large packet this probably is an MTU issue.

You can see re0 has a lower MTU than any other NIC. I assume you set that?

MyxomopeC

@stephenw10 Only my home network has MTU of 1480 ... others are by default which should be 1500 I think.

stephenw10

Exactly. I assume because you set that? And that's the subnet that cannot connect to those sites.

MyxomopeC

@stephenw10 OK ... So why now? This was working for year and something.

Here is info from dump when I try to curl reddit with 5 sec timeout:

stephenw10

Something else in the route changed probably and the PMTU discovery is failing.

Set the re0 NIC back to MTU 1500. Test that, it should be idebntical to the other interfaces at that point.
It that still fails set the MSS on that NIC to something lower, like 1452.

Steve