Routing between interfaces/VLANs
-
19:10:17.589300 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 570 19:10:17.589730 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 106 19:10:17.603093 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 0 19:10:17.605634 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 1440 19:10:17.605653 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 1183 19:10:17.608889 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 0 19:10:17.645570 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 620 19:10:17.645913 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 106 19:10:17.659340 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 0 19:10:17.661371 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 1097 19:10:17.665183 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 0 19:10:17.686632 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 620 19:10:17.687001 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 106 19:10:17.700343 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 0 19:10:17.702104 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 1097 19:10:17.705444 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 0 19:10:17.726415 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 620 19:10:17.726785 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 106 19:10:17.740321 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 0 19:10:17.741835 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 474192.168.150.36 is a Macbook on GASTLAN while trying to ping 192.168.3.1. But all you can see there is traffic to the internet caused by background processes, no ICMP
-
Set your sniff to icmp to only the the IP your pinging... See my example..
What is prob your issue can tell you right now is your dest box host firewall..
-
@johnpoz said in Routing between interfaces/VLANs:
Set your sniff to icmp to only the the IP your pinging... See my example..
Empty result.
I'm pinging the pfsense on TRUSTED (192.168.3.1). When the Macbook does that from within TRUSTED the pfsense replies, so how can it be that firewall blocking?
-
Dude if your not seeing anything on the sniff of pfsense for 192.168.3.1 - then its NOT going to pfsense... Maybe you have your masks wrong and your using 192.168/16 ??
-
sorry my fault, when I did the test couple of minutes ago I did not have the any-to-any rules set
with them I see these on packet sniff:
20:22:59.622362 ARP, Request who-has 192.168.3.1 tell 192.168.150.36, length 42 20:23:02.814369 ARP, Request who-has 192.168.3.1 tell 192.168.150.36, length 42however, still no reply to the ping
-
that is a arp. and that tells me you have mask wrong... Since a 192.168.150/24 address would never arp for a 192.168.3.x address.
Firewall rules have ZERO to do with a packet capture.. You could have zero rules, you could have all deny.. You would still see traffic hitting the interface via sniff.
And would explain why not working... Since if 192.168.150.x thinks 192.168.3.x thinks they are the same network, it would never send traffic to pfsense to get "routed" to the other network.
Your mask is wrong on the 192.168.150.36 machine. You prob have /16 (255.255.0.0) vs /24 (255.255.255.0)
-
I see. However, where else to configure if not here?
-
On your actual client.. 192.168.150.36
Did you set it static? Is it windows? you can view what the mask is with an ipconfig
$ ipconfig Windows IP Configuration Ethernet adapter Local: Connection-specific DNS Suffix . : IPv4 Address. . . . . . . . . . . : 192.168.9.100 Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : 192.168.9.253 -
Same on windows and mac:
-
Well there is something broken then on it... Because you do not ARP for an IP that is not on your network... You don't
So 192.168.150.36/24 would NOT arp for a 192.168.3.1 address just not how it works..
So why are you seeing arps for that address, vs sending traffic to pfsense at 192.168.150.1 to route it for you.
Your saying the windows machine is doing the same thing? 192.168.150.39
do the same test you did using the windows machine .39, try and ping 192.168.3.1 (pfsense interface on the lan).. Do still see these arps for that 3.1 address from 150.39 address?
A device on network A, does not "arp" for addresses on network B... Can not work, does not work - there is no point in ever doing it.. A device will only arp for an IP that is on its network. So if your seeing arps from a device on 192.168.150/24 for a 192.168.3.x address - there is something broken! That device does not have another interface does it, a wireless one on the 192.168.3 network??
This is how it works..
device with
192.168.150.36/24
gw 192.168.150.1Wants to talk to 192.168.3.1
if do not have the mac address of its gateway 192.168.150.1
It would arp for 192.168.150.1, once it gets the mac address of 150.1
Send traffic to that mac, dest for 192.168.3.1There is never a scenario where it would directly arp for 192.168.3.1 since that is NOT on its own network.
-
maybe the logical architecture of the network has something to do with it?
Here's how this setup looks like:
pfsense > VLAN 3 taggged connected to switch > VLAN 3 tagged to Wifi Access Points > Access Points broadcasts the VLAN 3 thru it's SSID as untagged.
For the Windows-Machine: thats a VM on ESXi, where ESXi passes the VLAN 3 as an untagged network to the VM
Everything is Level 2 Switching, so no routing involved by the switches
All clients are DHCP, nothing static. The DHCP is running on pfsense. So if anything is misconfigured, the source must be in pfsense
-
@techvic said in Routing between interfaces/VLANs:
maybe the logical architecture of the network has something to do with it?
No doesn't matter... Comes down to how tcp works at a basic level... The client would never arp for an IP outside its network... Because that is not how it works..
A client would never on its own arp for that IP, since with the settings on its interface that IP is not on its network.. So there would be no way it could ever see the arp... Arps are at Layer 2.. Could you manually send out such an arp - sure.. But the OS should never do such a thing on its own.
If your seeing arps for that 3.1 address from 150.36/24 then it thinks for whatever reason that 3.1 is part of its local network.
If devices arped for IPs that are not on its network - every single IP you talk to on the internet - the client would arp for...
You have something wrong for sure with that client to why you can not send traffic to its gateway 150.1.. Could it be something odd going on with esxi - ok maybe.. Not sure on the details of your setup... But in all the years working with esxi, no I can not think of reason such a thing would happen. Nor have I seen it... it makes no sense for anything to arp for an IP outside its network..
But what would explain it completely is that client thinks 192.168.3.1 is on the same network as 192.168.150 - so mask of /16 would do that..
Reset the networking on that client doing the arps...
And again - are you seeing the same sort of problem from your 150.39 device?
The DHCP is running on pfsense. So if anything is misconfigured, the source must be in pfsense
Well that makes no sense, since your pfsense shows mask of /24... Client "shows" a mask of 24... So to validate - do a simple sniff of yoru dhcp traffic... What is being handed to the client?
edit:
Do you have something on the network that might have the same duplicate IP sending those arps? Maybe it has wrong mask on it... What is your esxi vmkern IP? etc. etc..
edit2: Look at the actual details of the arp your seeing... Is the MAC of the arp from your your client?
Sniff on your windows machine.. While your pinging, that 192.168.3.1 address -- do you see it arp for the gateway IP? Do you see it actually send the pings?
wireshark is FREE, download it and install it on your client your having problems with... And again is your other machine having the same problem? You have given 2 machine 150.39 and .36 - are both of them doing the same exact thing??
-
@johnpoz said in Routing between interfaces/VLANs:
And again - are you seeing the same sort of problem from your 150.39 device?
no, indeed on the Windows VM (192.168.150.39) everything works as it should. So there I can ping 192.168.3.1 and can access other hosts on the 192.168.3.0/24 net.
So for testing I connected a physical windows machine to the Wifi on 192.168.150.0/24, where I see the same issue as on the Mac yesterday (nothing to see on the packet capture at all). The machine gets a DHCP lease (192.168.150.21) and has access to the internet, but can't reach any host on 192.168.3.0/24.
Then I connected this machine by cable to the switch to a port with VLAN 3 untagged and PVID 3. There everything works.
So the culprit must be somewhere around the wireless system (TP-Link EAP managed Wifi). How is that possible, that accessing Internet works normal but anything to another net is not passing the Wifi system? here's a packet capture of opening google.com on the client thru Wifi:
17:31:54.819122 IP 143.204.101.25.443 > 192.168.150.21.52711: tcp 411 17:31:54.824123 IP 188.172.219.132.5938 > 192.168.150.21.52647: tcp 24 17:31:54.853304 IP 192.168.150.21.52712 > 143.204.101.114.443: tcp 0 17:31:54.854656 IP 192.168.150.21.56643 > 192.168.150.1.53: UDP, length 46 17:31:54.854834 IP 192.168.150.1.53 > 192.168.150.21.56643: UDP, length 110 17:31:54.861782 IP 192.168.150.21.57307 > 192.168.150.1.53: UDP, length 46 17:31:54.876373 IP 143.204.101.114.443 > 192.168.150.21.52712: tcp 0 17:31:54.879854 IP 192.168.150.21.52712 > 143.204.101.114.443: tcp 0 17:31:54.879907 IP 192.168.150.1.53 > 192.168.150.21.57307: UDP, length 127Sure I'm aware this is nothing related to pfSense anymore, but any ideas would be appreciated.
-
@techvic said in Routing between interfaces/VLANs:
that accessing Internet works normal but anything to another net is not passing the Wifi system? here's a packet capture of opening google.com on the client thru Wifi:
Because!! 143.204.25.443 is NOT on a 192.168/16 network - so it would actually send this traffic to the gateway..
When you say wireless.. You sure client is actually getting lease from pfsense? And not your wireless doing nat with 192.168.150 on both sides?
So sniff on pfsense and ping pfsense IP from this client 192.168.150.1 Do you see this traffic in your packet capture? In the sniff check the mac address, validate its your actual clients mac address, and not the mac address of your wireless device.
-
I found the culprit: because this GASTLAN is a net for guests, there on the Wifi-settings were "client isolation" enabled. Due to that, it blocked any traffic not bound to the internet. I even wasn't able to ping the gateway 192.168.150.1
Thank you very much for your patience and not giving up to find the solution till the end. Thats very rare nowadays
-
But WTF was arping 3.1??? That makes no sense at all.
Not sure how your doing your wifi - but why would it be set to a guest, when its a specific vlan already.. Which you control at pfsense..
-
@johnpoz said in Routing between interfaces/VLANs:
Not sure how your doing your wifi - but why would it be set to a guest, when its a specific vlan already.. Which you control at pfsense..
thats why I disabled that feature now > not needed.
@johnpoz said in Routing between interfaces/VLANs:
But WTF was arping 3.1??? That makes no sense at all.
It's TP-Link, China quality, wouldn't surprise me if it's just a bug (wouldn't be the first one). Or is it a hidden spy feature?
-
What IP is on the tplink device - maybe its mask is wrong?
-
@johnpoz said in Routing between interfaces/VLANs:
What IP is on the tplink device - maybe its mask is wrong?
Or the VLAN isolation isn't throrough and leaking through. I thought to remember sth like that from the smallest TP-Link Switches a few years ago...
-
@JeGr said in Routing between interfaces/VLANs:
Or the VLAN isolation isn't throrough and leaking through. I thought to remember sth like that from the smallest TP-Link Switches a few years ago...
And also TP-Link access points. I have one here and can't configure a 2nd SSID because of leaks between VLAN and native.
