Routing between interfaces/VLANs
-
I found several threads with similar issues, however none of the solutions worked for me.
I have the following setup:
Interface TRUSTED (renamed from the default LAN) with 192.168.3.0/24 network, no VLAN
Interface GASTLAN with 192.168.150.0/24 network, VLAN 3
Interface OpenVPN (auto-created by enabling OpenVPN Servers)From all 3 of them access to the internet works. From OpenVPN Clients I can access hosts on TRUSTED (because the TRUSTED-net is defined as local network on the OpenVPN-Servers).
However, no comm is possible from GASTLAN to a host in TRUSTED.
For troubleshooting I created any-to-any rules on all 3 of these zones. I tried to find the culprit of my problem using the packet-filter, however, listening on GASTLAN-interface for any traffic to a specified IP (host on TRUSTED) brings an empty result.
Heres my rules:
Can anybody point me in the right direction where I have my thinking-issue?
-
Block private networks unchecked on TRUSTET interface?
-
yes
-
logs?
-
which/where to find?
-
@techvic said in Routing between interfaces/VLANs:
listening on GASTLAN-interface for any traffic to a specified IP (host on TRUSTED) brings an empty result.
If you see ZERO traffic to an IP in the trusted network hit your gastlan interface - how would pfsense route it if nothing there..
your client in gastlan not using pfsense as its gateway? Is it running some vpn client on itself?
-
@johnpoz said in Routing between interfaces/VLANs:
your client in gastlan not using pfsense as its gateway? Is it running some vpn client on itself?
it is using pfsense as gateway (192.168.150.1). Since access to the internet works thru that, this shouldn't be the reason?
-
@techvic said in Routing between interfaces/VLANs:
it is using pfsense as gateway (192.168.150.1). Since access to the internet works thru that, this shouldn't be the reason?
Well then what you say is not possible, or the client is pointing to something else for route to your lan network..
If you sniff on this gast interface, pfsense packet capture under diag... And you don't see any traffic - how could pfsense ever possible route it??
Here... Sniffing on my dmz (192.168.3/24), pinging box on my lan.. 192.168.9.100, the firewall doesn't allow the traffic on the dmz... But you can still see the traffic hitting pfsense dmz interface.
-
19:10:17.589300 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 570 19:10:17.589730 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 106 19:10:17.603093 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 0 19:10:17.605634 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 1440 19:10:17.605653 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 1183 19:10:17.608889 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 0 19:10:17.645570 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 620 19:10:17.645913 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 106 19:10:17.659340 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 0 19:10:17.661371 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 1097 19:10:17.665183 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 0 19:10:17.686632 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 620 19:10:17.687001 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 106 19:10:17.700343 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 0 19:10:17.702104 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 1097 19:10:17.705444 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 0 19:10:17.726415 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 620 19:10:17.726785 IP 192.168.150.36.56489 > 17.248.146.9.443: tcp 106 19:10:17.740321 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 0 19:10:17.741835 IP 17.248.146.9.443 > 192.168.150.36.56489: tcp 474192.168.150.36 is a Macbook on GASTLAN while trying to ping 192.168.3.1. But all you can see there is traffic to the internet caused by background processes, no ICMP
-
Set your sniff to icmp to only the the IP your pinging... See my example..
What is prob your issue can tell you right now is your dest box host firewall..
-
@johnpoz said in Routing between interfaces/VLANs:
Set your sniff to icmp to only the the IP your pinging... See my example..
Empty result.
I'm pinging the pfsense on TRUSTED (192.168.3.1). When the Macbook does that from within TRUSTED the pfsense replies, so how can it be that firewall blocking?
-
Dude if your not seeing anything on the sniff of pfsense for 192.168.3.1 - then its NOT going to pfsense... Maybe you have your masks wrong and your using 192.168/16 ??
-
sorry my fault, when I did the test couple of minutes ago I did not have the any-to-any rules set
with them I see these on packet sniff:
20:22:59.622362 ARP, Request who-has 192.168.3.1 tell 192.168.150.36, length 42 20:23:02.814369 ARP, Request who-has 192.168.3.1 tell 192.168.150.36, length 42however, still no reply to the ping
-
that is a arp. and that tells me you have mask wrong... Since a 192.168.150/24 address would never arp for a 192.168.3.x address.
Firewall rules have ZERO to do with a packet capture.. You could have zero rules, you could have all deny.. You would still see traffic hitting the interface via sniff.
And would explain why not working... Since if 192.168.150.x thinks 192.168.3.x thinks they are the same network, it would never send traffic to pfsense to get "routed" to the other network.
Your mask is wrong on the 192.168.150.36 machine. You prob have /16 (255.255.0.0) vs /24 (255.255.255.0)
-
I see. However, where else to configure if not here?
-
On your actual client.. 192.168.150.36
Did you set it static? Is it windows? you can view what the mask is with an ipconfig
$ ipconfig Windows IP Configuration Ethernet adapter Local: Connection-specific DNS Suffix . : IPv4 Address. . . . . . . . . . . : 192.168.9.100 Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : 192.168.9.253 -
Same on windows and mac:
-
Well there is something broken then on it... Because you do not ARP for an IP that is not on your network... You don't
So 192.168.150.36/24 would NOT arp for a 192.168.3.1 address just not how it works..
So why are you seeing arps for that address, vs sending traffic to pfsense at 192.168.150.1 to route it for you.
Your saying the windows machine is doing the same thing? 192.168.150.39
do the same test you did using the windows machine .39, try and ping 192.168.3.1 (pfsense interface on the lan).. Do still see these arps for that 3.1 address from 150.39 address?
A device on network A, does not "arp" for addresses on network B... Can not work, does not work - there is no point in ever doing it.. A device will only arp for an IP that is on its network. So if your seeing arps from a device on 192.168.150/24 for a 192.168.3.x address - there is something broken! That device does not have another interface does it, a wireless one on the 192.168.3 network??
This is how it works..
device with
192.168.150.36/24
gw 192.168.150.1Wants to talk to 192.168.3.1
if do not have the mac address of its gateway 192.168.150.1
It would arp for 192.168.150.1, once it gets the mac address of 150.1
Send traffic to that mac, dest for 192.168.3.1There is never a scenario where it would directly arp for 192.168.3.1 since that is NOT on its own network.
-
maybe the logical architecture of the network has something to do with it?
Here's how this setup looks like:
pfsense > VLAN 3 taggged connected to switch > VLAN 3 tagged to Wifi Access Points > Access Points broadcasts the VLAN 3 thru it's SSID as untagged.
For the Windows-Machine: thats a VM on ESXi, where ESXi passes the VLAN 3 as an untagged network to the VM
Everything is Level 2 Switching, so no routing involved by the switches
All clients are DHCP, nothing static. The DHCP is running on pfsense. So if anything is misconfigured, the source must be in pfsense
-
@techvic said in Routing between interfaces/VLANs:
maybe the logical architecture of the network has something to do with it?
No doesn't matter... Comes down to how tcp works at a basic level... The client would never arp for an IP outside its network... Because that is not how it works..
A client would never on its own arp for that IP, since with the settings on its interface that IP is not on its network.. So there would be no way it could ever see the arp... Arps are at Layer 2.. Could you manually send out such an arp - sure.. But the OS should never do such a thing on its own.
If your seeing arps for that 3.1 address from 150.36/24 then it thinks for whatever reason that 3.1 is part of its local network.
If devices arped for IPs that are not on its network - every single IP you talk to on the internet - the client would arp for...
You have something wrong for sure with that client to why you can not send traffic to its gateway 150.1.. Could it be something odd going on with esxi - ok maybe.. Not sure on the details of your setup... But in all the years working with esxi, no I can not think of reason such a thing would happen. Nor have I seen it... it makes no sense for anything to arp for an IP outside its network..
But what would explain it completely is that client thinks 192.168.3.1 is on the same network as 192.168.150 - so mask of /16 would do that..
Reset the networking on that client doing the arps...
And again - are you seeing the same sort of problem from your 150.39 device?
The DHCP is running on pfsense. So if anything is misconfigured, the source must be in pfsense
Well that makes no sense, since your pfsense shows mask of /24... Client "shows" a mask of 24... So to validate - do a simple sniff of yoru dhcp traffic... What is being handed to the client?
edit:
Do you have something on the network that might have the same duplicate IP sending those arps? Maybe it has wrong mask on it... What is your esxi vmkern IP? etc. etc..
edit2: Look at the actual details of the arp your seeing... Is the MAC of the arp from your your client?
Sniff on your windows machine.. While your pinging, that 192.168.3.1 address -- do you see it arp for the gateway IP? Do you see it actually send the pings?
wireshark is FREE, download it and install it on your client your having problems with... And again is your other machine having the same problem? You have given 2 machine 150.39 and .36 - are both of them doing the same exact thing??
