Intervlan traffic being blocked
-
Server has an interface ip of172.16.64.1/21, vlan 48
S has an interface ip of 10.20.160.1/24, on vlan 16testpc [connected into port1 of S switch] -> S switch[connected into port 1 of server switch] -> Server switch -> pfsense
@planedrop i have disabled the firewall on the server at 172.16.64.200 and that didn't work. I know the server vlan is configured properly because it can reach other subnets without any issue. its just having trouble with the S subnet
@johnpoz apologies, i should have added more information. These are unifi switches, all you have to do is define the network and their vlans, set the port to the appropriate vlan and unfi does the rest.
I connected my test pc to port 1 on the S switch -> which has the Server switch as its upstream. The server isn't using another gateway
I have tried adding a rule on the server subnet
interface = server
source = S net
destination = Server netand
interface = Server
destination = S net
source = Server netand still nothing worked.
-
@johnpoz This sounds like it could be a VLAN issue on the switch level, do you have this VLAN allowed on the switch port that the Server is connected to?
-
@greatbush said in Intervlan traffic being blocked:
I have tried adding a rule on the server subnet
interface = server
source = S net
destination = Server netPointless and useless..
The rules on server interface have ZERO to do with S network devices talking to the Server network - like I said they could be blank and you could still talk.
Can your pc ping pfsense IP for its S interface? Can it ping the IP of pfsense IP on the Server network.
You would have to adjust your rules on the S network interface of pfsense to allow that.. Your current rule only allows for a source IP of 10.20.160.72 to talk to 172.16.64.200.. But you want to to validate your connectivity is correct.. Maybe you have rules below that allow? Which would be fine.
But you want to validate your connectivity to pfsense on your S network.. If you can ping the IP of pfsense on the Server network, but not things on the server network - then that screams firewall on the server device.
-
@planedrop i have it allowed as an access port on the switch
-
@greatbush And it's tagged on the other end of the switch going to the firewall?
And to be clear, you did not configure the VLAN on the server, right? Since the switch is handling the tagging for you.
-
@johnpoz The test pc can ping pfsense ip for the s interface. The test pc is unable to ping pfsense ip for the server network.
. -
@planedrop No there are no vlans on the server. And yes its tagged on the trunk port. I have dhcp enabled for this interface and i am able to pull an from the pool i specified.
-
@greatbush I think it's time to start packet captures.
From the rule you showed, that device on S should be able to ping across to Servers.
I would start a pcap on the S interface, then ping the server IP and see if you get ICMP packets to show up on the S interface.
If you do, then go to the Servers interface and do a pcap there and see if the ICMP packets show up there, this will tell you where the breakdown is. If they do show up, then it's not the firewall, if they don't then something is blocking them.
-
@planedrop i have done that. all i see are echo ping requests but no response hence why i am scratching my head. I know the rules on the server interface aren't meant to be there but i was hoping that would fix the issue which it didn't
-
@greatbush Did you do the pcaps on both interfaces from the pfSense side?
So you're seeing the echo requests, and you see them hit the S interface, and then also see them leave the Servers interface?
If that is the case, then your device on the Server interface isn't responding and the issue isn't pfSense specific.
-
@greatbush said in Intervlan traffic being blocked:
The test pc is unable to ping pfsense ip for the server network.
Well then something is wrong.. is your client not using pfsense as its gateway - or is it running vpn?
On your pcap you did on the S interface you see the ping request from your S client to pfsense server IP?
I would guess possible mask issue maybe on your client - but that sure can not be the case if your pc is a 10 network and your server network is 172.x etc..
Hmmm - this is turning out to be something interesting with the info so far. You have no rules in floating? I would guess maybe a vpn on the pc? Like to see the pcap to pfsense S interface.. Its being sent to the pfsense mac address of S interface with destination IP of pfsense server IP.
example - I can simulate what sounds like your saying is happening by pinging an IP on another network on pfsense that doesn't exist.
So pc on my lan 192.168.9.0/24 pinging 192.168.3.14 (doesn't exist) but pfsense has an network 192.168.3.0/24
Notice the mac address it is sending to is pfsense mac address of my my lan interface on pfsense (192.168.9.253)
See how the mac in my packet capture is pfsense lan mac address.
If you are seeing similar when you sniff on S interface of pfsense - I can only guess floating rule blocking the traffic. Or something not right in pfsense.. Or like me your pinging an IP that doesn't exist? You sure you are pinging pfsense Server interface IP - etc..
edit: just for complete example - here is my pinging the actual IP of pfsense interface on that network 192.168.3.253.. See how the capture shows it still sending it to the lan mac address of pfsense.
-
-
@greatbush Is it possible the subnet is configured wrong? Like, is the Servers interface actually setup with the right subnet and mask?
From what I am seeing, pfSense is getting the packet and allowing the connection, but if you don't see it on Servers, then for some reason pfSense isn't routing it to that interface.
Which means maybe the interface is setup wrong, or maybe pfSense's routing table has another entry from something like BGP?
Check your Route Table on pfSense: https://docs.netgate.com/pfsense/en/latest/monitoring/status/routes.html
-
@planedrop i am hundred percent sure it is
test pc
unifi
pfsense
-
What about the Servers interface though? The 172 network.
-
@planedrop For the route
-
@greatbush I think we are getting closer, if that is your entire routing table, then pfSense does not have a clue where to route the 172 network, so it's just going to send it to it's default route which is likely your gateway, unless I am misunderstanding something about your configuration.
Is the Servers interface properly configured? Not the S but the Servers one.
-
@greatbush The server network is fine. It can reach my other subnets without any issue. Hmm. I don't know if this is going to help. the 10.20.160.x was the network at another site and an ipsec tunnel was setup for that site.
I don't know if this might be causing issues. -
@planedrop Server network
-
@greatbush I don't think IPsec is the issue here, you would see the ICMP responses on the Servers interface if that was the case, they just wouldn't route back to the S interface.
Again though, it looks to me like pfSense doesn't have a routing table entry for the 172. network so those echo request are never going to get to a device on any network, it'll just send them through it's default gateway, which is presumably your internet provider.
Also, why are you configuring actual networks on the Unifi side? Those should be VLAN only networks since you're using a 3rd party gateway (pfSense) for them. I don't think that is what is breaking things here but was worth mentioning.
I did just see that you're using Broadcom NICs though, check out this just in case cuz it might affect pings: https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#packet-loss-with-many-small-udp-packets:~:text=and%20disable%20msix.-,Packet%20loss%20with%20many%20(small)%20UDP%20packets,-%C2%B6
