Data Between VLANs being Blocked
-
Hi Guys,
I am running PFSense 2.2.1. Have been using the product for years and love it!
I have a handful of VLANs setup and they are all working well. Issue I have noticed is, when one device tries to access another device in a different VLAN (Eg: Windows File Share), it can connect but it is very slow, and eventually the connection dies and drops out)
I have noticed in the Firewall Log that packets are being blocked. I have allowed all between all VLANs so they should be able to communicate freely without blocks.
See attachment for what is in the logs.
The log shows my desktop PC trying to access a file share on a NAS in another VLAN.
I have also noticed when trying to use SSH to another device in another VLAN, it will connect but then eventually drop out.
How can I ensure that all ports and protocols are allowed between all VLANs? I have created a Floating Rule as well with an Alias off all VLANs to Allow All between them, but the problem still exists.
Cheers
-
Enable UDP as well so it gets its DNS entries and NLA.
-
Cheers.
I have just edited the Floating Rule to allow all protocols. Didn't seem to make and difference.
-
Yeah, if you want it wide open, why not protocol any?
Also, if you don't want to firewall the VLANs but just segment them, a Layer 3 switch might be a more appropriate solution.
-
Yeah I have a Cisco 3560G doing the switching. I can define IP's to the VLAN interfaces there and do it that way. I just moved all of the inter-VLAN routing to PFSense (from the switch), then noticed this issue despite there being allow-all rules between the VLANs.
-
Those are also not SYN packets, but out-of-state traffic. Meaning there is some trailing traffic after pfSense closed the state for whatever reason. I don't know if Windows sends any keepalives or anything to keep firewalls from closing share states.
There might be some tuning that could be done to alleviate it but I wouldn't know what it is. Firewalling windows file shares has never been my strong suit. I switch it.
-
Thanks for you insights, appreciate your feedback.
I moved the inter-VLAN routing to PFSense to avoid having to do Policy Based Routing and ACL's in IOS on the Cisco.
If I moved the inter-VLAN routing back to the Cisco, PFSense will then only be acting as the Internet Gateway for the VLANs, via the default route on the switch. (ip route 0.0.0.0 0.0.0.0 PFSense-IP-Address)
I have tried disabling the Windows Firewall (Windows 7 Professional), but the issue still seems to occur. :(
-
This should "just work" on pfSense. The symptoms you describe sound just like "asymmetric routing". pfSense is only seeing (some?) traffic in one direction, and the state that was created on the initial start of a connection gets timed out after 20-30 seconds and further traffic is dropped - resulting in shared file access, ssh, whatever stalling.
Perhaps there is also some layer 3 routing going on in the Cisco, and some device/s are using the Cisco as their gateway, or know a route to the other VLAN via the Cisco or some other way that some traffic can avoid going both directions through pfSense?
-
Yeah you have something else going on.. phil.davis is prob right on the money with some sort of asymmetric routing issue - I run multiple segments and vlans off my pfsense and have no issues routing or firewalling traffic between segments. If your seeing blocks on non syn packets you prob have asymmetric route problems.
-
Thanks guys. I think you're right. I'm going to check all the routes on the Cisco to see if there is something there which is wrong. I suspect their might be. Issue only happens for devices which are a member of VLAN120 which narrows it down a bit.
I can't see the need to have any routes on the L3 switch at all as PFSense is doing it all. There is a default route which is pointed at PFSense purely to give internet access to the switch.
I do remember adding some routes when I was mucking around with some stuff a few months ago on the Cisco, so it could be that.
Will check it out tonight.
Cheers!
-
Checked the Cisco switch. The only route added is the default route, which is 0.0.0.0 0.0.0.0 10.2.20.2
Which basically gives the switch Internet access via an IP on one of the VLAN interfaces within PFSense. When I had all the devices using the Cisco VLAN interfaces as their default gateway (switch acting as the inter-vlan router), they would all route out to PFSense via that route, and thus the Internet, which basically put all of the firewalling responsibilities etc, onto the switch as opposed to PFSense.
That was why I moved the whole lot to PFSense and basically just left the switchports a member of their respective VLANs and changed the default gateway to that of the VLAN interfaces on PFSense. PFSense then assumed the responsibility of the inter-vlan routing as well as the Internet g/w.
Routes from Cisco:
Gateway of last resort is 10.2.20.2 to network 0.0.0.0
10.0.0.0/24 is subnetted, 8 subnets
C 10.2.20.0 is directly connected, Vlan20
C 10.2.40.0 is directly connected, Vlan40
C 10.2.60.0 is directly connected, Vlan60
C 10.2.80.0 is directly connected, Vlan80
C 10.2.100.0 is directly connected, Vlan100
C 10.2.120.0 is directly connected, Vlan120
C 10.2.140.0 is directly connected, Vlan140
C 10.254.1.0 is directly connected, Vlan999
S* 0.0.0.0/0 [1/0] via 10.2.20.2Anyway, will keep digging. :)
Cheers
-
So you only have 1 floating rule.. You have not rules on the specific vlan interfaces?
So what pfsense IP .2 in all of those segments? And all clients default gateway is the .2 address in their specific network segment..
-
So you only have 1 floating rule.. You have not rules on the specific vlan interfaces?
So what pfsense IP .2 in all of those segments? And all clients default gateway is the .2 address in their specific network segment..
The .2 is purely so I can enable DHCP scopes on each VLAN and use PFSense to act as the DHCP server for all of the VLAN (scopes). It also can be used as the default gateway on clients when PFSense is doing the inter-vlan routing obviously. When the Cisco was doing the inter-vlan routing, the VLAN IP interfaces on the PFSense side don't get used for anything other than allowing DHCP to be enabled to serve IP's.
The DHCP server will not operate unless there is an IP address assigned.
Only the 1 floating rule, yes - to see if that helped to alleviate the issue which i was experiencing.
-
So what is your gateway?? If you removing the IP off the vlan interface in pfsense – how is suppose to route between your segments? Thought you said pfsense was doing all the routing.. Or do you have your cisco svi as gateway for any of your devices?
-
Sorry for the confusion.
When I have devices using the VLAN interfaces on PFSense as their default gateway, the issue I have described occurs.
When I have the devices using the VLAN interfaces on the Cisco switch as their default gateway, there is no issue at all.
It doesn't matter (?) that both the Cisco and PFsense have IP's on their VLAN interfaces. The Cisco switch does't need to have them if PFSense is doing the inter-vlan routing, but it doesn't hurt if they are there either. So I just left them.
PFSense needs to have the IP's on the VLAN Interfaces for:
a) So it can route between vlans (as a devices default gateway), if it is doing inter-vlan routing.
b) So DHCP can be enabled for the VLANs. DHCP will not be allowed to be active per-VLAN if there is no IP set.For now, I have moved inter-VLAN routing back to the Cisco switch to avoid this problem, or until I can figure out why PFSense is behaving that way. As such, all my devices now have their default gateway set to the IP address of their respective VLAN interface IP address on the switch. PFSense is now just doing DHCP and acting as an Internet gateway. The VLANs are getting to the Internet (via PFSense) through the default route on the Cisco.
There are no static routes, or anything like that within PFSense. I just have an ADSL PPPoE connection on the WAN interface of PFSense. The LAN is the trunk port to the Cisco which is carrying the VLANs.
Also, all of my VLANs on PFSense had the below rules added in addition to the Floating Rule.
-
Diagram your network. www.gliffy.com is always available. See below for the information needed.
-
This is how it is currently wired up.
Note I have changed the default gateway of the devices back to the SVI's on the Cisco L3 instead of the PFSense due to the original issue I have described.
Cheers
-
It looks to me like you didn't change the default gateway of the hosts on VLAN120 from 10.2.120.1 to 10.2.120.2.
