Routing between VLANs not working on SG2100
-
This problem is almost certainly related to the fact that I'm using a Netgate SG2100 in one of my locations, and it has one of those integrated Marvell switches. The integrated Marvell switch massively complicates running a network with VLANs; I have an identical setup in other locations where my Netgate devices don't have an integrated switch and everything works perfectly.
The problem is that hosts in the Main vlan can't communicate with hosts in the IoT vlan. Firewall rules allow this and the log records the rules allowing the traffic. More generally, traffic trying to move from one vlan to another simply does not work, even when firewall rules allow it, though my specific use case here is getting traffic to route from the main lan to the IoT lan.
Here's the setup
I have four vlans running. The default vlan (management vlan / untagged / default), vlan 2 (Main), vlan 11 (IoT devices), vlan 50 (guest wifi). I have an interface assigned for each vlan. I am using 802.1q tagging. The default vlan is set to untagged for every switch port on the Marvell switch, including the parent / uplink. Each other vlan is set to tagged on every switch port, including the parent / uplink. I'm including a screen cap:
In Interface -> Assignments -> VLANs I have vlans 2, 11, and 50 configured on mvneta1. I have enabled interfaces in Interface -> Assignments -> Interface Assignments, one for each vlan:
I have confirmed that firewall rules are not blocking my traffic. This works more or less as long as hosts in one vlan don't need to talk to hosts in another. I am also noticing, as a side effect of whatever is wrong here, that hosts in the main lan can't even ping the interface of the vlan they are a member of (firewall also allow this).
Something clearly is wrong, and after reviewing the Netgate documentation for the models with a Marvell switch built in, I am not seeing what the issue is.
-
@bp81 What does the Firewall->Rules interface tabs for each VLAN interface say?
-
@bp81 Why would you tag all vlans on all ports?
Are you connecting each port to a separate switch or the same?
Are those switchports tagged properly? -
@rcoleman-netgate said in Routing between VLANs not working on SG2100:
@bp81 What does the Firewall->Rules interface tabs for each VLAN interface say?
IoT (vlan 11) rules:
The alias 'PrivateIPv4Subnets' contains all Class A, B, C and private IP addresses. These rules block IoT network hosts from initiating connections to hosts in any other vlan but still allows outbound connections to internet destinations
MAIN (vlan 2) rules:
Outbound connections to the guest network are blocked, but other connections are allowed.
-
@jarhead said in Routing between VLANs not working on SG2100:
@bp81 Why would you tag all vlans on all ports?
Are you connecting each port to a separate switch or the same?
Are those switchports tagged properly?I am essentially not using the integrated switch as a switch. I have a Ubiquiti Unifi switch downstream from the integrated switch. Essentially I have port 1 on the integrated switch attached to Port 8 on my Ubiquiti switch. On the Ubiquiti switch, Port 8 is a trunk port, passing all traffic with vlan tags intact. I don't really NEED the switch on the pfsense box, a single LAN interface would've sufficed. I don't want the switchport doing anything to the traffic's tags, it needs to accept traffic from any vlan and pass it to the uplink into the router for routing as needed.
-
@bp81 So you're blocking IoT from private addresses, then you're allowing to anything except private networks.
See the problem?Why not set the rules to allow any between the 2 networks and see if it works?
Whenever someone says "Ubiquiti switch" I always go right there as the cause of any networking issues and it's usually the case. Take pfSense out of the equation by allowing all traffic, then go from there. -
@bp81 said in Routing between VLANs not working on SG2100:
These rules block IoT network hosts from initiating connections to hosts in any other vlan but still allows outbound connections to internet destinations
Note you haven't allowed from IoT to the pfSense port 53 (TCP+UDP) for DNS. (assuming the pfSense LAN is not a public IP)
-
@jarhead said in Routing between VLANs not working on SG2100:
@bp81 So you're blocking IoT from private addresses, then you're allowing to anything except private networks.
See the problem?Why not set the rules to allow any between the 2 networks and see if it works?
Whenever someone says "Ubiquiti switch" I always go right there as the cause of any networking issues and it's usually the case. Take pfSense out of the equation by allowing all traffic, then go from there.That's not how that has worked elsewhere. The firewall rules on IoT disallow connections outbound from hosts inside the IoT vlan to other vlans. That rule does not prevent connections from other vlans to hosts in IoT. I'm running this exact configuration in 5 other locations and it works perfectly.
I will do the test as you suggested just to confirm, but this is not the issue.
-
@steveits said in Routing between VLANs not working on SG2100:
@bp81 said in Routing between VLANs not working on SG2100:
These rules block IoT network hosts from initiating connections to hosts in any other vlan but still allows outbound connections to internet destinations
Note you haven't allowed from IoT to the pfSense port 53 (TCP+UDP) for DNS. (assuming the pfSense LAN is not a public IP)
The DHCP server for the IoT vlan is giving out a public DNS server for clients in that network. Opening port 53 for clients in the IoT vlan is not necessary, as those devices do not require internal dns resolution services and aren't setup to use it anyway.
-
@bp81 said in Routing between VLANs not working on SG2100:
@jarhead said in Routing between VLANs not working on SG2100:
@bp81 So you're blocking IoT from private addresses, then you're allowing to anything except private networks.
See the problem?Why not set the rules to allow any between the 2 networks and see if it works?
Whenever someone says "Ubiquiti switch" I always go right there as the cause of any networking issues and it's usually the case. Take pfSense out of the equation by allowing all traffic, then go from there.That's not how that has worked elsewhere. The firewall rules on IoT disallow connections outbound from hosts inside the IoT vlan to other vlans. That rule does not prevent connections from other vlans to hosts in IoT. I'm running this exact configuration in 5 other locations and it works perfectly.
I will do the test as you suggested just to confirm, but this is not the issue.
You missed the point. Both rules do exactly the same thing and are unnecessary. You can make the destination on the allow an ANY since privates are already blocked.
Plus, it'll help with the DNS problem you're gonna have since the IoT gateway is a private address also. -
@jarhead said in Routing between VLANs not working on SG2100:
@bp81 said in Routing between VLANs not working on SG2100:
@jarhead said in Routing between VLANs not working on SG2100:
@bp81 So you're blocking IoT from private addresses, then you're allowing to anything except private networks.
See the problem?Why not set the rules to allow any between the 2 networks and see if it works?
Whenever someone says "Ubiquiti switch" I always go right there as the cause of any networking issues and it's usually the case. Take pfSense out of the equation by allowing all traffic, then go from there.That's not how that has worked elsewhere. The firewall rules on IoT disallow connections outbound from hosts inside the IoT vlan to other vlans. That rule does not prevent connections from other vlans to hosts in IoT. I'm running this exact configuration in 5 other locations and it works perfectly.
I will do the test as you suggested just to confirm, but this is not the issue.
You missed the point. Both rules do exactly the same thing and are unnecessary. You can make the destination on the allow an ANY since privates are already blocked.
Plus, it'll help with the DNS problem you're gonna have since the IoT gateway is a private address also.I temporarily set all interfaces (except WAN) to allow all traffic. No change.
-
@bp81
So then the problem is the ubiquiti.
Show pics of the config.