Proper FW rules for PVLAN

JT40

Hello,

a long time ago I posted here about my setup, which is basically the following:

AP (L2 isolation is working at this level with Ubiquity because I have multiple SSIDs, I suspect that it wouldn't work if I manage it from the switch, due to the VLAN tag involved. In any case, the PVLAN is defined, but only for the internal network of the AP management, the other VLANs that I use for my devices are simple VLANs isolated by the AP functionality)
L2+ switch (I don't use L3 capabilities)
Pfsense
Modem/router (in brigde mode, or whatever is the name, basically it's only routing for the moment. I recently enabled also the AP functionality on it, but it's the same behaviour)

I wonder what's the correct FW rule to create a PVLAN-like behaviour at FW level.
Let me explain, I currently have set up PVLAN in my switch, but it doesn't work because each VLAN has a FW rule (VLAN to everywhere) , therefore, my VLANs can communicate each other and up to internet.

The issue that I always faced was the following, if I don't allow this rule, my communication to internet doesn't work.
Basically, my issue was never solved but now I've got time to work on it and I need to solve it once and for all.

By myself I tried to set rules like:

VLAN to Modem IP
VLAN to Modem network (something like 192.168.1.1/24

In both cases, the communication doesn't work or it doesn't work all the time (very strange)... It's not a routing problem as you can understand, so let's focus on the FW rule...

Another weird case is with the current FW rules setup, for some reason, I'm able to ping from a desktop (VLAN100) to my phone behind the AP (VLAN 120), but the phone or laptop can't ping that machine... Other machines can ping it, so I don't understand why this happens...
There is no other FW rule involved, this is very strange.
Before I applied the latest changes to PVLANs though (before I didn't have PVLANs), the communication was working between all the devices, so in this scenario I suspect that the switch is not working properly, or eventually there was some problem that I never noticed.

I need to clarify one thing, the current PVLAN ID that is used for the AP port on the switch, it's only one, I can specify only one PVLAN ID per switch port.
That PVLAN ID is used by default from the AP, mainly for the administration of it I guess, then there are a couple of additional VLANs for each SSID which have L2 isolation, but it seems that they can still communicate because they pass through the router, not at switch level.
Shall I test if at switch level they would communicate?
I think so, but it would be another challenge, I'd need to detach the router and configure the L3 functionality on the switch... I hope I don't need to test it :D

In order to recap, I need a FW rule for the following scenarios:

VLAN to modem/router
VLAN to modem/router ONLY to get private VLANs (probably an alias rule would work better so I can add all the VLANs in one shoot)
If it's not too much, I'd need some example about a specific FW rule, I need to use it for Time machine on a NAS and other services... So, only specific VLANs should be allowed to use the NAS (which would be on a dedicated VLAN), or specific devices but I don't think that a Firewall can filter out devices, maybe by hostname yes.

Thanks for any help.

johnpoz

@jt40 if you want help with rules - please post a screenshot of your rules on the interface be it native or vlan, etc.

Rules would be placed on where the traffic enters pfsense from the network pfsense attached too. Top down, first rule to trigger wins no other rules evaluated.

If you want a vlan on pfsense to not go some where, then rule would be above a rule that allows all traffic, etc.

Do you have any rules on floating?

JT40

This post is deleted!

JT40

@johnpoz @johnpoz said in Proper FW rules for PVLAN:

@jt40 if you want help with rules - please post a screenshot of your rules on the interface be it native or vlan, etc.

Rules would be placed on where the traffic enters pfsense from the network pfsense attached too. Top down, first rule to trigger wins no other rules evaluated.

If you want a vlan on pfsense to not go some where, then rule would be above a rule that allows all traffic, etc.

Do you have any rules on floating?

Not handy to take a screenshot in my situation, but the only FW rule that I have for each VLAN is that "the traffic from each VLAN network can go everywhere".
So, LAN and Internet, really everywhere, basically VLAN TO ANY.

But I have another rule to block all the traffic in ingress to the router (WAN interface), I can only go outside internet from each VLAN based on what I see.

Why did I create such rule? (which is the only blocking rule at the moment)
Because the Router is posed under the same modem/router network, another network would have been blocked for reasons I still don't understand, it could be also a limitation.
Is this enough to create such rule?
On top of my mind, no, but I wanted to be safe.

Now, I still need to block the traffic from VLANs to VLANs.
So you say that I need to create a rule for each VLAN or a floating rule, and place it on top of the "allow everything outside"?

johnpoz

@jt40 said in Proper FW rules for PVLAN:

Not handy to take a screenshot in my situation

how is that? Your on the gui right, how is it your on the gui and can't take a screenshot of what you see. simple keypress on whatever OS you might be using.

@jt40 said in Proper FW rules for PVLAN:

But I have another rule to block all the traffic in ingress to the router (WAN interface)

There is no need for such a rule, deny is the default on any interface.. Out of the box no unsolicited traffic would be allowed inbound on the wan.

But yes if you have a any any rule on lan, which is default - and you don't want it going to vlan X, then on lan above your allow any any rule you would need to create a block/reject rule with the destination your vlanX net

here blocking my lan from going to my test lan

JT40

@johnpoz said in Proper FW rules for PVLAN:

@jt40 said in Proper FW rules for PVLAN:

Not handy to take a screenshot in my situation

how is that? Your on the gui right, how is it your on the gui and can't take a screenshot of what you see. simple keypress on whatever OS you might be using.

@jt40 said in Proper FW rules for PVLAN:

But I have another rule to block all the traffic in ingress to the router (WAN interface)

There is no need for such a rule, deny is the default on any interface.. Out of the box no unsolicited traffic would be allowed inbound on the wan.

But yes if you have a any any rule on lan, which is default - and you don't want it going to vlan X, then on lan above your allow any any rule you would need to create a block/reject rule with the destination your vlanX net

here blocking my lan from going to my test lan

The concern for that FW rule is that my WAN in Pfsense is nothing else than a LAN communication between Pfsense and the modem/router.

I really can't create all these rules manually, I need a floating rule, I'll find the way to make one .

Just as a test, I created a rule in the WAN to block all the traffic from outside, basically:

Interface: WAN (Choose the interface from which packets must come to match this rule)
Source: ANY
Destination: ANY

I didn't understand the first requirement, it goes in contrast with the other 2, or probably it means that it will act only on that interface?
In this case, it should block everything, but it doesn't...
Probably is allowing the outbound connections only...
In any case this is not a VLAN, it's a physical interface, it's my UPLINK towards the modem/router which is sitting under the same network unfortunately.

The easiest would be to create a floating rule:
VLAN to WAN, so I apply it once and for all.

Then, if I need any exception, I'll do it for each VLAN, like for the NAS, Airplay etc...

johnpoz

@jt40 said in Proper FW rules for PVLAN:

s nothing else than a LAN communication between Pfsense and the modem/router.

No its pfsense WAN, doesn't matter if its the lan for some other router.. There is no inbound traffic allowed into pfsense wan unless you allow it, or its an answer to something you allowed out.

Again if you want help - please post your rules.. Or what is odd about your configuration, out of the box pfsense has lets call it 2 networks - its wan, that is how it gets to other networks - the internet for example - doesn't matter what network this is, be it actual public internet, the "lan" of some other router. No traffic is allowed unsolicited inbound.

Then there is pfsense lan, by default it can go anywhere, the any any rule. It is by default natted to the IP that is on pfsense wan.

How many other networks did you create on wan that you can not create rules on them - 1000? 100?

JT40

@johnpoz said in Proper FW rules for PVLAN:

@jt40 said in Proper FW rules for PVLAN:

s nothing else than a LAN communication between Pfsense and the modem/router.

No its pfsense WAN, doesn't matter if its the lan for some other router.. There is no inbound traffic allowed into pfsense wan unless you allow it, or its an answer to something you allowed out.

Again if you want help - please post your rules.. Or what is odd about your configuration, out of the box pfsense has lets call it 2 networks - its wan, that is how it gets to other networks - the internet for example - doesn't matter what network this is, be it actual public internet, the "lan" of some other router. No traffic is allowed unsolicited inbound.

Then there is pfsense lan, by default it can go anywhere, the any any rule. It is by default natted to the IP that is on pfsense wan.

How many other networks did you create on wan that you can not create rules on them - 1000? 100?

Thanks for the explanation.

In the last question, did you actually mean how many VLANs? At the moment, 20, but probably 15 in use max.

Anyway, above the pfsense router, there is only one modem/router and they are both under the same network.
The VLANs obviously are different networks.

johnpoz

@jt40 your saying your "lan" of pfsense is the same its "wan" network?

That won't work.. if your pfsense wan is 192.168.1/24 then your lan of pfsense should be say 192.168.2/24

I have 8 networks, some of them vlans - but network/vlan doesn't really matter the term is pretty interchangeable.. if they are tagged to be isolated on a port or switch, etc then you would be vlans - if they are just native untagged they would just be called a network. But you could call them untagged vlans, etc.

So you have 20 different network segments/vlans on pfsense setup - while it might seem daunting to setup rules on the 20 different interfaces.. Its normally a one time thing, you can copy and past common rules from 1 interface to another..

Than edit them as time goes on, etc.. for one off differences, etc.

You will thank me later when your looking for why something is blocked or allowed when it shouldn't be and how easier the rules are to interpret when their explicitly set on the specific interface vs having to figure out which rules in floating are on what interfaces, are they in or out, are they set quick. Or interface groups, etc..

I have 8 networks - I could setup all the rules again from scratch in a few minutes. 20 isn't all that many.. Now 200 I could see wanting to take shortcuts ;)

JT40

@johnpoz thanks a lot, it makes a lot of sense.
I'll try and let you know.

JT40

Does it make sense to block the traffic from where it is originating?

For example, if I want to block the traffic from VLAN1 to VLAN2, shall I write a block rule from VLAN1?

JT40

I blocked the traffic from the originating VLAN, it works well.
Now, it's not only about 20 VLANs, it's about 20*20=400 rules...
Looking better at my setup, it's about 8 VLANs currently in use, so 64 operations, even though it's copy paste, you still need to change a few items inside or at least check them, making mistakes during the setup or review is so easy.

Btw, I could prove that L2 isolation from Ubiquity works only for certain devices... No sense...

So, on top of that, I need to block the traffic anyway from Pfsense, not only because it should be like that, but because it doesn't work from Ubiquity...
One thing is strange here, even though L2 isolation is enabled, it doesn't explain why it blocks the traffic to some device, instead of allowing all, I mean, Pfsense should be in the middle in this scenario, there is no direct communication between the VLANs of Ubiquity as far as I know.

In fact, an AP can't assign IPs, I never got that option, precisely with Ubiquity. So the routing must happen in Pfsense.
Probably that L2 isolation works "well" only if there is an Ubiquity router on top, it's possible that is a proprietary functionality...

johnpoz

@jt40 said in Proper FW rules for PVLAN:

if I want to block the traffic from VLAN1 to VLAN2, shall I write a block rule from VLAN1?

Correct the rule would go on the vlan1 interface, you block traffic before it enters pfsense.. Why would you allow traffic into your house and then stop them from leaving your house.

Here is an analogy I like... Someone knocks at your front door and says hey can I go to your back yard.. Do you let them into the house and let them walk through your living room with their dirty shoes and then stop them as they try and leave out your backdoor. Or do you just not let them in the front door in the first place ;)

johnpoz

@jt40 said in Proper FW rules for PVLAN:

I could prove that L2 isolation from Ubiquity works only for certain devices

No idea what that is suppose to mean, I have unifi AP and had no issues with L2 isolation. So not sure what your doing or asking about.

Your talking about devices connected to the same SSID/vlan not able to talk to each other? Are they both wireless, is one wireless and other wired?

Are you trying to stop a wireless client from talking to wired client on the same L2? Or a wired client on the same L2 from talking to the wireless client on that L2.

Pfsense has zero to do with devices on the same L2 network from talking to each other - there is no way pfsense could do that, unless you setup device A on one side of bridge and client B was on the other side of the bridge.

JT40

@johnpoz said in Proper FW rules for PVLAN:

@jt40 said in Proper FW rules for PVLAN:

if I want to block the traffic from VLAN1 to VLAN2, shall I write a block rule from VLAN1?

Correct the rule would go on the vlan1 interface, you block traffic before it enters pfsense.. Why would you allow traffic into your house and then stop them from leaving your house.

Here is an analogy I like... Someone knocks at your front door and says hey can I go to your back yard.. Do you let them into the house and let them walk through your living room with their dirty shoes and then stop them as they try and leave out your backdoor. Or do you just not let them in the front door in the first place ;)

With this, I'll never forget it :D

@johnpoz said in Proper FW rules for PVLAN:

@jt40 said in Proper FW rules for PVLAN:

I could prove that L2 isolation from Ubiquity works only for certain devices

No idea what that is suppose to mean, I have unifi AP and had no issues with L2 isolation. So not sure what your doing or asking about.

Your talking about devices connected to the same SSID/vlan not able to talk to each other? Are they both wireless, is one wireless and other wired?

Are you trying to stop a wireless client from talking to wired client on the same L2? Or a wired client on the same L2 from talking to the wireless client on that L2.

Pfsense has zero to do with devices on the same L2 network from talking to each other - there is no way pfsense could do that, unless you setup device A on one side of bridge and client B was on the other side of the bridge.

I mean that by default, L2 isolation is not behaving in the way I expect, it should block the traffic between the 2 SSIDs I have, but it doesn't completely, some device is able to ping the other one, some not.
Plus, the same story is between the SSID VLAN and another "wired" VLAN, which comes from the switch, nothing to do with the AP.

Anyway, I've been testing a lot and I can see that nothing major changed.

These below are the most remarkable scenarios:

I create a rule to stop the traffic from one VLAN to all the others.
How did I do it? I created an alias with the correct IP range of all the VLANs, obviously is much more extensive but it's fine.
It doesn't work unfortunately, or it's intermittent, it's not the first time that I end up in this situation in Pfsense, not even a system reboot helped. On the fly, I also tried to clear the DHCP leases, reset the filter table, reset the state table etc, no positive result.
If I enable the rule above only, it goes in that intermittent behaviour, but after it blocks the traffic (almost entirely), there is no way back...
If I also add up a rule to stop the firewall access from that VLAN, it locks that too.
It doesn't make much sense this behaviour because the filter rules are reloaded almost on the fly, I could have considered my IP range as an issue if it didn't happen in real time...

There is one problem here, the gateway IP is in that range obviously, so, technically speaking, it makes sense to be locked out...
There is one funny thing though, the ping to 8.8.8.8 works :D , and this is the second inconsistency that I find with Pfsense...
The same ping to the modem/router works, which is out of range in that rule alias, but still, the Gateway IP is the first entry point and it should be blocked, so why it doesn't block everything...
I mean, how did I reach the modem/router without passing from the default VLAN gateway, which should be blocked...

Below I list more details:
VLAN 1
Gateway IP: 10.70.70.1 (this should be blocked)
Machine IP: 10.70.70.2
Modem/router: 192.168.0.1

VLAN 2
Gateway IP: 10.70.71.1 (this should be blocked)
Machine IP: 10.70.71.2
Modem/router: 192.168.0.1

The funny story is always the same, if I create a rule to allow the traffic from the VLAN 1 network to ANY, everything works, but I need to block the traffic to the firewall IPs (usually set automatically as every VLAN gateway IP), plus I need to block the traffic to other VLANs.
NOTE that what you said it's not valid in my box, Pfsense doesn't allow the traffic automatically everywhere, it blocks the traffic but not ping, so I need to create a dedicated rule to allow the traffic.

I can do the extensive manual test to lock these things one by one, but there is already some weird behaviour, I don't think it will change much...
I'll also try to act from other VLANs where I have other devices, in this VLAN used for testing, I smell Microsoft issues...