How configure VLAN WAN/UI access rules?

[Irk 0]

Thanks @stephenw10, that's really helpful .

I added another VPN (151 / 192.168.151.0/24) to port igb3 and can see that this was added to the automatic outbound NAT rules:

Perhaps I was tinkering with the gateway options for VLAN 150 before and that somehow flagged it as a WAN interface.

I couldn't see anywhere in pfSense that told me it thinks VLAN 150 was a WAN link though (as far as I could tell it didn't have a gateway assigned). After deleting VLAN 150 and re-adding it, it too was included in the outbound NAT rules, which are now just:

The fact I've lost 2 rows in automatic rules for VLAN 150 backs up your "pfSense likely thinks VLAN 150 is a WAN" observation (and makes me realise I should have included the automatic rules in my earlier post... sorry about that).

Once I get all the WAN and VLAN connectivity working I'll endeavour to adopt a better/default-deny strategy to address the !This_Firewall issue you pointed out.

I have a feeling my tempermental DNS may be related to this issue so I'm giving up on using pfSense's version of unbound for now and will endeavour to set up a seperate DNS server instead.

[stephenw10]

Yup, that fact it was adding outbound rules for TestPoc as a WAN confirms that. It must have had a gateway assigned at least at some point. I've no idea how it would still be using that if it wasn't shown on the interface config though.

I use Unbound everywhere and have never seen that issue. I guess it's possible.
If it was seeing TestPoc as a WAN though Unbound would be blocking queries from it by default.

Steve

[Irk 0]

Things make a bit more sense after some restarts "seemed" to break things even more.

The DNS Resolver seems to be working again now I've added a default IPv4 gateway back in System->Routing->Gateways.

I'd set both the IPv4 and IPv6 default gateways to None thinking it would be best to explicity define the gateway to use in this multi-wan setup. This mean't I had to specify the WANDSL gateway for VLAN 150, i.e. I had to use this rule (instead of the old disabled rule that I think worked courtesy of having a default gateway set):

But having no default route also meant the GUI's Diagnostics->Ping utility couldn't even ping 1.1.1.1 or 8.8.8.8; so there was no way Diagnostics->DNS Lookup was going to work either (or unbound).

Which leaves me wondering if it's possible to get unbound to work without having a default gateway set. That may be a question for another post...

[stephenw10]

You can set the outbound interface(s) for Unbound to use as source IP. In that case it should hit the 'route-to' for that interface and be sent to the gateway on it.

Steve

[Irk 0]

You can set the outbound interface(s) for Unbound to use as source IP

Thanks @stephenw10, I did try that but couldn't get it to work when no default gateway is set.

I set default gateways to None and the following DNS Resolver options:

Then rebooted pfSense and client dig requests just get status: SERVFAIL responses.

In this state pfSense's IPv4 routing table is:

Once the default IPv4 gateway is set to Automatic, client dig requests start working again; the routing table in this case being:

I've since found this post which claims:

"The outgoing interfaces option in Unbound is just an interface by interface on/off switch whether to use the interface at all for outgoing traffic, it doesn't allow any redirection of DNS queries, they will still follow the system routing table"

It seems unbound is only able to work when a default gateway is set. Which currently seems odd to me; I guess I have some more reading to do to understand how this could work in my (concurrent, rather than failover) multi WAN setup.

[stephenw10]

Yes, the outbound interface only sets the source IP that is used. But if that is a WAN the outbound firewall rule will look something like:

pass out  route-to ( em0 172.21.16.1 ) from 172.21.16.22 to !172.21.16.0/24 ridentifier 1000002761 keep state allow-opts label "let out anything from firewall host itself"

So traffic using it should routed via the gateway.

However PPPoE can be a special case here because the gateway in use is often not in the WAN subnet. It doesn't have to be in a point-to-point connection. It's still in your routing table on the link though....

Steve

[Irk 0]

OK, thanks @stephenw10, I see one can view those rules with pfctl so will look for similar entries on my box.

Should anyone else attempting to lock a VLAN to a single WAN interface wind up reading this, it turns out setting the default gateways to none wasn't the only way to do this (setting no gateways having the downside of stopping pfSense services such as DNS from working).

It turns out there's a "Skip Rules When Gateway is Down" option that I had to set under System->Advanced->Miscellaneous:

Until I set the above option the gateway I explicity set for the VLAN was being removed if it was down, meaning pfSense's default gateway was used instead (which being a gateway group with all WAN interfaces, meant traffic found a path out).

That seems an odd default thing to do to me; I'd expected the default to be no connectivity. If one wants failover to another gateway it may be clearer to explicitly say so by assigning a gateway group that includes the specific failover interface(s) required.

[stephenw10]

Yes, when I referenced that initially I suspected that might be the case. It's one reason why you should not rely on policy routing to block traffic. Use block or deny rules above it to prevent access via any gateway. But setting that will remove the rule entirely as you found.
It defaults to that because in most cases policy routing is used for failover or load-balancing and it's preferable to have connections succeed.

You can also see the ruleset pf loads in /tmp/rules.debug.

Steve

[Irk 0]

Eek. You did indeed mention the skip rules thing before! Sorry. I guess the significance of that statement didn't register with me at the time. Doh .

At one point in my journey I did try using reject rules to block the other gateways, but it seemed to me that the "default gateway" was treated as a special case and couldn't be blocked. It's entirely possible that I had things half-baked at the time though. Perhaps I will find time to revisit that approach at some point.

Is setting "skip rules when gateway is down" considered a bad practice, or something that should be avoided?

If there's a good reason to not use it I can revert to the default setting, but I'd prefer to take the approach of "block everything" and "only allow what one wants" that the "skip rules..." option seems to give.

For example, to allow VLAN 151 to access VLAN 150, but not VLAN 150 to access VLAN 151, with both having internet and DNS access, it appears I can just do the following (without having to explicitly block anything, which might be a pain should more VLANs be added later):

[stephenw10]

Yes, that should work. However generally considered best practice would be something like this:

If the 'pass-all' rule were to change to something without a gateway it would not then allow local traffic.

Steve

[Irk 0]

Thanks @stephenw10, the rule screenshot's really helpful.

I was able to get that rule set working (as long as the "skip rules..." option was selected).