nested VLANs can't connect through PFSense

Derelict

Switching to hybrid outbound NAT is not good enough.

You need to be sure that any source addresses that require NAT are actually included there.

If not they need to be manually added.

This is usually picked up by pfSense from the static route that routes the traffic for those networks to the Layer 3 Switch/Downstream Router.

tahunua

explicit NAT rules for each subnet were part of my original configuration before TS steps began. adding the static routes was the most recent change to config.

update: I have compared my NAT rules with the pool from my old gateway, and I did discover an error in the public IP mappings but after correcting this in the NAT rules I still have no change in operation.

Derelict

Well something wasn't done correctly or it would have worked.

Your best bet is to get it to where you think it should be working but isn't and post screenshots of all of the elements outlined in the dialog boxes in that diagram. Pretty sure I covered everything that needs to be in place there.

tahunua

ok I have finally figured out what has happened.

the PF sense appliance was not smart enough to find my core switch as an internal gateway unless I reconfigured the switch port as a routed port and assigned the IP address directly to the port, rather than an access port and using the SVI as the gateway address.
with all rules disabled, the PFsense still maintained the deny all rule at the end, so all traffic was being denied. after I just created an outbound rule that allowed everything, my local vlans can now pass traffic so it is just me being an idiot with my rules.

thank you very much for your help. I feel like such a newb right now.no to start securing it.

Derelict

@tahunua said in nested VLANs can't connect through PFSense:

ok I have finally figured out what has happened.

the PF sense appliance was not smart enough to find my core switch as an internal gateway unless I reconfigured the switch port as a routed port and assigned the IP address directly to the port, rather than an access port and using the SVI as the gateway address.

If your switch properly responded to ARP in that configuration it would have worked. It is all standards-based. There are no "smarts" necessary. If the gateway is defined on an interface and routed to, and it responds to ARP, that's where the traffic will be sent. Not familiar enough with your switch so say whether it would one way or another in that configuration.

This is my transit interface on my brocade switch. Nothing special needed on the switch ports. Just tagged to the LACP LAGG and an "SVI":

vlan 1000 name TRANSIT by port
 tagged ethe 1/1/35 to 1/1/36 
 router-interface ve 1000
!
interface ve 1000
 ip address 192.168.230.2 255.255.255.252
 ip ospf area 0
!

with all rules disabled, the PFsense still maintained the deny all rule at the end

Wholly expected.

so all traffic was being denied. after I just created an outbound rule that allowed everything, my local vlans can now pass traffic so it is just me being an idiot with my rules.

You mean a rule allowing those subnet source addresses into the pfSense side of the transit interface right?

JKnott

@tahunua said in nested VLANs can't connect through PFSense:

the PF sense appliance was not smart enough to find my core switch as an internal gateway unless I reconfigured the switch port as a routed port and assigned the IP address directly to the port, rather than an access port and using the SVI as the gateway address.

????

Switches are normally transparent. You have routers as gateways, not switches. BTW, where is this core switch. I only see 1 switch.

JKnott

@Derelict said in nested VLANs can't connect through PFSense:

If your switch properly responded to ARP in that configuration it would have worked. It is all standards-based.

????

Why would a switch respond to an ARP, unless you're accessing the management interface? I suspect the confusion may be caused by the OPs descriptions. What switch is he referring to? Is it layer 2 or 3? If 3, then it would respond to an ARP request, as would any router.

tahunua

You mean a rule allowing those subnet source addresses into the pfSense side of the transit interface right?

actually I did what you should never do in a live environment and did a shotgun rule to allow any source to any destination on any port using any protocol. my current set up is more targeted, with rules applying to specific ports and protocols with specific source IP networks. so far it is stable, and just has a few hiccups which I should be able to TS on my own from here out.

as for ARP responses, I don't know what could have been the source of the interference. so far the routed port has remained stable.

Switches are normally transparent. You have routers as gateways, not switches. BTW, where is this core switch. I only see 1 switch.

I am going to assume you are self educated when it comes to networking. there is nothing wrong with this, in fact I admire people who have figured this all out for themselves. however you appear to be missing a fair bit of nomenclature/terminology when it comes to network design and network diagrams.

a core switch refers to the backbone of a LAN, CAN, or MAN. it is the framework that disseminates data throughout a building, campus, or organization. typically because of a need for added security, compartmentalization, and optimization, core switches rarely are straight layer 2 forwarding devices. instead they have a lot of the capabilities of routers, firewalls, monitors and other layer 3 and up devices. my core switch is very basic so it does not have more advanced layer 3 features such as NAT, but it can do static and dynamic layer 3 routing, vlan tagging and segregation, etherchannel (also called channel aggregation), and DHCP services. on a network diagram, multilayer switches such as this do not have the normal rectangle with 2 arrows pointing left and right, instead they are typically a square with arrows fanning out in all directions. the device on the left of Derelict's diagram is a multilayer (Layer 3) switch, while the device on the right hand side is a basic layer 2 (mostly transparent) switch.

Derelict

@JKnott said in nested VLANs can't connect through PFSense:

Why would a switch respond to an ARP, unless you're accessing the management interface? I suspect the confusion may be caused by the OPs descriptions. What switch is he referring to? Is it layer 2 or 3? If 3, then it would respond to an ARP request, as would any router.

Layer 3 interface, man. Layer 3.

tahunua

I should also use the disclaimer that I tend to use a lot of Cisco Specific terminology so I apologize if there are vendor neutral terms that would make it easier to understand what I am trying to communicate. with the exception of the PFsense appliance running on an old Dell server blade, my entire network is Cisco.

JKnott

@Derelict said in nested VLANs can't connect through PFSense:

Layer 3 interface, man. Layer 3.

Yes, however, my point was he was not clear in what he was referring to. If a layer 3 switch, then that should have been explicit. The diagram refers to both L2 and L3/

Derelict

@Jknott knows all of this. It's a brain fart.

It was perfectly clear in the context of the rest of the thread and what has already been discussed. If you want to jump in and issue a correction for something basic like that, might be good to read a few posts back.

JKnott

@tahunua said in nested VLANs can't connect through PFSense:

I am going to assume you are self educated when it comes to networking. there is nothing wrong with this, in fact I admire people who have figured this all out for themselves. however you appear to be missing a fair bit of nomenclature/terminology when it comes to network design and network diagrams.

Actually, I'm a CCNA too. However, trying to follow your description is confusing. If you meant L3 switch, you should consistently say so. Your diagram refers to the switch as both L2 & L3, but the sketch is the one used for a swtich. You also refer to a core switch, but I don't see that term anywhere in the diagram. I just see the one switch.

Derelict

Dude, the rest of the post from which you decided to pull that one line makes it obvious we are talking about a Layer 3 switch.

tahunua

can I also point out I am not the one who posted the diagram, nor is that the configuration of my network? Derelict posted it to show me a proper routing configuration when dealing with multiple vlans... and the diagram clearly shows an L3 switch and an L2 switch.