Routing to internal VLANs on switch

jarush · Mar 16, 2019, 1:26 AM

OK, I just removed the upstream gateway from the OPT1 interface. I do have the gateway created and the routes for the downstream networks, however, I still can't get to those downstream networks from hosts using the pfsense as the default gateway.
Attached screenshots of the gw and static routes.

johnpoz · Mar 16, 2019, 1:31 AM

What PART do not get about not setting a gateway On the freaking interface!!

jarush · Mar 16, 2019, 2:17 AM

Apparently the core concept. In your response, are you telling me the screenshots are the way that I shouldn't be configuring it, or should? That is the way it is currently configured (with my pfsense side being 10.0.0.2 and downstream gateway being 10.0.0.1).

By Interface, I assumed you mean the interface that I see when I go to Interfaces -> OPT1. There is no upstream gateway set there. On the edit gateway screen, if I don't put anything in the gateway field, it says "dynamic". I don't understand how this is supposed to work if I don't tell PFsense what the downstream router IP address is.

My assumption, which is obviously wrong, is that I have to configure an interface on PFsense (interfaces -> OPT1) and then tell PFsense what the downstream router IP address is (System -> Routing -> Gateways), and then create a static route to tell PFsense to use this new gateway when going to certain networks. The routing table on the box seems to indicate that the way I have it configured is correct -
10.168.20.0/24 10.0.0.1 UGS ix0

And ifconfig shows the configured 10.0.0.2 IP address on ix0.

Can you tell me what I'm doing wrong? The current state is that the system is configured as you show above (again, with my pfsense interface being 10.0.0.2 and downstream being 10.0.0.1)

Derelict · Mar 16, 2019, 7:10 AM

johnpoz · Mar 16, 2019, 9:31 AM

@jarush said in Routing to internal VLANs on switch:

Can you tell me what I'm doing wrong?

Dude I gave you a freaking picture of how to do it - how can you not understand this basic concept??

So I flipped IPs in my picture - sorry... But come on who cares which side .1 or .2 is on... The point is NO gateway on the interface!!! This is the 4th time stating this..

Normally is the upstream router that would have the lower number..

Again if you put the gateway on the INTERFACE!!! Pfsense thinks is a WAN!! And will now NAT to this ip, etc. You do not want that! Its not a wan, its a transit to a downstream router.. That is not public. Since its a "lan" interface there is not gateway set on the interface.

jarush · Mar 16, 2019, 12:24 PM

I do not have a gateway specified on the interface.

I do have a gateway set on the routing -> gateways -> edit.

I made my settings look like yours last night prior to posting my response.

johnpoz · Mar 16, 2019, 2:09 PM

Ok then do a traceroute from client on your host network to your vlan network 10.168.20.1 interface

Does this hit
12.1, then 0.1 as hops

If you can get your 20.1 interface, but not client on 20.x then you have a problem on the host? (firewall?) Or you host in 20.x is not using 20.1 ?

On your firewall rules on lan and op1 are you forcing a gateway in the rules? If so your overriding pfsense routing tables.

Your L3 should have either default route pointing to 0.0.2 or specific routes to your 10.168.12 network.

jarush · Mar 21, 2019, 1:33 AM

Sorry for the delayed response, was traveling.

Ended up being the firewall. Had the rule in but it wasn't working - unclear why as all I did was move it above another pass rule - maybe the rules just needed to be reloaded.

Appreciate the patience.

Derelict · Mar 21, 2019, 1:39 AM

Rules are processed top down. First match stops processing. If you examine the rule set you will probably see why the more-specific rule needed to be higher.

jarush · Mar 21, 2019, 5:08 AM

But they were all pass rules - I have no deny rules explicitly defined.

johnpoz · Mar 21, 2019, 10:25 AM

f you route traffic out a gateway hard to get to your other vlan, etc..

marvosa · Mar 25, 2019, 8:49 AM

Derelict already explained this with his pic, but I'll add some specifics.

"System -> Routing -> Static Routes" should have a static route for all networks behind the Nexus with a gateway of 10.0.0.1 (Looks like this is done)
"System -> Routing -> Gateways" should have an entry on OPT1 with a Gateway of 10.0.0.1 (Looks like this is done)
Assuming you've enabled routing on the Nexus, remove the VLAN2 you created, re-configure e1/49 as a routed port then give it an IP of 10.0.0.1/30
Configure a default route (not a default-gateway) on your Nexus with the next hop of 10.0.0.2

That's it. I'm running this exact same setup at home. Just to reiterate what's on pic, all hosts behind the Nexus need to be using the IP configured on the SVI of each VLAN as their default gateway in order for the routing to work. You will also need to add helper addresses to each VLAN interface in order to provide DHCP behind the Nexus.