Vlan and routing

itian · Jul 10, 2015, 8:59 PM

Hello,

I've been setting up a pfsense box on a dell pc, i have a layer 3 switch with multiable vlans, i've followed this guide:

https://www.iceflatline.com/2013/09/how-to-create-and-configure-vlans-in-pfsense/

I managed to get the internet workings from a vlan (vlan10) but only if I pointed my laptop default gateway to the static IP for the vlan on the pfsence box (which was 10.52.10.250), but my layer 3 switch has a gateway on the vlan (10.52.10.254), I also have a route in place on the core:

ip route 0.0.0.0 0.0.0.0 10.52.100.123 (10.52.100.123 is my firewall)

DHCP is configured on the network with the layer 3 vlan default gateways.

If I change the pfsense vlan default gateway to match the layer 3 vlan default gateway I dont get any internet.

Any idea?

bennyc · Jul 10, 2015, 9:30 PM

Plenty of ideas ;)

That setup should work. However, we are missing quite some information here.
Assuming you did not mess up the ip/routing part, are you sure it is not simply a dns resolving issue? Who does dhcp? And what dns server does it hand out to its clients? (hint: it should be the ip of pfsense)

itian · Jul 10, 2015, 9:34 PM

The strange thing was before I made the change on the pfsense box (matching the vlans default gateways) everything was working fine, I could ping google dns server for example (8.8.8.8) and ping the pfsense box.

As soon as I amended the vlan ip on the pfsense box to match my layer 3 core, I was unable to ping the pfsense box and google dns server (no internet)

DHCP is dished out by my DC, i setup a static ip on my laptop during testing.

Derelict · Jul 10, 2015, 9:44 PM

Are you using the switch as a layer 2 or layer 3 switch? Give specifics as to segments, where and what the SVIs are, where the DHCP servers are, etc, what default gateway and DNS settings are on the specific clients.

Even better, diagram it - or at least enough of it to cover all types of interfaces.

heper · Jul 10, 2015, 9:45 PM

draw a schematic of your setup.

also is pfsense routing the vlans or is your switch routing the vlans ?

itian · Jul 11, 2015, 6:18 PM

Hi,

Thanks for the replies, I am using a HP ProCurve 2610 acting as my Layer 3 switch which does all my routing for my vlans.

My DHCP Server is my domain controller and I have have one DHCP Server, I have 4 VLANS:

VLAN10 for Servers (Default Gateway on Layer 3: 10.52.10.254)
VLAN16 for Clients (Default Gateway on Layer 3: 10.52.16.254)
VLAN5 for Switch Management (Default Gateway on Layer 3: 10.52.5.254)
VLAN100 for Firewall (Default Gateway on Layer 3: 10.52.100.254)

DNS on my clients are pointing to the Domain Controller.

johnpoz · Jul 11, 2015, 7:49 PM

This is basic setup - And should work right out of the box other than your lan rule would default to lan net that is not going to allow your down stream networks. So your default lan rules have to be modified, and routes to the downstream networks needs to be created.

So I have drawn up a sample network.. Using a 172.168.0/30 as the transit network.. The one question I would have for you is do you have clients on the transit network? Normally a bad idea to do that.

So your clients on your down stream segments would use your L3 switch SVIs in those segments/vlans as their gateway. The 192.168.x.1 addresses in the picture. Since your L3 is routing, its gateway would be the pfsense lan interface in this case 172.16.0.1 Keep in mind out of the box pfsense would only allow source IP of lan net on its lan interface so you would have to adjust that rule to include the networks that are downstream of pfsense.

You also need to create a route on pfsense that points to your downstream router for the networks attached to it. In my attached sample a simple 192.168/16 route pointing to 172.16.0.2 would work.

Keep in mind this is NOT a gateway you setup on pfsense lan, this is a simple ROUTE!!

itian · Jul 11, 2015, 8:24 PM

I've attached some pictures if this helps.

Derelict · Jul 11, 2015, 9:32 PM

With that configuration you are using the switch as a layer 2 switch.

You really should get a handle on layers 2 and 3 if you're going to have a prayer at getting this working.

http://www.ircbeginner.com/ircinfo/Routing_Article.pdf

johnpoz · Jul 11, 2015, 10:32 PM

so your switch is layer 2 or 3? And pfsense is the gateway for all 3 vlans then? There is a HUGE freaking difference!!!

itian · Jul 11, 2015, 11:11 PM

Switch is the Layer 3, the gateways on the vlans is my layer 3 switch.

heper · Jul 11, 2015, 11:40 PM

then why configure the vlans on pfsense at all?

itian · Jul 11, 2015, 11:48 PM

I see, so I dont need to configure any vlans on the pfsense box?

On the switch side I have tagged the vlan traffic going to the pfsense box, is that correct?

Derelict · Jul 11, 2015, 11:52 PM

Not necessary for a transit network but you certainly can. If I'm talking to a managed switch I usually tag it even if it's only one VLAN. That way you can add another if you need to without either taking it down or mixing tagged and untagged traffic.

itian · Jul 11, 2015, 11:59 PM

Okay, should I remove the vlans config from the pfsense box?

phil.davis · Jul 12, 2015, 3:24 AM

@itian:

Okay, should I remove the vlans config from the pfsense box?

General principles:
a) You can use a "layer 3 switch" as a "router" - the words "layer 3" and "switch" put together only make sense when translated to "router" :) - in that case each subnet on the "layer 3 switch" is in a separate VLAN and the "layer 3 switch" has an IP address in each subnet/VLAN which is the gateway for that subnet/VLAN. Then the "layer 3 switch" routes upstream to somewhere - in this case pfSense.

b) You can have a "layer 3 switch" and just use it for layer 2, ignore its routing capability. In that case it becomes like a "smart switch"/"VLAN switch" - you make multiple VLANs on it and then trunk all those VLANs straight up to the upstream device (pfSense) and put a VLAN trunk port on the upstream device and have the upstream device do all the routing.

If there is a lot of internal traffic between the subnets/VLANs then (a) is generally better for performance. Otherwise it is somewhat a matter of choice about where to do the routing.

Sounds like you are doing (a). So remove all the individual VLANs from pfSense and do like @johnpoz has described.

If you like, you can have a single tagged VLAN between "layer 3 switch" and pfSense - like @Derelict does - or you can leave it as ordinary untagged.

As you can see, there is more than 1 way to skin a cat, and sometimes it is just a matter of preference.

johnpoz · Jul 12, 2015, 11:36 AM

As phil pointed out where you do the routing can come down to a matter of choice, if you want to firewall between your segments for example then pfsense might be better then your l3 switch. Keep in mind that if all you have is 1 interface and your going to put all your segments on that via vlans then traffic in and out of that interface is shared by all segments.

You will be hairpinning traffic when vlan 10 wants to talk to vlan 20 all traffic goes in pfsense interface and then back out same interface again to go to vlan 20, etc..

Which is why if lots of traffic between vlan doing it at the switch is can be better for performance. If firewall between your segments is what your after then you might want to look to getting some more interfaces for pfsense other just all your in out traffic using 1 interface.. Unless that is a 10ge connection your prob going to run into performance issues.

itian · Jul 12, 2015, 4:01 PM

Okay, so I have done a factory restore on the box and given it an IP of 10.52.100.123 (VLAN100), connects to the internet etc fine, port the pfsense box is connected too is untagged for VLAN100, the other traffic for VLAN10 and 16 is tagged on the same port.

If I give my laptop a IP within the VLAN100 range I can get internet access fine, if I then plug myself into a vlan16 or 10 port im unable to get any internet access nore ping the pfsense box.

I have the following route setup on the pfsense box, see picture.

My ip route on the layer 3 is as follows:

ip route 0.0.0.0 0.0.0.0 10.52.100.123

I can ping 10.52.100.123 from the core CLI.

Derelict · Jul 12, 2015, 6:06 PM

You are telling pfSense to route traffic for your LANs out WAN. Probably not what you want.

IF YOU ARE USING THE LAYER 3 CAPABILITIES OF THE SWITCH YOU ONLY NEED THE UNTAGGED INTERFACE ON PFSENSE!

You need the default route on the switch, which you have. You also need to create a gateway on pfSense for the switch and create routes for the networks pfSense doesn't know about with a destination of that gateway, not WAN.

Then you need to make sure the firewall rules on your pfSense LAN interface will pass the traffic FROM the "foreign" networks (the networks behind the L3 switch.)

I think automatic outbound NAT is now smart enough to create the NAT rules and everything. if not, they need to be in place for all your LAN sources on WAN.

itian · Jul 12, 2015, 10:03 PM

Hi - just to update you, I have now managed to get this all working :)

Thanks for all your help.