Home Network Help - Cisco and pfSense

phlyingpenguin

I'm new to the networking field, just exiting high school, and I figured as a cool project and a proactive way to learn basic networking and network management tools such as Nagios, OSSIM, etc, I would build out my home network.

So, below is a diagram of how I want to have it laid out.

So, to explain a bit ... I want to have my ISP connection come into their equipment (in my case an ONT). From the ONT, I am going to have the WAN connection come into a pfSense firewall. My firewall has 3 NICs in it that I plan to use for WAN, LAN, and future DMZ.

The DMZ interface on the firewall will connect into a DMZ switch, where my DMZ systems connect. The LAN interface will connect into my Cisco 3560G switch, and from here I have 3 segments defined - LAN, WLAN, and Services. Each segment will be a separate VLAN, in my case LAN=1681, WLAN=1682, and Services=1683.

The IP scheme for my interfaces/VLANs is as follows:

Firewall LAN Interface IP - 10.10.10.5/30
Switch (G0/1 to FW LAN Interface) - 10.10.10.6/30
Switch VLAN 1681 LAN - 192.168.1.240/24
Switch VLAN 1682 WLAN - 192.168.2.240/24
Switch VLAN 1683 Services - 192.168.3.240/24

Now, getting down to my reason for posting. I tried to implement this last night, and failed. So, from each hop here were my results:

Firewall - Internet connectivity was established, and I was able to hit external sites. I could also ping my next hop on the Cisco of 10.10.10.6, as well as the various VLAN IPs, and my machine IP of 192.168.1.30.

Cisco - I was able to ping my next hop (the firewall) of 10.10.10.5, as well as all VLAN IPs and my machine's IP of 192.168.1.30. I could NOT ping external sites (via IP).

My machine - I could ping my default gateway 192.168.1.240, I could ping all VLAN IPs, and I could hit the firewall and bring up its GUI 10.10.10.5. I could NOT hit anything external via IP or DNS.

So, what went wrong? Here is how I configured everything, and yes I know its going to make a long post and I'm sorry.

On my Cisco:

I first enabled routing, and the ability to use CIDR addressng


Switch(config)#ip routing
Switch(config)#ip classless

Next, I setup my VLANs


Switch(config)# vlan 1681
Switch(config-vlan)# name LAN


Switch(config)# vlan 1682
Switch(config-vlan)# name WLAN


Switch(config)# vlan 1683
Switch(config-vlan)# name Services


Switch(config)#interface Vlan1681
Switch(config-if)#ip address 192.168.1.240 255.255.255.0
Switch(config-if)#no shutdown


Switch(config)#interface Vlan1682
Switch(config-if)#ip address 192.168.2.240 255.255.255.0
Switch(config-if)#no shutdown


Switch(config)#interface Vlan1683
Switch(config-if)#ip address 192.168.3.240 255.255.255.0
Switch(config-if)#no shutdown

I next assigned interfaces to those VLANs


Switch(config)#interface range gigabitethernet 0/3-7
Switch(config-if-range)#switchport mode access
Switch(config-if-range)#switchport access vlan 1681
Switch(config-if-range)#no shutdown


Switch(config)#interface range gigabitethernet 0/8-12
Switch(config-if-range)#switchport mode access
Switch(config-if-range)#switchport access vlan 1683
Switch(config-if-range)#no shutdown


Switch(config)#interface gigabitethernet 0/2
Switch(config-if)#switchport mode access
Switch(config-if)#switchport access vlan 1682
Switch(config-if)#description WLAN_AP
Switch(config-if)#no shutdown

Now, all that should have been left was to configure the interface up to the firewall, and setup my gateway of last resort.


Switch(config)#interface gigabitethernet 0/1
Switch(config-if)#no switchport
Switch(config-if)#ip address 10.10.10.6 255.255.255.252
Switch(config-if)#description FW_Transit
Switch(config-if)#no shutdown


Switch(config)#ip route 0.0.0.0 0.0.0.0 10.10.10.5

To my knowledge, that covered everything on the Cisco. Did I miss anything?

On my firewall, it already had a default route pointing out its WAN interface. So, to get communication up to my LAN, I ensured I had static routes configured for each subnet pointing to the LAN next hop of 10.10.10.6. I also ensured that all traffic originating from the LAN interface was allowed outbound keeping state.

pfSense does not support DHCP scopes yet, so I disabled the DHCP server which also stopped it from passing DNS info to my machines. So, I tried putting the FW as my DNS server, as well as manually putting my ISPs DNS. These both failed, and obviously the ISP servers failed as my machine couldnt establish an outbound connection (despite being able to manage the firewall via its GUI on an off-subnet IP).

So, that is my dilemma. I'm hoping this made sense, and someone can help me.

Any help is appreciated, and I'm sorry for the long post.

cmb

@phlyingpenguin:

I'm new to the networking field, just exiting high school, and I figured as a cool project and a proactive way to learn basic networking and network management tools such as Nagios, OSSIM, etc, I would build out my home network.

Good plan, you sound like a 10 year younger version of myself. ;D Keep it up and you'll do well.

I suspect your problem is you didn't edit the default LAN rule on pfSense. It only allows the local LAN subnet as source by default. Another issue may be missing static routes on the firewall, pointing the VLAN IP subnets to the switch IP, though it sounds like you may have those. Based on your description, I suspect it's one or both of those.

phlyingpenguin

Thanks for the reply, cmb.

I did have static routes pointing down to the switch for each individual VLAN segment. For the LAN firewall rule, I had changed it from permitting just the LAN segment (the default rule), which would only allow the /30 link between the firewall and the Cisco, to Any which to me should have covered anything coming into the LAN interface. I also made sure that it was set to keep state.

I did not try a rule for each VLAN segment, which I assume is what you are suggesting.

To go a bit farther, at an IP packet level, I believe on the MAC and the TTL change per hop correct? So, the destination address is the same, and the source would remain the same (which would be the 192.168.1.x address). So, from my PC the packet would have my NIC's MAC, its source IP, and the Dest IP of its target. When it hits the link between the Cisco and the FW, the MAC would change to the Cisco's MAC and the TTL would change again, but the source IP would be the same (still my machine). Is this correct? If that is correct, then the rule to keep state should allow it back in as long as it knows how to get back to the VLAN segments, which I had static routes in for. And the MAC it would know from its ARP table. Does that sound correct or am I missing something?

cmb

@phlyingpenguin:

I did not try a rule for each VLAN segment, which I assume is what you are suggesting.

No, actually I just meant to do as you did, open it up to source "any" if that's how you want your egress filtering to work. The antispoofing will limit "any" to internal networks as defined by the routing table.

@phlyingpenguin:

To go a bit farther, at an IP packet level, I believe on the MAC and the TTL change per hop correct?

Correct.

@phlyingpenguin:

So, the destination address is the same, and the source would remain the same (which would be the 192.168.1.x address). So, from my PC the packet would have my NIC's MAC, its source IP, and the Dest IP of its target. When it hits the link between the Cisco and the FW, the MAC would change to the Cisco's MAC and the TTL would change again, but the source IP would be the same (still my machine). Is this correct? If that is correct, then the rule to keep state should allow it back in as long as it knows how to get back to the VLAN segments, which I had static routes in for. And the MAC it would know from its ARP table. Does that sound correct or am I missing something?

Yep, that's all correct.

What does a traceroute from a VLAN host to the Internet look like?

phlyingpenguin

Well, my network is working 100% now. And I didn't do anything differently that I know of.

I went through the exact same steps I did before, figuring I would at least have access to my firewall again to mess around with the rules, and when I was done I figured what the heck let me just try for external access and it worked. Now, granted due to my design, pfsense doesnt support multiple DHCP scopes so I have to manually configure IPs and DNS entries for now but it works. I'm guessing I had a typo somewhere that I couldnt find the first time that was preventing me from getting outbound last time.

Thanks for all the help cmb. Now, I just have to get drives for my desktop, install VMware ESX, and begin getting all my virtual machines setup for in my DMZ and local network.