General Questions about PfSense…



  • I am running PfSense in a physical machine with two physical gigabit NICs.  BG0 is WAN, BG1 is LAN.
    I am using a Cisco SG300-20 managed switch in L3 mode with multiple VLANs, as follows:

    PfSense WAN interface connected to ATT 2-Wire router, in DMZ mode (WAN interface has actual Internet IP address via ATT DHCP).
    VLAN2 - 192.168.2.0/24 - port 2 PVID in Access mode, VLAN IP is 192.168.2.254, connected to PfSense LAN interface which has a static IP of 192.168.2.253.
    VLAN3 - 192.168.3.0/24 - port 3 PVID in Access mode, VLAN IP is 192.168.3.254, connected to a Server 2008 R2 physical box with static IP of 192.168.3.1, running DHCP, DNS and Active Directory.  (I set up PfSense firewall rules and options so that only my internal DNS server is used by all machines and PfSense.)
    VLAN4 - 192.168.4.0/24 - port 4 PVID in Access mode, VLAN IP is 192.168.4.254, connected to a 24-port gigabit switch with workstations, with IPs of 192.168.4.1-253 via DHCP.

    The Cisco switch is doing the inter-VLAN routing, and on the switch I set a default path of 0.0.0.0 going to the PfSense LAN IP (192.168.2.253).  I have set up static routes in PfSense for all VLANs except PfSense's (2.x) so that 192.168.x.0/24 uses 192.168.2.254 as the default route, which works.  Also, all hosts except the PfSense LAN interface, which is the only host on VLAN2, have their gateways set to their respective VLAN addresses (ex., 192.168.3.x has a gateway of 192.168.3.254).

    Even though everything is working fine (DHCP, DNS, Internet access, complete VLAN connectivity from all internal hosts to all internal hosts and to and from the PfSense box), I still have a few anomalies and one or two questions:

    1.  Since I am using Access mode on all of the ports on the L3 switch, and since the switch is doing the inter-VLAN routing, do I still need to set up VLANs in PfSense?  (I do realize that if I do, I will also need to change port 2 to Trunk mode, set the PVID to VLAN2 and set VLANs 3 and 4 as Tagged.)

    2.  Despite having a non-logged rule to block all traffic from the WAN IP to any destination, I am getting about 30 log entries per minute with the interface being a black arrow pointing left and "WAN," the source is my WAN IP and the destinations are external Internet hosts (most of which are Google, Akamaitechnologies and Amazon AWS servers).  The protocol for all of them is TCP-FA.

    As a test, I used the log entry to create an EasyRule to both block and allow it, but it still appears in the log is blocked under each scenario.

    Thinking this might be an issue with asymmetric routing, I have enabled All TCP flags and set the State Type to Sloppy State in the Advanced Settings section for every firewall rule.  I have set a gateway of 192.168.2.254 on the LAN interface as well as removed it.  I have set a static route for 192.168.2.0 to use 192.168.2.254 along with the rest of the VLANs as well as removed it.  I'm not sure what else I can do.  I want my Cisco L3 switch to to do the routing, not PfSense.

    Any help would be greatly appreciated.



  • 1.  Since I am using Access mode on all of the ports on the L3 switch, and since the switch is doing the inter-VLAN routing, do I still need to set up VLANs in PfSense?

    No, you don't need any VLAN's on PFsense for your setup, but I would make one small change:

    • Remove VLAN 2 and reconfigure port 2 as a routed port (instead of an access port) with an IP of 192.168.2.254

    2.  Despite having a non-logged rule to block all traffic from the WAN IP to any destination

    Can you elaborate on why you did this?  I don't see why it would be needed…. personally I'd remove it... but I'm curious to know why you put it in there.  My setup is nearly identical to yours

    Considering you've done a bunch of adding and removing, after you reconfigure port 2 and possibly remove your block rule above, I would bounce PFsense.  On occasion in the past, PFsense would not actually remove certain settings even though it was gone from the GUI and a reboot would fix the issue.

    I set up PfSense firewall rules and options so that only my internal DNS server is used by all machines and PfSense.)

    It may be just the way it was worded, but PFsense should not be using your internal DNS server.



  • 1.  I'm not sure what you mean by setting the port as a "routed" port.  AFAIK, ports can be Access, Trunk, General, Customer or Private.  Port 2 and untagged traffic to and from VLAN2 are already routed by the switch to and from the rest of the LAN and the Internet.  Also, if the port isn't an Access port, then traffic to and from 2.x will be tagged, which I would assume will be an issue since there are no VLANs set up in PfSense and therefore PfSense isn't expecting tagged traffic.

    2.  I put it in there because I didn't want the WAN interface itself initiating communication with any hosts, internal or external, without the request coming from either a host on the LAN or incoming from the Internet (which of course is also blocked except for data ports I wanted to allow for internal hosting of various services).

    Also, I am not sure what you mean by saying I should remove VLAN2 - do you mean delete the VLAN itself or just remove it from port 2?  If I remove it from port 2, how will hosts (PfSense) have a 2.x address?  Don't I need VLAN2 so that traffic from other VLANs can be routed to hosts on VLAN2 by the switch?

    Lastly, why shouldn't I force all hosts on my LAN, including PfSense, to use my internal Server 2008R2 DNS server and no other server?  I've set up firewall rules to allow the DNS server (192.168.3.1) traffice out to do DNS queries to a DNS server on the Internet.  I use this method to restrict what DNS servers on the Internet are accessed by any of my lookups/resolutions.

    P.S…how do you do multiple quotes from the same post like that?  I will use that for future posts to keep things a bit more tidy.

    Thanks in advance for your time.



  • 1.  I'm not sure what you mean by setting the port as a "routed" port.  AFAIK, ports can be Access, Trunk, General, Customer or Private.  Port 2 and untagged traffic to and from VLAN2 are already routed by the switch to and from the rest of the LAN and the Internet.  Also, if the port isn't an Access port, then traffic to and from 2.x will be tagged, which I would assume will be an issue since there are no VLANs set up in PfSense and therefore PfSense isn't expecting tagged traffic.

    a "routed port" meaning… remove the config from port 2... i.e. remove "switchport access vlan 2", "switchport mode access", etc... and add "no switchport" and assign it an IP.

    2.  I put it in there because I didn't want the WAN interface itself initiating communication with any hosts, internal or external, without the request coming from either a host on the LAN or incoming from the Internet (which of course is also blocked except for data ports I wanted to allow for internal hosting of various services).

    I understand the logic, but you'll have to give us an example of a scenario where the WAN interface would initiate communication with an internal host on it's own.  I think you're over thinking it.  All incoming traffic from untrusted interfaces (WAN) are blocked by default and all OPT interfaces are enable with no rules by default.  Someone correct me if I'm mistaken, but the only thing that's allowed by default is outgoing internet traffic from reserved addresses. In other words, out of the box nothing is going to traverse the firewall that isn't explicitly configured to do so.  I don't think you need your block rule…. it's most likely redundant and already blocked.

    Also, I am not sure what you mean by saying I should remove VLAN2 - do you mean delete the VLAN itself or just remove it from port 2?  If I remove it from port 2, how will hosts (PfSense) have a 2.x address?  Don't I need VLAN2 so that traffic from other VLANs can be routed to hosts on VLAN2 by the switch?

    If you're using a Cisco L3 switch with an IP base feature set, you should be using a routed port instead of an access port.  So, remove VLAN 2, then once port 2 is configured as a routed port, it turns into a L3 interface that can be assigned an IP.

    Lastly, why shouldn't I force all hosts on my LAN, including PfSense, to use my internal Server 2008R2 DNS server and no other server?  I've set up firewall rules to allow the DNS server (192.168.3.1) traffice out to do DNS queries to a DNS server on the Internet.  I use this method to restrict what DNS servers on the Internet are accessed by any of my lookups/resolutions.

    PFsense should be using upstream DNS servers (typically your ISP's) for DNS lookups.  If you look at the "forwarders" tab on your DNS server, you'll notice that it's pointed back at PFsense, which means your DC is forwarding requests it can't resolve to PFsense then PFsense should be forwarding requests it can't resolve to an upstream DNS server… not back to your DC.  That is going to slow down your DNS queries.  Also, if you take a look at Status -> Interfaces, in the "WAN Interface" section, you'll notice the DNS servers are labeled "ISP DNS servers".  The devs should probably note that in the General Setup section as well.

    As far as forcing hosts to use your DNS, that's fine, but unless you're providing content filtering via something like OpenDNS and you're trying to prevent people from bypassing it by manually setting their DNS it just seems like an unnecessary step.  Not that you're the first person to do it, but IMO if it's something you're worried about, just configure a GPO and prevent users from accessing the Ethernet properties.  It'll save you several phone calls down the road.

    P.S…how do you do multiple quotes from the same post like that?  I will use that for future posts to keep things a bit more tidy.

    When posting, look towards the middle of the formatting buttons… you will see a button with a white message bubble icon that is labeled "Insert Quote" when you hover over it.  It will insert quote tags where ever your cursor is... then just paste your text in between the tags.  You can also paste your text first, then highlight what you want quoted, press the "Insert Quote" button and it will insert the tags around what you have highlighted.



  • Well, I believe I have my network set up the way I want, I prefer all DNS requests going through my internal DNS server for logging and Active Directory purposes. I disabled the proper options in PfSense and my DNS server to ensure I didn't have the bounceback and looping you mentioned. I use ACLs on the Cisco L3 switch to restrict access to management stuff.

    Basically, what I really want to figure out is, why do I have all of these repeating, constant entries in my log? Here is what's in the "If" column:

    The "Act" column indicates blocked, and clicking that indicates "@6(1000000104) block drop out log inet all label 'Default deny rule IPv4.'" The "Source" is my WAN IP, and the destinations are various Google servers (i.e., dfw06s46-in-f14.1e100.net). The "Proto" column indicates TCP-PA for all of them.



  • I figured out what was going on and I am posting here in case anyone else has the issue and can use the information…

    It turns out the black arrow pointing to the left in the interface column in the firewall log means that it is a communication that is OUTBOUND from the WAN interface itself, not from a host using the WAN interface. This was happening because I had created a separate VLAN (say, VLAN5 at 192.168.5.0), assigned an Access port to it and set it as that port's PVID. I had then connected a LAN port on a Linksys 4-port router to that port on my L3 switch thinking that the Linksys would just act as a dumb switch and allow the wireless clients to connect.

    What was happening is that a wireless host would communicate OUTBOUND, and the external host would reply but the NAT on the router was routing the reply back to the WAN port on the firewall (since, to the Linksys, the initial communication came from there) rather than the internal host that had initiated the conversation. When the WAN port would attempt to reply, the default block rule blocked it and it was logged. That's why it continued to block and log it even after I set a rule, as a test, allowing the WAN port to communicate externally to any IP using any protocol.

    To fix it, I reflashed the router using DD-WRT which allowed me to set it up as a true dumb switch, which also allows wireless clients to connect and seem to the rest of the networks like they are simply hosts physically connected to one of the physical ports. Now the router does no NAT, the WAN port on the router is disabled and everything is working as it should. Thanks again to marvosa for the time and assistance.