Alright.  Looks like removing default gateway from the WAN interface, then, going command line and running: route add -net [network] [gateway] Fixes the issue.  Now able to access via devices on the sophos LAN, and it has internet connectivity. Tried advertising a default route from the WAN pfSense, instead of setting static via command line, but for whatever reason, it didn't want to work.  Even though the OSPF routing showed a 0.0.0.0/0 route learned, it wouldn't use it.

Thank you very much doktornotor  :) I'll try this today.

Wonderful thing, Google: https://www.youtube.com/watch?v=O9D2fM883Fc http://hubpages.com/technology/How-to-Set-Up-a-Captive-Portal-Using-pfSense http://pfsensebuddy.weebly.com/blog/setup-pfsense-captive-portal-in-easy-steps https://doc.pfsense.org/index.php/Using_Captive_Portal_with_FreeRADIUS

Makes sense, thanks!

https://github.com/pfsense/pfsense/search?utf8=%E2%9C%93&q=edns

Hi BlueKobold, Thanks for the reply.  I've thought of similar solutions where I put another router device between the modem and pfsense computer, however, I really don't want to introduce an additional component.  I would prefer to have everything work (as it should) with just the pfsense computer and HughesNet modem. Gordon

A small switch with a mirrored port is able to get for less than $100. Netgear GS105Ev2 Netgear GS108Ev2 So if the network tap is able to get for $59, take it! Together with a laptop, because it is USB powered you could not get it better in my eyes. And together with WINDUMP and WireShark you might be able to sniff what you want and on top you could dig into the stored captured packets. The APU1D4 could be sorted with FreeBSD or a Linux version of your choice likes CentOS that came hardened by default and for Linux you may able to get many programs for free of charge and easy to install if you are often on the road.

Thanks for the quick response.  Re-installing with hdd this time instead.  Will get a chance to test try the snort restore I posted about last week too.

Might be that a L2TP/IPSec tunnel would matching right?

I've not have had an issue with this at least with 2.1.5 and 2.2.4 using AD Auth for WebGUI, Console, and OpenVPN login. My admin account has a period in between and I can authenticate just fine (tested with 2012 R2).

So I'm testing out BandwidthD and Darkstat and both look like they will work. I have DNS resolving hostnames now so this should work perfectly, thanks.

Sorry for open this topic, but is about what I want to ask… It would have been better for you to start your own thread in the Traffic Monitoring forum… Any advice to see ? Lots of ways to do it.  My favourite fast way is Status - Traffic Graph.  Set Interface to LAN and Display to Host Name.

Bump? Sorry to be double posting but does seriously no one know anything about this?

@cmb: files.pfsense.org is at NYI's New Jersey datacenter. Generally at least 900 Mbps of bandwidth available at any given time. I'm getting about 7 MB/sec from home in Austin right now via either v4 or v6, with wife watching Netflix and misc other going on. I was thinking from the start is was some problem between here(São Paulo - Brasil) and our beloved(hated) connection through Level3. I asked just to be sure. Every time the route goes through Miami I have issues with bandwidth on servers hosted on Austin or anything north of it. Not only on pfSense, Spiceworks for example are there and I get similar problems. Anyway, thanks for the feedback.

Sounds like you very well may be onto something there! Unfortunately we didn't get chance to reboot last night but what you said makes sense - last rebooted a few months ago but updated on November 4th. Thank you.

I know I am likely missing something, but I am wondering why pfSense routing speed is low compared to a my Debian system? Linux and FreeBSD are not the same OS as many peoples would imagine. And they are often not really comparable in what discipline ever you want. Linux is more smooth and liquid running and more agile function. BSD based systems are more stable and on the right hardware also smooth and liquid running, but often a little bit tuning is needed. pfSense is not a set up and forget it solution as many users might expecting. I've tried several different hardware setups and the Debian system always performs better. It is an open secret that the Linux systems are often better recognized by the hardware vendors and on top much better sorted with drivers. BSD system are mostly not so good sorted with drivers by the vendors and there fore you might be more comparing BSD against BSD based systems and Linux against Linux based systems, to come closer to the point you want. I am able to route 10GB on my Pentium G3220 Debian system regardless of size of rule set, WAN - LAN or what routing we are talking here from? Debian is using all CPU cores and FreeBSD where pfSense is based on is only using one CPU core at the WAN Port using PPPoE. So what? yet pfSense on the same hardware can't go beyond 1.4GB/s. Please read the line above. My G3220 shows 0.4% CPU usage @ 1.4Gb/s and pfSense on the same hardware is 100%. pfSense is only using one CPU core at the WAN Port and your Linux is using all cores. Even pfSense with a 6 core HT E5-2620 can't keep up with the dual core G3220 on my Debian system. I am using an Intel E3-1285v3 @3,5GHz and all is running like hell, 2 x 200 MBit/s at the WAN side and 10 GBit/s to the DMZ and LAN Switch. All is smooth and liquid running for me. Did you thew following? Enable TRIM support for mSATA or SSDs Enable PowerD (hi adaptive) High up the mbuf sizes This is what I was talking above from, in pfSense you are able to tune or fine tune many things that your system will more running likes you want and with Linux this might be not even the time. How your test was running? From where to where? 1 PC on the LAN side (192.xx.xx.) 1 PC on the WAN side (172.xx.xx) Using iperf -a -p So you might be able to see also pfSense is able to route all the things to route likes other systems are doing. Debian is not a hardened Linux OS, so I would be aware of using it likes a firewall! CentOS came from house hardened and might be a better choice for that or perhaps taking ClearOS might be a good choice for this action.

…LET'S START OK, after a few months later I'm answering my question by providing a step-by-step guide for the other curious people who are interested in this network setup. So if I should give a title for this configuration it would be "Access OpenWRT managable VLAN switch interface via pfSense router subnet a.k.a how to access an OpenWRT VLAN switch remotely from your pfsense router/firewall". First thing first, the network diagram helps a lot, so here is: [image: Network_diagram_vlan_switch_localy_managable_via.jpg] So here is a more improved version of the diagram(updated 2015-12-29): [image: Access_Open_WRT_VLAN_switch_remotely_via_pf_Sense.jpg] 1)In order to be clear for everybody who is attempting to accomplish this configuration I summarize the hardware+OS/firmware specs first: Hardware/OS: Intel Jetway NF9D-2550+pfSense router/firewall 2.1.5 installed release Hardware/firmware: ASUS RT-N16 Gigabit SOHO router+OpenWRT  14.07(codename Barrier Breaker) 2)Second, the ideea is to creating tree VLANs(VLAN 3,4,5), the first VLAN(VLAN3) is only for remote management purpose, via this VLAN you can access the VLAN switch(OpenWRT running Asus RT-N16 soho router) GUI/CLI interface remotely from one of the pfSense router/firewall subnets. The vlan switch's VLAN3 interface doesn't have physical port access on the switch, the other two VLANs(4,5 which corresponds for the internal port 3,4==external lan port label 1,2) have ports that provide network access for client devices, so if you plug you pc/laptop etc into one of these ports on the vlan switch you will get internet access. !Note:The internal port numbering(which is the VLAN switch interface) for the VLAN switch(Asus RT-N16 soho router) is in reverse order against the external port label(the 4 ethernet LAN ports on the back of the router), so internal port nr 1==external port label 4, internal port nr 2==external port label 3, and so on… The VLAN id numbering starts from 3, to avoid the possible issues, VLAN1 is the default VLAN on the pfsense router/firewall, same is true for the OpenWRT running asus soho router, additionally the WAN interface of the asus router is assigned to the VLAN2 so all those ports are set to off mode. Here are the corresponding physical interfaces for the pfSense router/firewall and the OpenWRT VLAN switch: re0 -> pfSense WAN port   re1 -> pfSense  local LAN port   re2_vlan  -> pfSense VLAN trunk port   internal switch port 1 -> OpenWRT VLAN1 port   internal switch port 2 -> OpenWRT VLAN trunk port   internal switch port 3 -> OpenWRT VLAN4 port   internal switch port 4 -> OpenWRT VLAN5 port So the pfSense router/firewall and the OpenWRT vlan switch has 4 VLANs(subnets): VLAN1 corresponds to 192.168.1.0/29 which is the default VLAN used for local management/network and internet access on the pfsense firewall   VLAN3 -> 192.168.3.0/29 for remote access  from the pfsense router to the vlan switch   VLAN4&5 -> 192.168.4.0/27&192.168.5.0/27 only for internet access   VLAN1 -> 192.168.1.8/29 default VLAN used for local management on the VLAN switch One important thing that have to be mentioned that both the pfSense and OpenWRT running hardware are capable of 802.1Q tagging/trunking, from the pfsense side the Intel Jetway motherboard is a good choose with an Realtek AD3RTLANG Gigabit Ethernet NIC, on the switch side the ASUS RT-N16 soho router has an BCM53115 chip which supports this VLAN functionality. Ok , here are the steps that have to followed: pfSense router/firewall configuration steps !Note: The configuration is completed with address resolution mechanism what means you can access your remote manageable VLAN switch via its hostname and the domain name specified in the pfSense settings, the OpenWRT hardware doesn't have hardware clock so it has to be syncronized via the pfSense NTP service, so I would like to provide here this configuration too. Here you can see the pfsense router's hostname, the domain name which is belongs to, the VLAN switch will belongs to this too, the first DNS server is the local DNS server address which is important because queries related to the local network will be served by the dnsmasq, thus you can access you pfsense router and VLAN switch by their hostnames, here is defined the NTP server address used for time syncronization too. [image: firewall_4.jpg] Enable the local DNS server in order to be able to resolve DNS queries comming from your network destined for your local network. The VLAN switch's static IP address will be registered in the DNS forwarder. If the DNS queries are destined for the local network then it will be resolved locally by the first DNS server. [image: firewall_7.jpg] Settings for the local LAN interface on the firewall with static IP. [image: firewall_10.jpg] Specifying the VLANIDs, in this case 3, 4, 5 and assign the physical(parent) interface for the VLANs(trunk port for the VLANs on the pfSense side)… [image: firewall_12.jpg] …do this for all the 3 VLANs. [image: firewall_11.jpg] Assign the network port(physical interface) to the virtual interfaces (RLANVID3, VLANVID3, VLANVID5, this is the trunk port which pass throught the multiple VLANs on the pfsense side). [image: firewall_13.jpg] Enable the RLANVID3 interface, edit description to RLANVID3 (by default is OPTx)… [image: firewall_14.jpg] …same for VLANVID4... [image: firewall_15.jpg] …same for VLANVID5. [image: firewall_16.jpg] Enable DHCP server on the LAN interface and define DHCP pool for the local LAN on the pfsense router… [image: firewall_18a.jpg] Statically allocated IP addresses for the management client devices, here we have two, a desktop pc and a netbook. [image: firewall_18b.jpg] Enable DHCP server on the RLANVID3 interface and define the DHCP range for this, using "Deny unknown clients" option which avoids to assign IP address for any other devices, from the DHCP pool leave out the statically allocated IP address for the VLAN switch(in this case is 192.168.3.2 and specify it in the "DHCP static mapping for this interface" section) . [image: firewall_19.jpg] The VLAN switch IP address is a statically allocated DHCP IP address because the switch is fixed part of the network, so every time the switch sends a DHCP request, it gets the same IP assigned to its MAC address, so the switch's hostname will be resolved to this IP address. [image: firewall_20.jpg] …enable DHCP server on the VLANVID4 interface, similar to RLANVID3 interface... [image: firewall_21.jpg] …same for VLANVID5 interface. [image: firewall_22.jpg] Allow Network Time Protocol Service on the RLANVID3(VLAN3) interface, from the VLAN3 subnet comes the NTP request (initiated by the VLAN switch's NTP client). [image: firewall_31.jpg] Here you can specify some aliases(placeholders) for a group of IPs, these are the management hosts in the local LAN subnet which can access the router's interface, also group the accessible LAN/VLAN interface IPs,(these are the accessible management GUI/CLI interface IPs) … [image: firewall_23.jpg] …and you can create aliases for the group of ports for the management host, and ports for the clients which are not able to access the management interfaces(Router_interfaces==pfsense router interfaces+OpenWRT vlan switch interface, see previous image). [image: firewall_24.jpg] Firewall rules for the local LAN interface: 1st rule - Allowing DNS queries from the local LAN subnet   2nd rule - Allowing ping request from the local LAN subnet(ICMP messages) !Note:Here we are using these first two rules to see if the interface is alive for any type of client device(by allowing DNS, ping and implicit DHCP services) 3rd rule - Allowing only for Management_hosts(specified in the alias section) to access the Router_interfaces(pfsense router interfaces+OpenWRT VLAN switch interface IPs) only via Management_ports, so if a client device is not part of the Management_hosts alias on the local LAN subnet and trying to access the Router_interfaces on the port that is different from the Management_ports alias list then…   4th rule - ...it will be rejected by the firewall   5th rule - Allowing all client devices(those what are and aren't Management_hosts) to HTTP/S access(only webpage access), the rules are evaluated on the first match basis so the previous two rules treats all of the client devices(hosts that are part and aren't part of the Management_host alias list) to gain or reject access to the Router_interfaces, so this rule allow access for all of the client devices who want to reach the internet regardless of what type of hosts they are(Management_hosts or not).   6th rule - Rejecting any other type of communication, this treats communication between client and client devices, for ex., if you hook up two pcs on the local LAN interface(with an intermediate simple switch) they can't ping each other, you can't connect from one device to another remotely etc. [image: firewall_26.jpg] Firewall rules for the RLANVID3(VLAN3 subnet) interface: 1st - Allowing DNS queries from the VLAN3 subnet(important for the NTP request what is comming from the NTP client used by the VLAN switch)   2nd - Allowing NTP requests from the VLAN switch's NTP client   3rd - Rejecting any other type of communication (for ex. Management_host or other client devices to reach the GUI via 192.168.3.1 IP address destined from the 192.168.1.1 subnet) [image: firewall_27.jpg] Firewall rules for the VLANVID4(VLAN4 subnet) interface: 1st - Allowing DNS queries from the VLAN4 subnet   2nd - Allowing HTTP/S access to webpages except the Router_interfaces IPs for all clients, the main difference in contrast to local LAN interface rule that here you don't want any type of client device to reach the Router_interfaces thus…   3rd - ...it will fall into this rule, it will be rejected by the firewall(not blocked thus reducing network load), plus communications between client and client devices will be rejected too [image: firewall_28.jpg] Firewall rules for the VLANVID5(VLAN5 subnet) interface: -same rules are applicable for VLAN5 subnet [image: firewall_29.jpg] OpenWRT VLAN switch configuration steps !IMPORTANT NOTE: The VLAN switch(ASUS RT-N16) configuration has to be done in the sequence listed here, completely unplugged from the pfSense router hardware, after the network interface confugration for the local LAN(VLAN1) is done(5th image about the VLAN switch configuration, vlanswitch_11.jpg) you can plug the ethernet cable which connects the pfSense router/firewall and the VLAN switch (the trunk link between the two network devices), otherwise the VLAN switch stoppes operation and have to be reset all the setting and start the whole configuration from the scratch. Here you can see the hostname of the VLAN switch, the time zone and here specify the NTP server address of the router/firewall, in this case it is the pfSense's router NTP server address. [image: vlanswitch_4.jpg] Here the VLAN1 is the switch's default VLAN, all the 4 internal LAN ports are part of this VLAN, so you has to leave only one port(internal port 1==external port label 4) for the VLAN1, so you can access the router management interface from the VLAN switch's local LAN(VLAN1 represents 192.168.1.8/29 local subnet) via this port. The ports assigned for the VLAN2 subnet is disable, don't have to use it because VLAN2 subnet is assigned for the WAN port, here the Asus router acts as a VLAN switch so don't need WAN interface, the other reason that VLANID numbering starts from 3 to avoid VLAN configuration and operation issues on the VLAN switch side. Here you specify the VLANIDs, in this case 3, 4, 5 same as on pfSense router/firewall and assign the physical(port) interface for the VLANs(internal port 2==external port label 3, will be the trunk port for the VLANs on the VLAN switch side), this port pass through the multiple VLANs on the switch side. Internal port 3(external port label 2) is assigned to VLAN4, and internal port 4(external port label 1) to VLAN5, these VLANs(4,5) has physical port access on the VLAN switch, set to untagged mode both in terms of their corresponding VLANIDs. So if you connect a pc/laptop into internal port 3 the device will be assigned to VLAN4 and for internal port 4 for VLAN5, both are be able to access the internet. !Note: One important thing, both the local LAN subnet(VLAN1) and the RLANVID3 subnet(VLAN3) has to use the CPU port in tagged mode, thus packets comming from locally into the VLAN1 subnet or from remotely into the RLANVID3(VLAN3) subnet reach the CPU port which is an internal port, this ensures that to display the VLAN switch's GUI/CLI in your browser, so if packets won't reach the CPU port you are not able to access the GUI/CLI interface on the VLAN switch. [image: vlanswitch_7.jpg] Here you can see the two created virtual interfaces, RLANVID3 for VLAN3 subnet for remote access, and LAN for the local LAN, for local access. There was a WAN interface which you can delete it optionally because it is useless such as the VLAN2 VLANID in the switch configuration section. [image: vlanswitch_8.jpg] Detailed configuration options for the local LAN interface(VLAN1 subnet): It has static IP address(VLAN1->192.168.1.9), netmask and broadcast address. [image: vlanswitch_9.jpg] Here the important thing is to tick the "Bring up on boot" option so when the switch is plugged in the interface will be accessible, this is the local LAN so it has a local DHCP server(that is why "Disable DHCP for this interface" box is left unticked), where the usable IP address range starts from 192.168.1.10 and the limit is 5, and their lease time is 24 minutes but you can adjust these setting if you want. [image: vlanswitch_10.jpg] The local LAN(VLAN1 subnet) interface is unbridged from the switch's interface, the physical interface is eth0.1 where .1 represents the LAN(VLAN1) interface on the switch, which is assigned to the LAN interface, from now on the two network devices can be connected via an ethernet cable to pass VLAN traffic from pfSense router to VLAN switch and vice versa. [image: vlanswitch_11.jpg] Here you can see the created firewall zone for the local LAN(VLAN1) interface. [image: vlanswitch_12.jpg] Detailed configuration options for the RLANVID3 interface(VLAN3 subnet): It acts as DHCP client sending the DHCP request along with its hostname, in this case is "vlanswitch" to the 192.168.3.1 address on the pfSense router, and the router assign the 192.168.3.2 IP address knowing that the device with the unique MAC address and hostname sended by the DHCP client has a static ARP entry in the "DHCP static mapping for this interface" section, which is correspond to this IP address. [image: vlanswitch_13.jpg] Here the important thing is to tick the "Bring up on boot" option, "Use default gateway" and "Use DNS server advertised by peer" option, thus the pfsense router serves its gateway address and DNS server which is in the RLANVID3(VLAN3) subnet (the gateway and DNS server IP is 192.168.3.1). [image: vlanswitch_14.jpg] The  RLANVID3(VLAN3 subnet) interface is unbridged from the switch's interface, the physical interface is eth0.3 where .3 represents the VLAN3 interface, which is assigned to the RLANVID3 interface. [image: vlanswitch_15.jpg] Here you can see the created firewall zone for the local RLANVID3(VLAN3) interface. [image: vlanswitch_16.jpg] Here you can see the general firewall zone settings for the VLAN switch, one for the local LAN(VLAN1) and the other for the RLANVID3(remote VLAN3), this is a Linux firewall so you can control network traffic(restrict access to the VLAN switch) via the built-in chains, here the only important chain is the input chain. You want to restrict incomming access to the VLAN switch so you reject any incomming traffic and define what type of network traffic allow to pass the input chain, in other words you filter what type of traffic are able to reach the VLAN switch interface from the LAN(VLAN1) subnet and from the RLANVID3(VLAN3) subnet which is comming from the pfSense router/firewall. [image: vlanswitch_17.jpg] Firewall rules for the local LAN(VLAN1 subnet) interface: 1st - Allowing DHCP requests from the local LAN subnet   2nd - Allowing ping request from the local LAN subnet(ICMP messages) !Note:I'm using these first two rules to see if the interface is alive for any type of client device(here I omited the DNS, have to include the DHCP service)   3rd - Allowing client devices to access the GUI via HTTPS protocol from a specified MAC address(this is the desktop PC used for management purpose) [image: vlanswitch_18.jpg] 4th - Allowing client devices to access the GUI via HTTPS protocol from a specified MAC address(this is the Netbook used for management purpose)   5th - Allowing client devices to access the CLI via virtual terminal (SSH protocol) from a specified MAC address(this is the desktop PC used for management purpose)   6th - Allowing client devices to access the CLI via virtual terminal (SSH protocol) from a specified MAC address(this is the Netbook used for management purpose) [image: vlanswitch_19.jpg] 7th - Allowing incomming DHCP response from the pfense router/firewall MAC address from the RLANVID3(VLAN3) subnet   8th - Allowing incomming NTP response from the pfense router/firewall MAC address from the RLANVID3(VLAN3) subnet   9th - Allowing incomming DNS response from the pfense router/firewall MAC address from the RLANVID3(VLAN3) subnet !Note: these tree rules ensures that the responses coming from one of the pfsense router/firewall port and only from one single port which has the corresponding MAC address. [image: vlanswitch_20.jpg] 10th - Allowing ping request from the RLANVID3(VLAN3) subnet from the pfSense router/firewall !Note: Rule number  7 and 10 is used to to see if the RLANVID3(VLAN3 )interface is alive from the pfsense router/firewall side   11th - Allowing HTTPS request from the RLANVID3(VLAN3) subnet from the pfSense router/firewall(thus the VLAN switch is accessible via the GUI)   12th - Allow virtual terminal from the RLANVID3(VLAN3) subnet from the pfSense router/firewall(thus the VLAN switch is accessible via the CLI) [image: vlanswitch_21.jpg] The VLAN switch is accessed remotely from the pfSense local LAN subnet, here you can see the results: The physical eth0.3 interface(RLANVID3==VLAN3) settings, the important thing is the default gateway and the advertised DNS server address which appears for the interface… [image: vlanswitch_2.jpg] …So the VLAN switch responds to ping request via DNS resolution... [image: vlanswitch_0.jpg] …and the VLAN switch GUI is accessible via DNS resolution too, same is true for the SSH. [image: vlanswitch_1.jpg] [image: vlanswitch_1b.jpg]

I think there is more to the gui hang than dns I also get a mail flood that seems to happen and am sure that contributes to the gui hang cause if i reset gui from console flood stops and gui becomes accessible