Must be lot of fun with doing maths and waiting where it overflows… but - the circular log is not suitable for archiving purpose, at all. Use a remote syslog server, or at least install the syslog-ng package and log to normal logs, rotating them as needed.

Yes, reinstall is the only way to fix UFS. I've filed multitude of bugs about UFS and fsck. fsck is so broken that it needs multiple successive manual runs to even try to repair the filesystem, and then it gets all sort of things wrong, and segfaults, or spits out various confused nonsense, and eventually screws the filesystem to the point where you cannot boot any more. I got the below patch from one of the pfSense devs for debugging, and while it tries to run fsck much aggressively, as noted above, the only result in the end was complete FS destruction. Also, it would need updating for 2.3.2 or newer, apparently. diff --git a/src/etc/rc b/src/etc/rc index e82a5ba..970fa9c 100755 --- a/src/etc/rc +++ b/src/etc/rc @@ -54,7 +54,7 @@ fi if [ -e /root/force_fsck ]; then echo "Forcing filesystem(s) check..." - /sbin/fsck -y -F -t ufs + /sbin/fsck -y fi if [ "${PLATFORM}" != "cdrom" ]; then @@ -77,18 +77,37 @@ if [ "${PLATFORM}" != "cdrom" ]; then if [ ${FSCK_ACTION_NEEDED} = 1 ]; then echo "WARNING: Trying to recover filesystem from inconsistency..." - /sbin/fsck -yF + ntries=0 + fsck_rc=1 + until [ $ntries -ge 3 -o $fsck_rc -eq 0 ]; do + /sbin/fsck -y + fsck_rc=$? + ntries=$((ntries+1)) + echo "DEBUG: Run #${ntries} - rc = ${fsck_rc}" + sleep 1 + + # Sometimes first call returns 0 but filesystem is still broken + # Run fsck in preen mode again just to be sure + /sbin/fsck -p -F + fsck_rc=$? + echo "DEBUG: (-p) #${ntries} - rc = ${fsck_rc}" + sleep 1 + done + + if [ $fsck_rc -ne 0 ]; then + echo "Automatic filesystem recovery failed. Starting recovery shell!" + tcsh + reboot + fi fi /sbin/mount -a 2>/dev/null - mount_rc=$? - attempts=0 - while [ ${mount_rc} -ne 0 -a ${attempts} -lt 3 ]; do - /sbin/fsck -yF - /sbin/mount -a 2>/dev/null - mount_rc=$? - attempts=$((attempts+1)) - done + + if [ $? -ne 0 ]; then + echo "Filesystems could not be mounted. Starting recovery shell!" + tcsh + reboot + fi if [ "${PLATFORM}" = "nanobsd" ]; then # XXX This script does need all filesystems rw!!!!

@KOM: You were either using NAT reflection, or you had your internal DNS handing out LAN IPs (known as split DNS). thank you for your answer, i managed to get this fixed using a internal split brain DNS. much appreciate it your support.

Took PFSense out of the equation, and things did improve a bit. I've migrated from using the RG in router mode to pass-through with the WAN address MAC now being assigned the RG's IP via DHCP. So that much is working.  Throughput is still not where it ought to be.  So next step is to possibly replace my TPLink 24-port Gigabit switch with a Cisco 24-port to see if that helps. Though all of this created a new problem with my OpenVPN install no longer working, even after I changed its IP and created new certs for OpenVPN Connect.  Posted in the OpenVPN section on issues I've got there now…OpenVPN Connect will authenticate to the firewall, but then has access to nothing.

If you're pinging raw IP addresses then pfSense has nothing to do with problem unless (very unlikely though and you aren't providing enough information) there is an IP address conflict in your network and pfSense is somehow the cause of the conflict. If you're pinging DNS names then you need to look at the DNS forwarder/resolver logs on pfSense for signs of anything amiss.

Thanks,  doktornotor @johnpoz: Yes, normally you would do that at the switch however I have an unmanaged switch. I was planning to put a TAP switch but the fact that I have three NIC's already on the firewall box and pfsense has spanning capabilities, I was thinking of going that route.

Looking more closely at everything, it almost seems like you HAVE to have NAT enabled on the pfsense machine in order to direct specific addresses through the VPN or not.

As mentioned above, from the WAN side it will be pretty safe out of the box. My first step even before I connect it to a WAN or LAN would be to change the default password though.

https://doc.pfsense.org/index.php/Firewall_Rule_Basics

1 more thing, this is from the cable modem log: Mon Feb 06 14:59:54 2017 Critical (3) Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out So the cable modem had a hiccup with the comcast signal, and when the gateway (wan) handled the change, one interface (lan) was able to cope with it, but one (opt1) was not.  I fear I'm out of my league here as they're setup the same.

@webtyro: Bad pun I know. I have one lan called "home" and another called "untrusted". I know you can guess where the visiting Microsoft units are put. Hard to trust them. I run all Debian for my own use. Microsoft had a good run but I feel they have gone rogue. Post update if it shows again. Hope not though. I've seen a quality drop from nearly every big tech company there is.

Perhaps try FreeRADIUS.

Here's my network details if it helps: Network Diagram: http://prntscr.com/e58gvo Switch network config: http://prntscr.com/e58cfj Switch VLAN settings: http://prntscr.com/e58cs4 Firewall LAN interface: http://prntscr.com/e58d8y Firewall VLAN3 interface: http://prntscr.com/e58dpp LAN DHCP settings: http://prntscr.com/e58e2b VLAN3 DHCP settings: http://prntscr.com/e58eau Relevant LAN rules: http://prntscr.com/e58eth VLAN3 rules: http://prntscr.com/e58f2p