Problem solved.  Proxy configuration.  Since I'm running on an APU4, I'm running nanobsd.  Since I'm running nanobsd, I have Hard Disk Cache set to Null (i.e. no disk cache).  I had values in some of the variables specifically dealing with disk cache.  I figured it was a moot point because my Hard Disk Cache was set to Null.  It looks like these values were causing Squid to think there was disk cache, and when it tried to retrieve it, nothing was there and I got the error.  I changed the settings such that Squid would only ever look to memory for cache et voila!  It works great.

@DemoNIck: Any packet loss appearing on the graphs is ONLY due to p2p traffic. Now I am almost certain that this "packet loss" has nothing to do with the pfSense system itself Excellent. I suspected this was the case. Now some simple traffic shaping ought to tidy things up so that your high priority traffic isn't impacted. @DemoNIck: But before we do so could you be so kind and be more specific on the "WAN saturation" indication. I apologize for not writing more clearly. I have a bad habit of thinking faster than I type.  ;) By "WAN saturation" I meant the outbound bandwidth utilization on the WAN link reaching or near 100% [due to p2p traffic]. It has been my experience that retail WAN technology (POTS, DSL, Cable, etc.) is subject to extreme performance degradation (i.e. packet loss, etc.) under high bandwidth utilization conditions. When sustained bandwidth utilization exceeds 80% then smaller traffic bursts start hitting the "artificial" bandwidth limit imposed the Internet service provider (ISP). When traffic hits that artificial limit, the ISP begin to "rate limit" the traffic. Often their rate limiting is very "brute force" by simply discarding packets thus creating all sorts of timeouts and retransmits. Although you never specifically mentioned the bandwidth of your connections, I know from experience that most Internet connections are asynchronous and so are highly sensitive to outbound traffic (i.e. p2p). Your WAN traffic graph shows periods of intense outbound bandwidth utilization with "wan-out-pass" reaching 742kb/s which is at the limit of the most common outbound speed (768kbps) so WAN outbound bandwidth utilization became the primary suspect especially since the rest of the graphs have values in reasonable ranges. @DemoNIck: Once again thank you in advance for your time and effort. It's been my pleasure. Thank you for creating an interesting topic.

@Derelict: There is an easy way.  Use a switch. Thanks. I'm extremely grateful for all of your advice. The kind people on this forum have been phenomenal.

Why such an old version?

Diagnostics->Backup/Restore ??

Using an alternative bootloader is really only necessary in a tiny proportion of hardware and usually it would be for  systems that don't boot at all. What hardware are you using? Steve

Thanks for the answers. KOM, bandwidth control can also be done by the Syno untill I can get it to work on pfsense. The only purpose of the VPN is to let users connect to the Syno for the cloud service. They will connect based on their own username and password. On the Syno I can configure guaranteed and maximum bandwith per user. That will give me some time to figure it out on pfsense.

@duncane: You might try as well, It might as well be a bug of a policy applied uncorrectly following an upgrade of the firmware. Changing this settings seemed to have fixed that on my side. I'll give it a shot tomorrow, I just updated the firmware an hour or so ago, will see if that has changed anything tomorrow after work. If not, I'll enable the guest network and try it, then disable the guest network and try it again.

@Supermule: Flaky switch?? I'm going to try exchanging the device plugged into the DMZ this weekend (it's a USB -> Ethernet adapter for Wii U, rather than a switch). I hope that solves the issue!

Ok Thank for your reply and i will follow your step. I hope i can do this as well.  :) :) :) :)

Thanks for coming back with a result, many don't.  ;) Steve

Nice.  :) One thing to be aware of is that to add NAT rules you will have had to switch to manual outbound NAT. That means that you have to remember to add NAT rules for any interfaces you now add, new NICs, VLANs, VPNs etc. An alternative is to leave outbound NAT set to auto and add a gatway to the modem access interface. pfSense will now NAT that connection. Make sure your real WAN gateway is set as default though or your'll loose all internet access! In 2.2 there will be a hydrid NAT mode where rules are auto generated but manual rules can be added. That will negate this issue. Steve

Thank you very much. In 2.1.4 you would sove it by using the guii Diagnostics -> NanoBSD ->  Media Read/Write Status  -> Permament -> Save. SOLVED

also, how do i reset the net.inet.ip.intr_queue_drops counter?

You've probably already worked this out, but squid + sarg will tell you by IP address (not user). But, you can assign IP addresses to specific MAC addresses with DHCP. Captive Portal by default links the usernames to the MAC and IP addresses… so the squid report should work for you. As for storing the data off the firewall I would use rsyslogd or failing that look at options for a network mount (NFS or SMB/CIFS). https://doc.pfsense.org/index.php/Copying_Logs_to_a_Remote_Host_with_Syslog

I think you are asking for help in determining why pfSense is blocking the radius packets? Can you sketch out a topology? i.e. WIFI CLIENTS <–> [ARUBA 7200      ] <–> [LAN  pfSense  WAN] <–> Internet                   Radius client &        RADIUS service &                   Captive Portal          User database