Okay, so this issue could be due to VirtualBox VM connectivity issue, but I had someone at work complain Friday evening and Saturday that they would get kicked off after about 15 minutes using their pc. I tried to test things on "PC" since I use macbook to connect via VPN and have never had a problem. I tested it on another pc at home and was on for almost 40 minutes with no problems, so I'm chalking up to VirtualBox and maybe that I needed to reboot the pfSense system. Other than that one person I have not heard of anymore problems, but I made the last update Friday evening and not many people have got on since then. If no one else is having this problem, then it's probably my setup.
Thanks!