For anyone looking at this in the future:
Looks like I figured it out (after 3 hours last night, and 4 hours tonight using the backup xml config files).
We had the MAC address spoof/cloning turned on. Looks like that isn't working properly since when I turned it off, everything works like a charm (no static ports needed).
I'm fairly sure that MAC address cloning was not a problem with any of my previous routers, so maybe this is a pfSense bug.
How I trouble shot this:
After trying the suggestions that were supposed to work (ie: static ports), I gave up and nuked the router and reset it to factory. I checked this worked. It did. I then downloaded the backup config xml file and compared it to my config that had problems.
Slowly I tried adding things from my bad config to the simple config until I got a problem. Eventually I found the change that caused problems was the MAC Address cloning/spoofing.
Enjoy!
Keywords: Gaming, Games, pfSense, MAC Address spoof spoofing cloning