I think I figured out the errors reported on my switch. I enabled EEE on my switch and it seems my Intel i350 is the only NIC that actually supports EEE. There seems to be a correlation between my error count incrementing and the ports being idle. This would explain why I saw a few errors on my ports shortly after restarting PFSense from the upgrade, no traffic.
Recently, my ISP did an upgrade late at night, and my switched showed the ports going up an down a few times because they turn off when no traffic and an EEE device is plugged in. The next morning, I saw 2 more errors.
I can't get a causational link, but it seems highly correlated. Even after 9 days of uptime, I only have 5 total errors and they were only spotted shortly after something would have caused WAN traffic to cease.