OK, so I figured this one out a while back and haven't made it back on the forums 'till now.
This was caused by the fact that pfSense tried to assign the interfaces itself after I made so many changes, and ended up putting Opt2 - Opt5 (renamed as VLAN10, VLAN20, VLAN30, and VLAN40) all on em1.
This was resolved by me dropping down the interface assignment from em1 and putting it on LAGG0. Easy fix. Very nice.
Everything is working now, plus I have my AP on a VLAN20 port on the switch now instead of its own interface, which was kind of the point of giving this a try. Works well.