I just wanted to give an update regarding achieving full throughput on 2.7 CE especially given the recent pfsense plus licensing debacle:
I was able to attain the full 23.5 Gbps throughput on 2.7 CE straight from a fresh install and the aforementioned enabling of the hardware offloads by enabling SR-IOV on the Proxmox host and passing into pfSense the virtual functions (virtual nics). In this situation, pfSense uses the iavf driver which is included in CE and precludes the need for if_ice.ko and ice_ddp.ko.
On a related note: I was able to hit 31 Gbps on pfSense through an e810-cam2 (which uses the same driver setup as the e823. Though I've only just started playing with this 100GbE nic, so 31 is the starting point.