Yes, I would agree with drewy and athurdent. When you install bare metal on atom you can use the hardware offload options on your NICs and significantly reduce cpu overhead. When installing pfsense in a VM it's now using virtual nics and all of the networking has to be processed by the cpu. Atom doesn't support VT-d or you might be able to pass the nics directly to your pfsense vm, and gain the benefits of hardware offload.
From the VMware ESXi guide:
In a native environment, CPU utilization plays a significant role in network throughput. To process higher levels of throughput, more CPU resources are needed. The effect of CPU resource availability on the network throughput of virtualized applications is even more significant. Because insufficient CPU resources will limit maximum throughput, it is important to monitor the CPU utilization of high-throughput workloads.
Use separate virtual switches, each connected to its own physical network adapter, to avoid contention between the VMkernel and virtual machines, especially virtual machines running heavy networking workloads.
To establish a network connection between two virtual machines that reside on the same ESXi system, connect both virtual machines to the same virtual switch. If the virtual machines are connected to different virtual switches, traffic will go through wire and incur unnecessary CPU and network overhead.
www.vmware.com/pdf/Perf_Best_Practices_vSphere5.0.pdf
I think when you install bare metal and test the same workload you might see half the cpu utilization. Given the 90% number you saw on the ESXi box itself it sounds like all of the cpu is being used up. 70% is used by pfsense, and the other 20% being used by the hypervisor to process the virtual switch and actually send the data out on the network. (Just a guess)
