The part that is missing is the outbound NAT. The Factorio server is a client to the factorio pingpong servers that are used for NAT punching(1). The source ports when talking to these pingpong servers must not be mangled, so an outbound NAT rules is needed to prevent this (PFSense mangles ports by default). Just got all this working today.
Firewall/NAT/Outbound:
Outbound NAT Mode: Hybrid Outbound
Add this mapping:
Interface: WAN
Source: <internal address="" of="" your="" server="">Source Port: udp/34197
Destination: *
Destination Port: udp/*
NAT Port: *
Static Port: YES
(1) https://www.factorio.com/blog/post/fff-143</internal>