After suggestion of rawtaz on IRC I've done some packet capture on the pfSense host on both LAN and WAN and I found that when Chrome's ERR_CONNECTION_REFUSED is shown the reply seems to come directly from the upstream web server (or at least from the upstream modem), as a TCP RST,ACK packet is sent back to the client.
This is the packet. 192.168.1.3 is pfSense's WAN, 13.81.48.58 is the webserver. The Vodafone mac is the modem.
Frame 17201: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) Encapsulation type: Ethernet (1) Arrival Time: Mar 30, 2017 13:51:58.816331000 CEST [time] Epoch Time: 1490874718.816331000 seconds [time] [time] [time] Frame Number: 17201 Frame Length: 60 bytes (480 bits) Capture Length: 60 bytes (480 bits) [Frame is marked: False] [Frame is ignored: False] [Protocols in frame: eth:ethertype:ip:tcp] [Coloring Rule Name: TCP RST] [Coloring Rule String: tcp.flags.reset eq 1] Ethernet II, Src: Vodafone_bb:98:40 (64:59:f8:bb:98:40), Dst: RealtekU_16:64:d2 (52:54:00:16:64:d2) Internet Protocol Version 4, Src: 13.81.48.58, Dst: 192.168.1.3 Transmission Control Protocol, Src Port: 80, Dst Port: 39161, Seq: 1, Ack: 1, Len: 0 Source Port: 80 Destination Port: 39161 [Stream index: 108] [TCP Segment Len: 0] Sequence number: 1 (relative sequence number) Acknowledgment number: 1 (relative ack number) Header Length: 20 bytes Flags: 0x014 (RST, ACK) 000\. .... .... = Reserved: Not set ...0 .... .... = Nonce: Not set .... 0... .... = Congestion Window Reduced (CWR): Not set .... .0.. .... = ECN-Echo: Not set .... ..0\. .... = Urgent: Not set .... ...1 .... = Acknowledgment: Set .... .... 0... = Push: Not set .... .... .1.. = Reset: Set .... .... ..0\. = Syn: Not set .... .... ...0 = Fin: Not set [TCP Flags: ·······A·R··] Window size value: 0 [Calculated window size: 0] [Window size scaling factor: -1 (unknown)] Checksum: 0x220a [unverified] [Checksum Status: Unverified] Urgent pointer: 0 [SEQ/ACK analysis] This is weird because if I connect directly to the modem the error doesn't happen.[/time][/time][/time][/time]