Internal network routing with a public interface/IP address

[MakOwner]

Simple diagram of my net work. I have two ISP accounts one in each location.
The two locations are connected by a private 10GB link.
I am in the process of setting up an internet facing server in one location, using one of the public facing static IP addresses in the /29 provided by that ISP. (I need the host on a public IP not relevant to this routing issue.)
At present, I can access any system in any wired 192.168.0.0 address space behind either pfsense from any other. To do this I set up static routes on each pfsense box to routes.
192.168.10.0 routes to 192.168.20.0 via 192.168.30.0 interfaces on each pfsense.

Works fine, although DNS is slow.
(Rather than manually duplicate the resolver space I just have each client use the local pfsense as first DNS server, the other pfsense as second. If a resolution isn't found locally you can detect a sizable delay as the query fails and gets redirected.)

When I set up the new server, everything works as expected. So long as the public interface is down, routing works fine. Access from both internal networks is fine.
I bring up the public interface and I can the only access the server from the local private network (And the public IP of course).
I have included a somewhat simple diagram of the setup.

I have set the IP adress on the server internally both via dhcp and statically and tried adding a route to the 10 and 30 network, but nothing seems to be working.

the route table for internal networking (route -n) doesn't appear to change when bringing up the public interface, but access to anything other than the .20 network immediatetly drops.

I am missing something here blindingly easy, I know it ...

Edit:
I have setup a bare linux VM, using the simplest network configuration I can.

dhcp from the local pfsense works exactly as expected -- so long as there are no other interfaces.

I set both interfaces in the affected VM like this to play around with the client side routing:

The primary/first nic configured network interface
auto eth0
iface eth0 inet static
address 172.nnn.nnn.nn/29
gateway 172.nnn.nnn.54

auto eth1
iface eth1 inet static
address 192.168.20.13/24
gateway 192.168.20.1

Adding this route gets a ping response from 10.1 but no other IPs on that subnet and doesn't allow inbound traffic from any host on 192.168.10.0/24.

route add -net 192.168.10.0/24 gw 192.168.20.1 eth1

Downing the eth0 interface, and things work as expected.

Since each pfsense has the routing table to direct traffic between the subnets, why -- so long as the traffic makes it to pfsense - is the traffic being dropped?

Would changing the static public IP address to the second interface change this behavior?
I'm assuming doing that would cause all outbound traffic to run across the pfsense gateway, but I need this server to send all internet bound/sourced traffic over the public IP.

[viragomann]

@makowner
Possibly the server sends packets out to the internal interface with its public source IP?
Sniff the traffic to check that out.