Sorry for the late reply, thank you very much for helping! In the end, it ended up being NAT reflection on the port forward being set to default instead of enabled. For whatever reason I assumed that this was on by default, I'll RTFM next time! After enabling that, I can now connect to LAN2 properly through LAN1 using the external WAN2 IP!

@derelict Ahh.. OK. Thank you! That was exactly what I was looking for! /Raj

Yes, it's possible, but I would not do it like that. I would put each pfSense on its own transit network, such as 10.1.10.0/30 for the link to the top pfSense and 10.0.10.4/30 for the link to the lower pfSense. You can keep them on the same network like they are if you want to, say, enable an OSPF area containing all three routers so they all know where to send the traffic without relying on hairpinning, ICMP redirects and other nastiness. Or maintain static routing tables pointing everything where it needs to go.

@derelict Yeah that was a typo. After some digging, https://developers.google.com/speed/public-dns/faq : "Google Public DNS is a validating, security-aware resolver. All responses from DNSSEC signed zones are validated unless clients explicitly set the CD flag in DNS requests to disable the validation." OpenDNS does not indeed. So I moved to some of the verisign servers that do (according to https://wiki.ipfire.org/dns/public-servers). So far no issues.

Those are fine. The rules on LAN sourcing from ESXi and the rules on ESXi sourcing from LAN don't make any sense but shouldn't be blocking the traffic. Based on that though you should probably take a look at these: https://doc.pfsense.org/index.php/Firewall_Rule_Basics https://doc.pfsense.org/index.php/Firewall_Rule_Troubleshooting

Pretty obscure checkbox to have been checked. Glad you found it.

I would return that switch if v2, they have firmware for v3 that is supposed to fix the vlan.  But v2 model you can not remove vlan1 from every port so its no better than a dumb switch. That is not a layer 3 switch, so you would do 2 vlans and pfsense would route between them. your vlans are tagged on the port connected to pfsense, and untagged to your PCs.

@viragomann: Add a second phase 2 for that site to the IPSec configuration. On pfSense: Local Network: 192.168.120.1/24 Remote Network: The network you want to route over the vpn And also on the Mikrotik with inverted values. thanks for your reply. very unexpected sollution, but it works

both sides pfsense?

See my recent post 'Netgear LB2120 as WAN failover'. It explains how I got the LB2120 working as my failover WAN. The firmware update for the Netgear LB series corrects the bridge issue. You can upgrade it directly from the LB2120's web administration. On the LB2120 itself, you'll need to have it connect to the wireless APN. This means that it is always on but that's what you need for a failover. You can limit the amount of pinging that the pfSense does to limit bandwidth usage under the pfSense Gateway Advanced section. I doubt Netgear would recommend anything but their antenna (Netgear 6000450 MIMO). If you go to an online LTE / antenna store, they might be able to provide you details on maximum cable length. That being said, unless you're carrier has exceptionally bad reception, or you're in a building that has shielding qualities, the antenna should be fine. Just using the internal antenna's I've got two bars or 3 bars, located 15 feet from an exterior wall.

As Derelict says pfsense has really nothing to do with this - it would all be at your switch setup.  Layer 3 switch with /30 would be way to go - but your /26 is not going to allow for that. Why would your users be setting static IPs on their routers that could conflict when your just going to hand them their IP via dhcp.. If you do not have a single switch that can handle all the ports, prob want to break your /26 into say 2 /27 and use 2 48 port switches for each half, etc.  or a 48 and 24… There are much better switches than the unifi ones with much better feature sets at same sort of price point.. But if your worried about isolation of the customers you would have to check to see if it does private vlans, etc.

Hi Nog, Tried all the above and no better I'm afraid! If anyone has any further suggestions please do let me know Thanks!

Post Interfaces > Assignments Select ONE of the interfaces that isn't working and post its interface configuration and its Firewall > Rules Post Firewall > NAT, Outbound

UPDATE: this is probably not going to be AirVPN as my 50/50 (without VPN) connection yields a 0.5/0.2 (with AirVPN)…

SOLVED, I migrated to NethServer. Thanks for nothing!

You probably want to look at this: https://www.netgate.com/docs/pfsense/solutions/xg-7100/switch-overview.html In particular, you want to take some of the ports off of VLAN 4091 (LAN) and put them on separate VLANs tagged through the uplinks to newly-assigned pfSense VLAN interfaces. You likely don't want 1:1 NAT because you can only 1:1 NAT one address to one other (hence why it is called 1:1). What you can do is create Outbound NAT rules so each subnet egresses from a different address. Port forward inbound are controlled by which address the outside clients are told to connect to. Any outside address can be forwarded to any inside address.

Unfortunately, there is no way to put this in the middle of the wan's as I do not have another pair of fiber between buildings locations. I may just go with using as a fail-over for location two.

@duren: The simplest solution would be to stop using the modem as a router, add a switch by the modem, wire the modem to the wan port of pfsense and wire back a lan port to the switch. Another alternative depending on the flexibility of the DHCP server of the modem is to tell it to give the pfsense box as the gateway and DNS for clients. They will all go through that, of course this assumes the pfsense box is wired as lan to the modem and it's DHCP is off. Given the physical constraints, the second option sounds much more promising. This would of course mean that the WAN and LAN are directly on the same physical system, and that the hosts should all treat pfSense as their primary gateway, yes? I can turn DHCP off entirely on the Modem, so this may work. I'll have to poke at it and see how it behaves. I'm unsure if pfSense will allow me to use the same network segment on multiple interfaces (WAN, LAN, etc.). If so this should be fine, and would allow all the clients to resolve to each other as if they were all physically in the same segment, including the virtual systems.