As the new gateway is now at its final destination and doesn't need this kind of hack anymore, I guess I can close this post.

@pfuzer pfsense with pfblockergng-dev and suricata

If your trying to get forwarded back in from your wan IP, you would have to setup nat reflection for that to function. But if the server is local, why would you not locally resolve the fqdn to the local IP and just access it without going through the nat reflection nonsense.

Well, the part with 2 LANs and 2 WANs is quite easy. You configure the transit network interface as defined by your second ISP. You configure e.g. 129.x.?.1/24 as a static IP on your "Public LAN". You either set the NAT mode to "Manual Outbound NAT rule generation." and set all NAT rules manually, or you set it to "Hybrid Outbound NAT rule generation" and manually add a "Do not NAT" rule for the traffic between your new LAN and WAN. This should already create the appropriate routing table entries so that incoming traffics finds your 129.x.?.1/24. What's missing to tell the outgoing traffic which gateway to use. This can e.g. be done by specifying the gateway of the second WAN interface in the "allow to any" (or whatever firewall rule you use to allow internet access) firewall rule on your "Public LAN" interface. Regarding the public IPs for your 192.168.x.1/22: From my perspective, the clean solution would be to give them a second network interface (e.g. using VLANs) in the "Public LAN" network. This also makes it easier to separate the administrative from the public traffic, e.g. only enable SSH on the interface in 192.168.x.0/22 network.

I finally got this sorted out. Here's how I have done, in case anybody in that situation happens to find this thread. This method does not require creating NAT outbound rules. Assign the WAN2 interface with DHCP or static. This is the WAN of my additionnal public IP Create a VLAN and assign it (I'll call it LAN2 for clarity). VMs using the additionnal IP will be connected to this VLAN. Go to LAN firewall rules, edit the default IPv4 allow rule, and set the gateway to your WAN gateway. Go to LAN2 firewall rules, edit the default IPv4 allow rule, and set the gateway to your WAN2 gateway.

@wickeren said in Routing incoming traffic from HAproxy to endpoint over IPSEC VPN: Inter-machines traffic is prevented by the local firewall on the customer machine. You can't trust clients to self-police. All it takes is one OS level vulnerability they can exploit on the client and the whole thing gets compromised. I wouldn't trust it, but it's not my network, so *shrug*

@Bob-Dig This solution worked also for me. Thank you!

Thanks so much Virgomann. It makes perfect sense.

@Ximulate said in After failover, delay recovery: I'm using the default IP. Some may not agree, but try another IP. I have had luck with 8.8.8.8 (Google DNS). I don't think it will solve this issue since it sounds like you have actual latency impacting you. Edit, On second thought, leave it alone. Particularly since you will be lowering the threshold, using 8.8.8.8 may have moments of latency and packet loss that will be normal and your lower values will make matters worse.

Well at the moment we don't know anything about your setup /LAN network. So it's almost impossible to tell you exactly whats wrong. Generally speaking, running through the pfSense wizard will leave you with a working default configuration, providing Internet access from the LAN Interface. https://docs.netgate.com/pfsense/en/latest/book/config/setup-wizard.html -Rico

Ok I think I have resolved this, after going through debugs and more logs I saw the inactivity timer was being triggered on PFsense. This to me said it was not getting hello's from the multicast / neighbor.. therefore I created a firewall rule to allow any from the source of the neighbor to pfsense and the recurring adj messages stopped.

OK, I got this finally. I've spend ridicules amount of time on this one and it seems to be some kind of bug. So I'm starting with a clean installation. At the beginning instance has only ena0 configured and ena1 is disabled. Changing ena1 to enabled makes Anti-Lockout Rule travel to ena1 (1st thing that shouldn't happen if you ask me) and I can not access 443 using EIP on ena1, just private ip. Then I'm adding allow all rule to both interfaces so all traffic should be accepted and I still can not access 443. [image: 1595373961478-screenshot-2020-07-22-at-01.13.37.png] nc -zv $EIP 443 does not work either from pfsense or remote computer. It works using private ip. Only after removing Anti-Lockout Rule things are getting back to normal.

Hello! Member Down should trigger on the upper threshold for latency or packet loss. Please review and/or post your gateway settings - gateway ip, monitoring/action settings, advanced settings, etc... John

@wolfsden3 This works as expected Go to openvpn client config and change the interface to the charter port. Or even better, create a failover group and assign it there, with the needed priorities.

@ahmetakkaya said in Different Network Structure: @chpalmer Will it work if we connect two devices with port and write route ? Yes. The routed package writes the route for you. But it can be all done manually..

well, I didn't find any issue from your work, and you have some basic routing and switch problems. I will try to make it simple... nat isn't for routing, nat is to prevent private networks to expose on public internet which cannot route private address, and nat is used to allow private networks to access public internet. port forwarding is used to provide private services to public networks. vlan is used for separate networks, vlan can't communicate with other vlan directly, they communicate by routing. so, in your work, nat is used on wan port which replaces source (private network address) to public address, then the public address know how to transmit back. port forwarding is like someone access your public address port 80, which is HTTP, then your firewall forward the request to the server in your private network in your private network, your firewall performs like a router, who knows every network, so if an address from LAN want to access an address in SERV, then it will forward the packet to the gateway of the LAN, which is your firewall LAN address, then the firewall know where next-hop is, it will forward the packet to SERV network.

Thank you!

It gets weirder, made the factory LAN 10.0.13.254/24 and made the 172.16.120.1 another interface (basically flipped the drop downs, and adjusted the IP's and DHCP, etc). Now when attempting to reach 10.0.6.159, it originates from the VPN /30 IP, so weird.

And actually- Ive been wanting to experiment a little with the cable modem and have now figured out that issue as well.. Thanks Raffi_ You got me working on this and Ive solved an issue now.