@gertjan Thanks Gertjan!

My 2440, one of the problem machines, crashed. I am going to replace it. Perhaps it has been hardware all along. That little box has problems with the clock on the Celeron. Still, having the information on the location of the code and how the code works for SMTP will be immensely helpful for me on the other machine.

@core7 said in different IP ranges:

two of them in bridge mode

Any special reason for this?

Can you have two bridges with different IP ranges?

Yes. But this question is unsuitable, since you only have one bridge, as I got the above.
Or do you mean different subnets on the member interfaces of a single bridge?

@mindtwist it’s a YY/MM date based version. They are targeting 3 Plus versions per year.

@jarhead I didn't do a range, I couldn't remember if it assigns 1 or 254 as the router, so I just did both, for each subnet I've seen.

You are correct, I did use the word "range", but I meant to "cover those two ranges for possible router addresses". Sorry for the confusion.

@steveits That fixed it! You're a superstar! I owe you a pint. Or six.

@steveits That fixed the problem - changing the gateway from "Default" to the gateway group resolved my issue. Thanks.

@testcb00 said in Trying to connect two devices behind two interfaces:

Finally, I find that I have to set up a static route in the NUT server.

You shouldn't have to do that in a normal setup.. That would only make sense if this nut sever was not using pfsense as its gateway.

If you can ping the pfsense IP of this vlan interface, but not devices on this vlan. That normally screams host firewall not allowing remote IPs, or again this device not using pfsense as its gateway either.

I have found the answer to the first part of my own question:

https://docs.netgate.com/pfsense/en/latest/config/advanced-misc.html#skip-rules-when-gateway-is-down

I needed to check "System/Advanced/Miscellaneous tab/Gateway Monitoring section/Do not create rules when gateway is down", this makes the rules behave the way I wanted.

If anyone can still explain when/how/why OpenVPN needs a default gateway for connections to be made successfully, and whether it can be made to work without one, that would still be useful.

I applied all the changes and tested and everything works! Thanks for all the help.

@aduzsardi
Basically the default gateway is used. But if a request goes to an IP out of the second subnet pfSense uses this IP for response as well, of course. Now if the default gateway lies outside of this subnet it will use the gateway that matches the subnet.

@eds89 That's the same as I meant, i.e. setting priorities on LAN to PfSense traffic !!

If the shaping rules are the same, then the only change is the WAN config. What is different ?

@ervin23 I would guess you would divide those groups by vlans but if you don't want to, it should be doable like you have described it, not done it like that myself though.

You should beginn with something like this and get it working. Also see this.

@nikim
Did you by any change nat the outbound of pfSense to the CARP VIP?
Show the outbound NAT rules please, if unsure.

Did you state an alternative monitoring IP?

@johnpoz I wouldnt. Sorry. I misunderstood you :)

@robh-0 What happens if you restart the dpinger service instead?
There are situations where the pings are responding properly (do you have a monitoring IP configured?) and a dpinger bounce will help that.

Are all your shared gateway rules switched to the new gateway group or did you simply update your original failover group to reflect the change?

Lastly is it a case of some but not all of the traffic is routing out WAN1 after fail-back? If so that's a states issue and they will reset when the states clear out for the WAN2 connection.

@jemadsen
Thank you Steve for the recommendations.
I will look into reflection and it sound like it will help.
I have been doing traceroutes, tcpdumps, netcats. I setup a dual setup, where the VM FW handles the mail, VoIP and Web traffic. The Proctectli handles the rest of the traffic thru Starlink.
I scanned some of the troubleshooting document and tried the recommendations.
I initially started with the VM configuration, but when it didn't work, I reset and manually configured the Proctectli. The VM was my learning environment with more "STUFF" to take into account to troubleshoot.
I have built up several pfSense FWs over the years and I know most of my mestakes, unfortunately, I am also good making new ones. 😊
Having the VM FW to use as reference. I searched for some example similar to mine, but all were failover/load balancing.
I need to get my servers back online so I am using both FWs one that works for the CLWAN and the other for the SLWAN. It working now. Next I will build up a test setup on my VM Server and trouble with that.
I will continue to look for an example or recipe. It is my birthday this weekend. 🎂