Suggestions for linking two pfsense setups

[stephenw10]

Ok, then I would start a continuous ping on one side and check the state tables and firewall logs on both firewalls to see where that traffic going.

Ultimately run a pcap on each interface involved to check the packets are arrving/leaving at each point.

Steve

[bingo600]

@MakOwner
Here is a "Crash course" in Vlan tagging on pfSense , if ever needed.
https://forum.netgate.com/post/944381

I hope you find it usable, even though i would not recommend you begin on that stuff right now. Tame your current beast, before releasing a new.

/Bingo

[MakOwner]

So, I thought since I have sort of got this working well enough thanks to all your help, I'd share what I have done -- so far anyway. Thanks for all the help!

I have had to revert to 2.4.4-p3 on both ends.
2.4.5 doesn't seem to play well at this point.
Gateway monitoring seem to be the downfall for some reason.
I'll wait for another update before trying again.

I currently have a physical layout like this:

DHCP servers are active with very limited ranges on each interface.
The office interface handles the DHCP interface handles the .30/24 on the link between buildings and provides the addresses for the management interfaces on the two switches.
Each DHCP server is configured to serve addresses within a specified range and each /24 has a different range just for conveniance. .10/24 uses .110-.120, .20/24 uses .120-.130, .30/24 uses .130-.140 for example.
Not a single DHCP collision or bomb - yet, anyway.
All active physical systems use static DHCP assignment outside the assignment range so that it becomes very obvious if anything goes south.

Al internet bound traffic in the Lab goes out the .20/24 network, all internet bound traffic from the Office goes out the .10/24 network.

Wireless is isolated on both sides to just internet access.

I added the .30/24 interface of the opposite system as a gateway on each pfsense.
I added static routes to .10/24 and .20/24 networks as appropriate.
On the .30/24 interface on each pfsense I added an any-to-any firewall rule.

That let traffic pass both directions.

From there I began working on DNS.
The DNS resolver in 2.4.4 doesn't quite behave as I expected.

Still piddling with that.

After DNS is working like I expect I'll move on to configuring the direct link.

[bingo600]

@MakOwner said in Suggestions for linking two pfsense setups:

So, I thought since I have sort of got this working well enough thanks to all your help, I'd share what I have done -- so far anyway. Thanks for all the help!

I have had to revert to 2.4.4-p3 on both ends.
2.4.5 doesn't seem to play well at this point.
Gateway monitoring seem to be the downfall for some reason.
I'll wait for another update before trying again.

Why does GW monitoring pose a problem ?
It's usually just a "Ping of the GW".

DHCP servers are active with very limited ranges on each interface.
The office interface handles the DHCP interface handles the .30/24 on the link between buildings and provides the addresses for the management interfaces on the two switches.
Each DHCP server is configured to serve addresses within a specified range and each /24 has a different range just for conveniance. .10/24 uses .110-.120, .20/24 uses .120-.130, .30/24 uses .130-.140 for example.
Not a single DHCP collision or bomb - yet, anyway.
All active physical systems use static DHCP assignment outside the assignment range so that it becomes very obvious if anything goes south.

Special way to "reckognize" which DHCP server hands out the ip.
But if it works for you .....
Once you trust DHCP , you could move to something more "default".

I usually let my dhcp start at .129 .. 250
And uses 25..125 for static assignments (1..24) are for infrastructure like routers , switches etc.

I added the .30/24 interface of the opposite system as a gateway on each pfsense.
I added static routes to .10/24 and .20/24 networks as appropriate.

Great ... Do you understand why you have to do that now ?
Tell one pfSense how to "reach" a remote lan on the "other".

On the .30/24 interface on each pfsense I added an any-to-any firewall rule.

That is "fine" , basically making the "Connect net" transparent.
You could fine tune that , later if ever needed.

The .30/24 rules , are like all other "normal" pfSense interface rules "Enter" rules :
See it as: "This permitted traffic" is allowed to "enter" the pfSense-box via "this interface" , everything not permitted is blocked , and will NOT enter the receiving pfSense box.

From there I began working on DNS.
The DNS resolver in 2.4.4 doesn't quite behave as I expected.

What is your issue ?
Still Unbound crashes ?
Did you try the service watchdog ?

After DNS is working like I expect I'll move on to configuring the direct link.

???

Remember when making changes to :
Backup pfSense configs often.
Don't make 10 changes at the same time.

All in all a nice excersise , you certainly got your hands dirty.
Well done , especially for not giving up.

/Bingo

Ps:
If you ever want to "do it right" more flexible , you we should enable multi-vlan on ALL your switches. That would give you the flexibility to make ANY switchport in any building , a member of "ANY vlan".
Ie. a server in building-1 , would be seen as a member of the building-2 Lan.

But that would require that you are reading up on "tagging" on all your switches , including the old "access" switches you use.
And start playing a little with vlans & tagging , before starting that up.