Using pfsense with multiple WANs

lewis

It's still like this cept the masks are now /30 and static is disabled on both sides. Where is that 10.0.0.1 coming from then?

lewis

@stephenw10 said in Multi LAN networks to one pfsense:

@lewis said in Multi LAN networks to one pfsense:

He says, create a dnat on pf02 10.100.0.120:443 to 10.0.0.120:443

10.100.0.120 doesn't exist but assuming you create it as a VIP on pf01 you can do that. But it will only work for one pfSense instance forwarding traffic because that is the gateway on pf01 DCLAN.

If you try to something similar from pf03 without a GRE tunnel replies will go back via pf02 resulting in an asymmetric route and blocked traffic.

Steve

What I meant is that instead of a VIP, what if I created a new /24 network interface instead of a tunnel? Most everything is port forwarded so really, to move things from pf02 to pf01, I mainly need to access one host/server at a time, then forward that access from pf02 to pf01.

lewis

I'm just not sure where that 10.0.0.1 is coming from. Maybe it was something I was playing with when I took the pic but it's gone now.

Not sure if it's strange or not but to simply try something new, I deleted all the GRE stuff on both pf. I then created a VIP of 10.100.0.120 on pf02 and port forwarded that to a vm at 10.0.0.120 but even with a full any any rule, I can't reach that vm from pf01.

Weee, how could nothing work?

stephenw10

You need to change the parent interface for the GRE tunnel on pf02 to the VIP.

Right now it's set as LAN in your screenshot and that's 10.0.0.1.

Steve

lewis

Was not able to find that. I removed everything so I have a clean slate. I'm starting all over right now.

pf01: I have 10.100.0.1/24 on DCLAN.
pf01: I have 10.0.0.1/24 on LAN (10g1LAN).

pf02: I have 10.0.0.1/24 on LAN.
pf02: I create a VIP of 10.100.0.2/24 on LAN interface.

pf01: I create a GRE tunnel on DCLAN.
Remote address: 10.100.0.2
IPv4 local: 10.102.0.1 /30
IPv4 remote: 10.102.0.2/30
I do not enable static.
New interface shows up so I add it. I rename it to GRE and enable it.

pf02: I create a GRE tunnel on LAN.
Remote address: 10.100.0.1
IPv4 local: 10.102.0.2/30
IPv4 remote: 10.102.0.1/30
I do not enable static.

pf02
New interface shows up so I add it. I rename it to GRE and enable it.

Now I can ping from either side but only to the GRE IPs, not the tunnel ones. (Sorry, don't know the correct terms yet).

pf01: ping 10.100.0.2 = replies
pf02: ping 10.100.0.1 = replies

This is where I'm at now.

States on pf02 still show 10.0.0.1.

stephenw10

@lewis said in Multi LAN networks to one pfsense:

pf02: I create a GRE tunnel on LAN.

Nope, it must be on the VIP otherwise the source IP will be wrong as you are seeing.

lewis

This post is deleted!

lewis

Ok, updated. Look right now?

pf01

pf02

lewis

From cli on both pf, Still cannot ping between 10.102.0.1 and 10.102.0.2 but I assume the next thing is rules.

My first planned test is like this.

From 10.0.0.71 on the pf01 LAN side, I want to reach 10.0.0.120 behind the LAN on pf02. Any port.

stephenw10

Yup, the tunnel looks good now, states both ways on both sides.

If you have rules on the GRE interfaces to allow it you should be able to ping between them.

Steve

lewis

@stephenw10 said in Multi LAN networks to one pfsense:

Yup, the tunnel looks good now, states both ways on both sides.

If you have rules on the GRE interfaces to allow it you should be able to ping between them.

Steve

To save 10 messages and to make sure I don't accidentally take something down, can you tell me what I would do at this point.

For my test, from 10.0.0.71 on the pf01 LAN side, I want to reach 10.0.0.120 behind the LAN on pf02. Any port.

What rule do I need on pfxx and is the rule on the LAN or GRE side? With this one example, I should be able to start adding more rules safely.

johnpoz

@lewis said in Multi LAN networks to one pfsense:

10.0.0.71 on the pf01 LAN side, I want to reach 10.0.0.120 behind the LAN on pf02.

I just don't see how you think that could happen? Those are the same network..

lewis

@johnpoz said in Multi LAN networks to one pfsense:

@lewis said in Multi LAN networks to one pfsense:

10.0.0.71 on the pf01 LAN side, I want to reach 10.0.0.120 behind the LAN on pf02.

I just don't see how you think that could happen? Those are the same network..

LOL, yes, if you just walked into this without reading the thread, then you will be confused.

lewis

I don't care if I use gre or simply find a way to move servers/services one at a time using just one or two rules and later forward from pf02 to where they would then be, pf01. I wondered about using a virtual IP or set of VIP's on pf02 to accomplish this but I can't figure out how to set up rules for that sort of scenario.

johnpoz

@lewis said in Multi LAN networks to one pfsense:

LOL, this is confusing. That's been the point of this post all along. What happened?

Not sure what the point of this thread is, your drawings don't make that much sense, and you have confusing vips and lans and overlaps.

So want to bridge over a GRE tunnel - not even sure that is possible with pfsense.. I haven't done any sort of deep dive into the thread - but you sure @stephenw10 understands your trying to do a L2 bridge across your GRE?

stephenw10

Hmm, that's not the direction I expect that to work in.

Currently you have a bunch of servers behind pf02 that are accessed using public IPs via port forwards in pf02.

You need to move those servers to be behind pf01 instead as I understand it?

So what you need to do is change the port forwards in pf02 to point to the pf01 GRE tunnel IP instead. And then on pf01 add port forwards from the GRE tunnel IP to the servers in the pf01 LAN.

I said before that you won't be able to connect directly from pf02 LAN subnet to the servers behind pf01 because the source IPs would conflict. The same would be true the other way.
If you really need to do that you would need to use outboubnd NAT to mask the source IP and use the GRE tunnel IP directly as the destination.

Also just to reiterate; you don't actually need the GRE tunnel for just two pfSense instances like that but as soon as you add another (or more) you would so it make sense to do them all the same.

Steve

lewis

@johnpoz said in Multi LAN networks to one pfsense:

@lewis said in Multi LAN networks to one pfsense:

LOL, this is confusing. That's been the point of this post all along. What happened?

Not sure what the point of this thread is, your drawings don't make that much sense, and you have confusing vips and lans and overlaps.

So want to bridge over a GRE tunnel - not even sure that is possible with pfsense.. I haven't done any sort of deep dive into the thread - but you sure @stephenw10 understands your trying to do a L2 bridge across your GRE?

I don't think it's that confusing other than you've not been part of the entire and yes, very long thread.

Steve suggested using gre to accomplish what I need but I had a couple of mistakes in there that I've since fixed.

There is no point in re-explaining the whole thing since its all up above and just adds to a very long thread.

The short of it is, I have two pf firewalls, both are using a LAN of 10.0.0.1/24. I cannot change the LAN ips, I have to migrate servers/services from the first pf to the second one and only have a VLAN to do it with.

stephenw10

@johnpoz
The problem we are trying to address is this:

Muliple pfSense instances with servers behind them all configured to port forward public IPs to those servers. All instances share a common LAN subnet.
Those servers are being physically relocated to all be behind a new pfSense install.
The servers much remain in the same private subnet.
Because of reasons the public IPs cannot be routed to the new pfSense instance so traffic to each public IP has to go through the existing installs.

lewis

I think at this point, the simplest way to do this would be as follows.

Use the public interface of pf02 to get at the servers/services on its LAN. Once moved to the new network, forward the traffic from pf02 to pf01 over the VLAN and to 10.100.0.1/24.

So, a rule on pf02 that is say port forwarding ports 80/443 to 10.0.0.120 on its LAN would be changed to 10.100.0.1.

On pf01, I would just have rules for each service as I do now for LAN on pf02.

In my mind, without the amount of knowledge you guys have, that sounds like it would work.

stephenw10

Yup, that will work.
But only for pf02 because it's acting as the gateway for the pf01 DCLAN interface. If you try to do the same for pf03 it will fail.