Using pfsense with multiple WANs

lewis

This post is deleted!

lewis

@stephenw10 said in Multi LAN networks to one pfsense:

Ok, so the problem becomes apparent. On one side (looks like pf02) the GRE tunnel is incorrectly using 10.0.0.1 when it should be using the VIP address, 10.100.0.2.
So probably it is set to be on the LAN there and should be on the VIP.

Steve

I see that in the image I shares but not in the config anywhere.

stephenw10

@lewis said in Multi LAN networks to one pfsense:

He says, create a dnat on pf02 10.100.0.120:443 to 10.0.0.120:443

10.100.0.120 doesn't exist but assuming you create it as a VIP on pf01 you can do that. But it will only work for one pfSense instance forwarding traffic because that is the gateway on pf01 DCLAN.

If you try to something similar from pf03 without a GRE tunnel replies will go back via pf02 resulting in an asymmetric route and blocked traffic.

Steve

stephenw10

@lewis said in Multi LAN networks to one pfsense:

I see that in the image I shares but not in the config anywhere.

I expect that is i the GRE tunnel config on pf02.

lewis

It's still like this cept the masks are now /30 and static is disabled on both sides. Where is that 10.0.0.1 coming from then?

lewis

@stephenw10 said in Multi LAN networks to one pfsense:

@lewis said in Multi LAN networks to one pfsense:

He says, create a dnat on pf02 10.100.0.120:443 to 10.0.0.120:443

10.100.0.120 doesn't exist but assuming you create it as a VIP on pf01 you can do that. But it will only work for one pfSense instance forwarding traffic because that is the gateway on pf01 DCLAN.

If you try to something similar from pf03 without a GRE tunnel replies will go back via pf02 resulting in an asymmetric route and blocked traffic.

Steve

What I meant is that instead of a VIP, what if I created a new /24 network interface instead of a tunnel? Most everything is port forwarded so really, to move things from pf02 to pf01, I mainly need to access one host/server at a time, then forward that access from pf02 to pf01.

lewis

I'm just not sure where that 10.0.0.1 is coming from. Maybe it was something I was playing with when I took the pic but it's gone now.

Not sure if it's strange or not but to simply try something new, I deleted all the GRE stuff on both pf. I then created a VIP of 10.100.0.120 on pf02 and port forwarded that to a vm at 10.0.0.120 but even with a full any any rule, I can't reach that vm from pf01.

Weee, how could nothing work?

stephenw10

You need to change the parent interface for the GRE tunnel on pf02 to the VIP.

Right now it's set as LAN in your screenshot and that's 10.0.0.1.

Steve

lewis

Was not able to find that. I removed everything so I have a clean slate. I'm starting all over right now.

pf01: I have 10.100.0.1/24 on DCLAN.
pf01: I have 10.0.0.1/24 on LAN (10g1LAN).

pf02: I have 10.0.0.1/24 on LAN.
pf02: I create a VIP of 10.100.0.2/24 on LAN interface.

pf01: I create a GRE tunnel on DCLAN.
Remote address: 10.100.0.2
IPv4 local: 10.102.0.1 /30
IPv4 remote: 10.102.0.2/30
I do not enable static.
New interface shows up so I add it. I rename it to GRE and enable it.

pf02: I create a GRE tunnel on LAN.
Remote address: 10.100.0.1
IPv4 local: 10.102.0.2/30
IPv4 remote: 10.102.0.1/30
I do not enable static.

pf02
New interface shows up so I add it. I rename it to GRE and enable it.

Now I can ping from either side but only to the GRE IPs, not the tunnel ones. (Sorry, don't know the correct terms yet).

pf01: ping 10.100.0.2 = replies
pf02: ping 10.100.0.1 = replies

This is where I'm at now.

States on pf02 still show 10.0.0.1.

stephenw10

@lewis said in Multi LAN networks to one pfsense:

pf02: I create a GRE tunnel on LAN.

Nope, it must be on the VIP otherwise the source IP will be wrong as you are seeing.

lewis

This post is deleted!

lewis

Ok, updated. Look right now?

pf01

pf02

lewis

From cli on both pf, Still cannot ping between 10.102.0.1 and 10.102.0.2 but I assume the next thing is rules.

My first planned test is like this.

From 10.0.0.71 on the pf01 LAN side, I want to reach 10.0.0.120 behind the LAN on pf02. Any port.

stephenw10

Yup, the tunnel looks good now, states both ways on both sides.

If you have rules on the GRE interfaces to allow it you should be able to ping between them.

Steve

lewis

@stephenw10 said in Multi LAN networks to one pfsense:

Yup, the tunnel looks good now, states both ways on both sides.

If you have rules on the GRE interfaces to allow it you should be able to ping between them.

Steve

To save 10 messages and to make sure I don't accidentally take something down, can you tell me what I would do at this point.

For my test, from 10.0.0.71 on the pf01 LAN side, I want to reach 10.0.0.120 behind the LAN on pf02. Any port.

What rule do I need on pfxx and is the rule on the LAN or GRE side? With this one example, I should be able to start adding more rules safely.

johnpoz

@lewis said in Multi LAN networks to one pfsense:

10.0.0.71 on the pf01 LAN side, I want to reach 10.0.0.120 behind the LAN on pf02.

I just don't see how you think that could happen? Those are the same network..

lewis

@johnpoz said in Multi LAN networks to one pfsense:

@lewis said in Multi LAN networks to one pfsense:

10.0.0.71 on the pf01 LAN side, I want to reach 10.0.0.120 behind the LAN on pf02.

I just don't see how you think that could happen? Those are the same network..

LOL, yes, if you just walked into this without reading the thread, then you will be confused.

lewis

I don't care if I use gre or simply find a way to move servers/services one at a time using just one or two rules and later forward from pf02 to where they would then be, pf01. I wondered about using a virtual IP or set of VIP's on pf02 to accomplish this but I can't figure out how to set up rules for that sort of scenario.

johnpoz

@lewis said in Multi LAN networks to one pfsense:

LOL, this is confusing. That's been the point of this post all along. What happened?

Not sure what the point of this thread is, your drawings don't make that much sense, and you have confusing vips and lans and overlaps.

So want to bridge over a GRE tunnel - not even sure that is possible with pfsense.. I haven't done any sort of deep dive into the thread - but you sure @stephenw10 understands your trying to do a L2 bridge across your GRE?

stephenw10

Hmm, that's not the direction I expect that to work in.

Currently you have a bunch of servers behind pf02 that are accessed using public IPs via port forwards in pf02.

You need to move those servers to be behind pf01 instead as I understand it?

So what you need to do is change the port forwards in pf02 to point to the pf01 GRE tunnel IP instead. And then on pf01 add port forwards from the GRE tunnel IP to the servers in the pf01 LAN.

I said before that you won't be able to connect directly from pf02 LAN subnet to the servers behind pf01 because the source IPs would conflict. The same would be true the other way.
If you really need to do that you would need to use outboubnd NAT to mask the source IP and use the GRE tunnel IP directly as the destination.

Also just to reiterate; you don't actually need the GRE tunnel for just two pfSense instances like that but as soon as you add another (or more) you would so it make sense to do them all the same.

Steve