STP and network
-
You get a WAN interface of, say 198.51.100.32/30. Your default gateway is 198.51.100.33 and your interface is 198.51.100.34/30.
They route 203.0.113.64/29 to 198.51.100.34.
You put 203.0.113.65/29 on an inside interface and turn off NAT.
You give hosts on that network 203.0.113.66 - 203.0.113.70.
No bridging mess.
No NAT.
Exactly how it's supposed to be.
-
Ok, I have asked my ISP about this and wait for answer. I do also have some failover system that is not mentioned here, that happens transparent to me.
But, to have it transparent like today with the LACP-trunk, how would I do it? The way I have it working as of today, is apparantly by filtering on the member interface. As soon as I filter only on the bridge, the traffic is loose. In my mind (without thinking about networks), it seems logical that new LAN-team-interface is beeing filtered this way.
-
Just to stress.. Having your public range routed to you is way better than any transparent/bridge nonsense ;) What size public range do you have? /29 is pretty small… But if /28 or bigger I would for sure think it should be routed to you vs just attached to their network.
I personally even if having to work with attached network vs routed would just nat it and use port forwards. Simple enough to use your specific IPs for different servers via vips..
-
Have /24-range, with 256-addresses. It hasn't been stressfull so far the last 10 years, since I don't do a lot of network-stuff or have any special routing/requirements. Think this is the first time I have had problems and that is because I want it to be more redundant by using LACP :)
Most of my servers are web-servers with control-panels that requires a certain IP to bind to (due to licenses). If I was to have local ips on all servers and have mapping to the public-ip for all servers, I suddenly have to mange 256-addresses * 2. And that is before I have to NAT all ports for common services like DirectAdmin and cPanel-servers use. Now I can simply group the servers based on profile.
But I'm sure there are ways to do this simple in NAT as well.
-
It is pretty much insane to have that network on your WAN interface. It should be routed to you instead.
-
You have a /24 and its not routed to you?? Wow.. That is nuts dude.. I would for sure change that.. put pfsense in carp, then get some stack switches between your pfsense carp and your servers and now your cooking with gas.. ;)
-
Have never even been thinking of that, or that there was any disadvantage of running it transparent. When starting this business, I was told that NAT-was slower (performance-wise) and required more setup. The FortiGate I started with supported that easy.
But basically, with your suggestion, I would get assigned additional small network with public static IPs just for my WAN-area. And I could then just remove the bridge on my LAN side and threat the public IPs like I would do on a private network? I don't have any NAT today, so wouldn't have to change there.
Based on this, I shouldn't even have to change the fw rules I think, so that's a good thing. Let's see what my ISP says, maybe there are some kind of setup here that differ from the normal. But I'm still curious to how I would complete the setup in case my ISP says no..
-
Wish I could be more help with bridges on pfsense. But software bridges should be avoided at all costs if you ask me. While your use of it is very valid setup when under the restrictions of having to have a public range directly attached vs routed and wanting to put a firewall in between.
So while your use of transparent is valid, I would suggest if possible migrate away from it. If you had small amount of space like a /29 or even /28 nat with vip and then 1:1 would remove your issues of having to deal with port forwards.. Doesn't remove the issue if you have software licensed to some IP… What if you loose your public space? Do you actually own this /24 in arin or whatever RIR you might be in? If so you should be able to get your own ASN and just route it yourself to wherever you want via your ISP your using, etc.
I manage a /16 from arin.. So never run into these sorts of issues. We just advertise the space we need to use where ever, and be done with it ;) You just need to work with whatever ISP to accept and advertise out your routes, etc.
But if you just got said /24 from some DC network your located in - they really shouldn't have any issues with routing it to you vs directly attaching it to their equipment.
-
What if you loose your public space? Do you actually own this /24 in arin or whatever RIR you might be in?
I don't own it, just renting it as long as I need it. If I was to change ISP, it would be a bit difficult (lot of dns to change..), but not impossible. Can change the IP for the license from control-panels.
OK, I'll hope that my ISP comes with good news and if not, I'll just have to try and fail until it works ;)
-
I have two ports in my datacenter assigned with my current C-net/24-net (an Catalyst owned/administered by my ISP only). Both of the ports work, I currently only use one of them.
I have two pfSense and I consider doing CARP on them (instead of having just a cold-turned-off ready) and have one port to each of my fw.
Would I ask my ISP for a network of two IPs and then assign WAN-IP to each of the pfSense.. or do they need to route it to only one IP/device? I'm beginning to suspect that your suggestion - along with a carp setup - requires me to introduce one additional switch/router. Then I migth as well keep the cold-backup to avoid introducing (more) single-point of failture.
"You get a WAN interface of, say 198.51.100.32/30. Your default gateway is 198.51.100.33 and your interface is 198.51.100.34/30."
BTW: is this reccomendation just because how pfSense works with bridge or would you reccomend this setup no matter what type of firewall?
-
If you need more address on your wan transit network for a carp, then use a /29 vs /30.. Yes carp requires 3 IPs.. So /30 wouldn't work.
-
Doesn't carp communicate over a local IP only? So a dedicated cable on a port not part of LAN/WAN, just a dedicated cable with virtual 10.0.0.1/10.0.0.2 on each?
And then one public WAN-IP for each.
-
My ISP says this:
"This should be possible yes - the challenge is how to be able to route the current network, if you have two firewalls at 2 different WAN-addresses. Without major changes, I can only route /24-network to one address.
-
you could always split the /24 into 2 /25s and route 1 to each.. All comes down to how you want it. Or if you setup carp on your 2 firewalls then you would only be routing to 1 IP, the CARP address on your wan side.
I would have to go back and read the thread if you had laid out how you have your 2 firewalls setup and different networks behind them, etc.
-
"Or if you setup carp on your 2 firewalls then you would only be routing to 1 IP, the CARP address on your wan sid"
This is the prefered method, but I assumed it wasn't an option? If so, it is perfect!
Let's say that they assign a 5 public static IP-transport-network to me, where 80.80.80.81 is the main/assigned interface. The fw1 gets .82 and fw2 gets .83.
I create a local link between a free interface on both, with two static local IPs to maintain the carp… and I put .81 on the cluster.
Is it as simple as that? If so, it would be pefect, but I assume it is more to it ;)
-
Like the drawing attached. I'm using fake static IPs of course, but maybe it is more clear what I want to do?
The 4.4.4.0-network indicate the current /24 network I'm assigned today. I wouldn't need to change the servers from what I have today (I think)
The 8.8.8.0-network indicate the new small transport-network, that will be assigned both WAN and the cluster/CARP on WAN-side.
-
Nope its really that simple ;)
https://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_(CARP)
I have not read thru that doc in awhile - so maybe its a bit dated, maybe something has changed in newer versions. But yeah its pretty simple to setup the carp..
This shows a nat network behind - but you could put your routed network behind there two.. You setup your stack switches and some laggs - and yeah buddy cooking with gas.. And remove all your SOPF issues.
-
I think it was the routed network that made me thing it wasn't possible.
What would my GW be on the inside on each machine, would it be the same as the cluster IP from the transport network like 8.8.8.1 in my drawing? Or can I create additional interface on the cluster (virtual IP or something) so that I can have the same gw as today? (4.4.4.1).
-
sure just use that IP of your routed segment as your carp on the "lan" side of pfsense.. Before you had this
PE (provider equipment) 4.4.4.1 –-- 4.4.4/24 ---- CE (pfsense - BRIDGE) ---- 4.4.4/24 ---- 4.4.4.X Server..
You end up with this
PE x.x.x.1 --- transit x.x.x/29 ---- x.x.x.2,.3,.4 CE (pfsense CARP) 4.4.4.1, .2, .3 ---- 4.4.4/24 ---- 4.4.4.x Server
Does that help?
-
Please read this for a short explanation of the basic elements of building a CARP/HA pair: https://forum.pfsense.org/index.php?topic=136085.msg744802#msg744802
