Bridge and Translation Address causes pfSense to freezz
-
Hey
I maybe think i have found a bug i both 1.2.3 and in the newest build: pfSense-2.0-BETA4-20100919-0059 (amd64).
I have installed a pfSense 2.0beta4 and added following interfaces:
WAN - IP: xxx.xxx.xxx.3 - Gateway: xxx.xxx.xxx.1
LAN - IP: 10.0.0.2
DMZ - Bridged with WAN in bridge0Then I have added some CARP IP's for the interfaecs: WAN = xxx.xxx.xxx.2 and LAN = 10.0.0.1 (Because I want to run the setup in failover)
Then I have created Advanced Outgoing Nat rules for LAN, where I have added the WAN CARP IP as the Translation Address for the LAN NAT rule.
When I’m then downloading some data from the internet (WAN), from a computer inside the LAN, the pfSense box stops to answer (aften 30-60 sec), also the console stops and the only thing I can do, is a reboot of the server.
If I remove the bridge between WAN and DMZ, OR the Translation Address from the NAT rule, it works perfect.I have created another post long time ago, when I found the problem in the 1.2.3 version, but now I have tested with 2.0 and have the same problem. http://forum.pfsense.org/index.php/topic,27118.0.html
I hope anyone with 3 interfaces would test this, to comfirm. (I have tryed with two dirrerent servers, so i don't think it's an hardware error)
/Jacob
-
Has anyone tryed?
-
Sounds like something I ran into last week. I wasn't able to do CARP with bridging in 1.2.x or 2.0 or it resulted in the machine seizing. Seems to be an OS bug of some kind, not sure there's much we can do, other than do not use CARP+bridging (which we've never recommended…)
It used to work on 1.2, but there were a lot of underlying OS changes to bridging since then.
-
To expand on that, I changed the exact same setup to routing+carp, and it's been flawless for a week on the exact same hardware.
-
I can get CARP+briding to work, but when I add the CARP IP to the Translation Address for the NAT outgoing rule, the system craches.
If I can't use bridge+CARP, how should I then setup a high-availability firewall for our hosting setup. Do we have to NAT all IPs? Or do we have to buy some commercial software?
-
I did full routing, no NAT, but I had a ton of IPs handy.
-
Really for what you're doing you also need four interfaces, you seem to be missing a sync interface for a proper high-availability setup. Doing the sync can work on LAN, but it is not ideal.
Also, to clarify, was your NAT translation from LAN -> WAN CARP, and it did not involve trying to apply NAT to traffic from the bridge?
Another problem with doing CARP+LAN+Bridge like that is you cannot get a route from the bridged items to the LAN. You cannot contact a CARP VIP on WAN from the Bridged segment (or you used to not be able to, I still don't think it works) so those items can't get a route back through.
-
And one more thing to try: Does it still break if you assign bridge0 as WAN instead of what used to be your WAN interface? (then add the old WAN as another OPT and make sure the bridge interface is still bridging the right way)
-
Yes. The servers have 6 NIC's but to simplify the problem, I have disabled the pfSync, to check if it had anything to do with it.
Yes. The NAT translation was for the LAN IP's where I added the CARP WAN IP.
How can I check for that the system not are natting traffic from the bridge? (I have added an screenshot of my outgoing NAT configuration)Its right, I can't contact the WAN CARP IP from within the bridged segment.
Should I try to assign the bridge0 to the WAN interface? And then add the old NIC til an extra OPT? WHwre should I then setup the IP. On the WAN og OPT interface?
-
IP would be setup on the new WAN (which would really be bridge0) the old WAN (physical interface) would merely be assigned and enabled with an IP type of none as a stub so it can be used in the bridge GUI.
Only work on WAN (bridge0) once you assign it that way.
The NAT rules look OK, as long as none of those IPs match items in the DMZ.
-
I will try that, an then return with a result.
-
OK. I also started a ticket with some info from when I hit it last week, if you can provide any other input/observations over what I have in there already, I can add them.
http://redmine.pfsense.org/issues/910
-
Could you maybe explain how we should setup it up, if we are going with the routing mode.
What are the pro/cons of that choise?
-
You'd just move items in the DMZ to a separate subnet from those in WAN.
If you have a large enough WAN subnet and control of how it hits your network, you could split what you have into two smaller subnets and use some in the DMZ also.
There isn't anything fancy about that kind of setup, it's a standard CARP+DMZ setup. Each unit would need an IP in the new subnet, plus one shared CARP VIP to act as the gateway for the DMZ items.
Routing it is much cleaner, you won't have to worry about bridging or not being able to reach LAN, and failover is known to work flawlessly in that scenario.
Alternately, you could try two pairs of routers. A CARP HA setup for LAN, and a separate CARP HA setup that bridges that doesn't involve NAT (if that does turn out to be the problem).
-
We have a C-class provide from our co-location company.
The IP xxx.xxx.xxx.1 is the gateway provided from the co-location company.
We need to route rest of IP's to the DMZ. Are there an how-to, where it explains how to setup this in a HA enviroment.
-
Not sure if there is a how-to, some of that is covered in the book.
If many of your things will be in the DMZ, you might split that up into two /25's and have your ISP adjust their side, and route the higher IPs to your WAN CARP VIP, so you can use them internally.
The subnetting could get a bit trickier if you need more than that, and might involve some other changes on the ISP side as well.
-
I have following in my network:
One /24 subnet
One /26 subnet
One /29 subnetCan i then use the /24 subnet for the DMZ and ie. the /29 subnet for the WAN? Without having the ISP to do anything?
-
That should work, if your Master/Back/WAN CARP VIPs are in the /29, and the /24 is completely on the DMZ (Master, Backup, and Shared CARP for DMZ)
It should work fine
-
And it would be a better solution, instead of a bridge+CARP configuration.
Are there any downsides of using the routing setup combined with CARP?
-
For at routing setup I do the following:
Subnets: xxx.xxx.183.168/29 and xxx.xxx.214.0/24
pfSense - Master:
WAN: xxx.xxx.183.171
DMZ: xxx.xxx.214.3pfSense - Backup:
WAN: xxx.xxx.183.172
DMZ: xxx.xxx.214.4CARP IPS:
WAN: xxx.xxx.183.170
DMZ: xxx.xxx.214.2WAN are using xxx.xxx.183.169 as Gateway (Provide by ISP)
Hosts on the DMZ network are using xxx.xxx.214.3 as gateway (The CARP IP for DMZ)Is that a working enviroment?
What about the IP: xxx.xxx.214.1 that are provide from our ISP as Gateway for the xxx.xxx.214.0/24 subnet?
-
Your ISP would have to stop being a gateway for that subnet. The entire 214.0/24 subnet should be routed to x.x.183.170.
-
jimp: Thanks for all your help. I have contacted our ISP.
I have found the example of setting up CARP and routing to DMZ in your book, so now I only need the ISP to make their changes.
-
By the way: What are the technically definitions of the operation our ISP has to do?
-
They would just be considered routing changes.
-
Perfect. Thanks. I was looking for it, at their self-service system.
-
Now I got it all to work. And the failover works perfect.
But when I try to traceroute a host inside my DMZ, the last HOP before the host is the gateway. But it is not the CARP IP of the WAN interface, but the IP-assigned on the interface.
The CARP is: xxx.xxx.xxx.170
pfSense1: xxx.xxx.xxx.171
pfSense2: xxx.xxx.xxx.172It is the active pfSense's interface IP, that are returned by the trace route. Is that correct?
I have checked that my ISP, are routing the DMZ subnet to the xxx.xxx.xxx.170 IP. -
The way traceroute works I think that is expected. If the IPs are really routed to the CARP VIP, I wouldn't worry about it, though if you want to be absolutely certain, you can always do a traceroute, force a failover, and try it again.
If it works both times, it's probably just a quirk of how traceroute shows up in the scenario.
