Cisco PIX 506e and pfSense 2.0 with multi-WAN, routing and NAT issues…

[microchip]

First of all, apologies if this is in the wrong place - there were quite a few categories it could potentially go under, and I wasn't entirely sure which one it belonged under. I'm sure someone will move it if it's in totally the wrong place.

I'm attempting to migrate our current system to a pfSense-based load balanced solution. The setup at present is as follows:

Current internet setup: 2MB leased line with 4 IPs and a gateway IP (block of 6 including broadcast)

Network –> Windows 2003 domain controller with screening web proxy --> Cisco Pix 506e --> Leased line internet

The Cisco has a main external IP used for direct traffic, and another IP is on a 1:1 mapping with the server for incoming mail and other connections. It is also used by support contractors to hook in via the Cisco VPN client.

I've established another ISP connection via an ADSL2 connection, which will cost a whole lot less than the above leased line. I wanted to set it up load balanced, preferably with a weighting towards the ADSL connection as it's quicker, which I looked at doing via the traffic management system in pfSense 2.0, but it was optional, and reserved for "after it all works". The ADSL2 router is set up to directly bridge the IP into the pfSense box.

Unfortunately, at present, I'm not allowed to take the PIX out of the loop, so my idea was as follows:

Network --> Win2K3 box --> Cisco PIX --> pfSense --> ADSL / Leased Line connections (dual WAN).

I set up the pfSense box to have a LAN IP of 192.168.100.1, and connected the leased line to one of the network cards on the back, assigning the main IP to the . I also connected the ADSL2 to the pfSense box.

If I have a machine directly connecting, using DHCP on the pfSense side, outgoing connections work beautifully, hopping between the assigned IP of the leased line and the IP of the ADSL2. Obviously, this doesn't apply to incoming connections, as no ports are forwarded.

I then went to set up the Cisco PIX. I reassigned the main port, labelled "outgoing" on the PIX config, to 192.168.100.1, and set the default gateway to 192.168.100.10, which I assigned on the pfSense box as the LAN IP. Outgoing routing via the PIX to the pfSense box worked fantastically, again with the IPs swapping as I accessed websites, and it running beautifully. I routed port 500 UDP to the Cisco box, along with layer 7 traffic on AH and ESP, but was unable to test the incoming connections at the time.

The problem comes with the second IP. I added external IP of the server (as assigned on the leased line) to map to 192.168.100.2 on the internal LAN, and added the IP to the external connection on the pfSense box (tried both virtual IP and Proxy ARP if I remember correctly), which remained unpingable from the outside world. My thought was to add the external IP to the pfSense box, and originally do a 1:1 NAT, but I'm under the impression that doing so would route all traffic coming from the server's internal IP (which includes all proxied web traffic) straight down the 1:1 NATted IP, which would defeat the point in having the WAN load balancing completely. (Please feel free to correct me if I'm wrong here, it'd solve a lot of my headaches if I was!). Ideally, what I'd like to be able to do is accept traffic on a bunch of ports on the additional IP, forward it to the Cisco on the alternative port, and then have the Cisco forward again to the server.

I set up the above setup, thinking it should work in theory. Doing a telnet to port 25 on the new "mail server" IP on the Cisco resulted in being able to talk SMTP to the server in question, which worked fine. However, attempting to telnet externally into port 25 on the new IP didn't work. Checking the logs on the PIX router revealed a lot of connection teardowns with SYN timeout mentioned. From what I read online, this equates to it not being able to communicate back with the computer attempting the connection. Other ports didn't work either. In an ideal world, I'd like to just forward anything on that particular IP straight at the Cisco router, but I wasn't entirely sure how to do that without potentially screwing up the main WAN load balancing setup. Also, ideally, we could do with mail from the server being routed down that IP for SPF reasons, although the SPF record can be altered without too much of a problem.

Am I missing something really obvious that I could do to fix this?

Thanks in advance,

• Rick

[Guest]

You should not be running pfSense 2.0 in production.  It is labeled an "alpha alpha alpha" release and is not suitable for any sort of production use.