Routed IP's - Multi WAN or Virtual IP?
-
I'm trying to use a PPPoE* WAN connection where we have a /30. *It's actually a PPPoA but we have a Draytek Vigor 120 bridging it to the pfSense box as PPPoE, letting pfSense authenticate and take the first IP.
What I was hoping to do was to set pfSense up using the first of the available IP's as a normal router, then set a different box for spam filtering using the next available IP, but reading up on routed IP's, normally the first router will be just routing and not have NAT enabled, and this will be the default gateway for the remaining IP's.
I did a little messing around and couldn't seem to get that configuration working. Then I tried setting up a proxy ARP VIP, but didn't realise these don't respond to ICMP/ping so at the time I assumed it wasn't working.
Then on pfSense I set a 2nd WAN NIC up using the second IP with the default gateway for this as the WAN/1st IP. I managed to ping this from inside and outside, but I couldn't get a port forward to work (with hindsight I believe I didn't set the firewall on the WAN NIC to allow SMTP traffic through to then 2nd NIC which was likely the issue).
Also thinking about it, normally it is a cardinal sin to have two interfaces of a router on the same subnet, although the only purpose of 2nd WAN in this scenario is to allow SMTP through WAN and port forwarded to a mailserver, while to also have SMTP through WAN2/OPT and port forwarded to an anti spam box, which will then port forward to the actual mail server, all general outbound will be through WAN.
I left it at that point as time had run out late on Friday, but still left the box running as it seemed to be working fine as a router from WAN to LAN. First thing this morning, accessing internet from LAN was barely usable and ping to e.g. 4.2.2.1 had major packet loss so I removed the box altogether.
I'm now wondering if this is due to misconfiguration - should the WAN2/OPT interface actually have been a CARP or IP Alias interface?
However I did a little more testing - wiped the whole box, set up ESXi and pfSense again (this time just plugging WAN into the office network double NAT'd with the main router), and what I noticed was using the Broadcom NIC's onboard a Dell server, I was instantly getting something like 1% packet loss, but when I changed to an Intel 1000 MT, suddenly this disappeared.
I would be very grateful for any input as I'm certain the problem will be one of the two if not both, but I'm scared of trying again as a lot of important services run through that connection.