Additional statically-routed WAN subnets from ISP
-
Hi,
I’m having problems getting a statically routed subnet (y.y.y.64/26) which is delivered through my equipments WAN IP working correctly with pfSense. I’ve dug through pages of this forum, tried a number of different solutions, all which either don’t work in my case or are unsuitable.
My equipment is connected to my ISP provider through a fibre connection on IP subnet x.x.x.248/29. I have a gateway of x.x.x.249 and have previously set up my equipment to have an IP of x.x.x.250, allowing for failover by manually switching the fibre connection from one machine to the other. Here’s what I’ve tried with pfSense:
Attempt 1: Dirty hack: Setup pfSense WAN on y.y.y.64/26, and insert the following into the pfSense’s /etc/inc/interfaces.inc file, just before the line reading filter_configure():
_ system("/sbin/ifconfig em2 x.x.x.250 netmask 0xfffffff8 broadcast x.x.x.255 alias");
system("/sbin/route delete default");
system("/sbin/route add default x.x.x.249");_
This in theory adds the IP address for my ISP connection x.x.x.250 to the pfSense WAN interface, and forces the routing table to use the x.x.x.249 address as the gateway, allowing it to receive data for the x.x.x.248/29 and the y.y.y.64/26 subnet and send data back to the ISP. Surprisingly this actually worked; internet access from machines on the LAN worked fine. The only problem was incoming NAT/rules for CARP IP’s on y.y.y.64/26; the packets would enter, get passed by the rule, the HTTP server would receive them (confirmed via Wireshark/pfSense logs) but the return packets wouldn’t make it back out. I assume this is because the NAT/rule doesn’t use the default routing gateway, but the gateway specified in the WAN interface setup (which doesn’t accept the x.x.x.249 IP as it’s not in the y.y.y.64/26 subnet). Dead end (unless anyone out there knows which additional file to hack to force outgoing packets to x.x.x.249 :-).Attempt 2: VLAN / 1:1 NAT: I setup a VLAN on the WAN interface with subnet 10.10.80.64/26, did a 1:1 NAT from my IP subnet y.y.y.64/26 to 10.10.80.64, created an allow-all rule from WAN to VLAN, setup a CARP IP on the 10.10.80.64/26 and a NAT/rule for port 80 traffic on 10.10.80.100 to my end HTTP server. This didn’t work. Even more confusingly, the packets did enter, did get NAT’d to my HTTP server (confirmed via Wireshark/pfSense logs) but the return packet was being interpreted by pfSense as traffic originating from my HTTP server, as it created an additional outbound entry on the pfSense firewall log. Also, the really weird thing was that the requesting PC I was using for testing (which I was using through the same connection, but using a web anonymizer website to ensure it was working from the outside) was prompting for the username and password for my pfSense machine, and not anything to do with the content on my HTTP server. This suggests that pfSense is confused or not designed to perform this kind of thing???
Attempt 3: ProxyARP: I setup my pfSense WAN on x.x.x.248/29 with the address of x.x.x.250 and gateway x.x.x.249. I setup a ProxyARP on an IP in the y.y.y.64/26 subnet and NAT it to my HTTP server. This works, but as ProxyARP doesn’t work with the firewall, and isn’t suitable for my high-availabiltiy setup, hence it’s not an option.
Can anyone see or does anyone know how to correctly setup a WAN connection with an additional statically-routed IP subnet without using a router???
I’m also looking at the possibility of getting my ISP to VLAN tag the subnet/s, or add a gateway IP from the y.y.y.64/26 onto their interface which currently has the x.x.x.249 IP. Will post my reply when/if I get it…
Many thanks in advance for any help,
- Warwick
-
Usually your new IP block is routed to a specific IP address in your main block (e.g. x.x.x.250) you then use "Other" type Virtual IPs, and use them with NAT or whatever.
You could also assign an IP address from that block to another interface, and use only IPs from that block for devices there (as in a DMZ for public servers) - I think the new block still needs routed to your "main" IP in that case as well.
-
Ok…
So with help from a modified bridging script, kindly provided by Darth Android on post http://forum.pfsense.org/index.php/topic,19231.0.html I’ve finally got a working solution.
Step-by-step instructions for those who may need them:
- Install pfSense, specifying your IP connection providers settings (in my case this was a /29 subnet w/gateway IP)
- pfSense: Diagnostics -> Edit File: /usr/local/etc/rc.d/wan_bridge.sh. No point in pressing load, as file doesn’t exist yet. Files in this location get run after booting, installing the bridge and reloading the configuration each time.
- Paste in the script at the bottom of this post, changing the LOCAL_IFACE to your WAN adaptor and the VIRT_IFACE_MAC to something different from your WAN adaptor & press save.
- pfSense: Diagnostics -> Command: chmod 755 /usr/local/etc/rc.d/wan_bridge.sh. This makes our script file executable.
- Reboot.
- pfSense: Interfaces -> (assign), press the + in the bottom-right of the screen to show the new adaptor (should be ngeth0)
- pfSense: Interfaces -> Optional x
- Enter a name, your public IP range settings (in my case a /26 subnet), an IP for the interface in this range, enable & save.
- Reboot.
- Setup some CARP IP’s in the public IP range.
- Setup your NAT’s and rules to use WAN as the incoming interface, and specify CARP IP’s in the public range.
- Manual Outbound NAT’s can also be setup, using the CARP IP’s as the translation IP address, making outbound traffic appear from your public range.
Script below, thanks again to Darth for the main body of the script. The main modification was the addition of a few lines of PHP at the bottom, which reloads the pfSense settings after setting up the bridge.
Use at own risk!!!
#!/bin/sh
#A simple virtual interface script - USE AT OWN RISK
#Creates a virtual interface and bridges it with a physical interface.
#Author: darthandroid@gmail.com#User Variables - Modify these to suit your needs. Both need to be customized for the current system
#This is the name of the physical interface device. Look it up in `ifconfig' if you don't remember the name from when you configured pfSense
"WAN" is most likely NOT correct.
LOCAL_IFACE="eth0"
#This is the mac address of the new virtual interface. It should be different from the physical interface
VIRT_IFACE_MAC="00:00:00:00:00:00"Non-User code
BRIDGE="bridge0"
#create the bridge
ngctl mkpeer ${LOCAL_IFACE}: bridge lower link0 || exit 1
ngctl name ${LOCAL_IFACE}:lower ${BRIDGE}
#restore packet flow to the physical interface
ngctl connect ${BRIDGE}: ${LOCAL_IFACE}: link1 upper
#create virtual interface
ngctl mkpeer ${BRIDGE}: eiface link2 ether
#set virtual mac address and bring the interface up
ifconfig ngeth0 ether ${VIRT_IFACE_MAC}
ifconfig ngeth0 up
#make sure we can read packets from the physical interface directed to the virtual one and
#that we can write packets out without the virtual mac being overwritten
ngctl msg ${LOCAL_IFACE}: setautosrc 0
ngctl msg ${LOCAL_IFACE}: setpromisc 1
#do some php and reload some stuff
echo "" | php -a