Default gateway on a different subnet to host

[shawventures]

Hello everyone. I've been searching for a while now to try and solve my problem, however so far I have turned up nothing.

I have a hosted server which runs Proxmox for virtualisation. I currently have a debian based firewall protecting my virtual network. I have been trying to replace this with pfsense for a while. My problem is that I cant set the default gateway as it resides on a different subnet.

INTERNET
     |
IP.aaa.aaa.aaa  - (Public Ip of virtual machine)
     |
    pfsense        -(currently running fine with debian, config below)
     |
-------------------------------     -Virtual network, private addresses
|                  |               |
VM1             VM2            VM3

All of the above resides on a virtualised host, the host machine IP and the default gateway reside on the same subnet, the public IP's assigned to me do not.

This is the config that I am using on the Debian machine;

auto eth0
iface eth0 inet static
        address 94.aa.bb.cc
        netmask 255.255.255.255
        broadcast 94.aa.bb.cc

        post-up route add 91.xx.yy.254 dev eth0
        post-up route add default gw 91.xx.yy.254
        post-down route add 91.xx.yy.254 dev eth0
        post-down route add default gw 91.xx.yy.254

I hope some can help point me in the right direction.

Thanks in advance

[Cry Havok]

The default gateway must be on the local network.  There is absolutely no way to reach it otherwise.

[Brightwire]

I've got the same issue as shawventures has. This is for using pfSense as firewall in a virtual environment with OVH on one of their dedicated servers.

My short question is that, regardless of how stupid this seems, is there a way to add a default gateway on another network anyway?

The explanation for asking this is that though I'm not sure of how they do it, OVH (the people that shawventures is talking about, and the people who also provide my VMware ESXi server) have a system whereby if you put in a gateway on another network it does work without any other modification. I'm not entirely clear on how this works, but I do know that if you put in this configuration into Windows where it has (for example) an IP address of 178.32.32.100, a netmask of 255.255.255.255 and a default gateway of 199.21.21.254 it "just works".

So, can we add a default route rule manually like this?

Many thanks,

Colin

[wallabybob]

Suppose you have an interface 192.168.10.157/24 and you want to make 199.21.21.254/24 your default gateway.
How do you get from the 192.168.10.0/24 network to the 199.21.21.0/24 network? "Normally" this will require a router with an interface on the 192.168.10.0/24 network and an interface on the 199.21.21.0/24 network (say 192.168.10.1 and 199.21.21.1 respectively). If your network functions without such a router then there is probably something about it you haven't told us :-)

Therefore, on the 192.168.10.0/24 network you make 192.168.10.1 the default gateway.

Think of the default gateway as the system that gets you off the "local" network, not as the system with a physical connection to the internet.

Have I missed something?

[Cry Havok]

@wallabybob:

Think of the default gateway as the system that gets you off the "local" network, not as the system with a physical connection to the internet.

Have I missed something?

Not at all.

[Brightwire]

Can I give you an example of this working on Linux, and you could tell me if it was possible to do the same thing with pfSense?

When I set up a virtual machine on the VMware ESXi host that uses Linux, I configure it in pretty much the way that shawventures, who started this thread, mentioned (though I think he made a couple of typos).

To give you a concrete example: my VMware ESXi host has an IP address of (say) 91.121.183.101. I log into the datacentre's custom web interface to create a "Virtual Mac" address which allows them to route requests for another IP address such as (say) 178.32.70.132 to the VMware host. The instructions from the datacentre are to set up the VM as follows (this is for Linux):

1. Set the IP address to 178.32.70.132
2. Set the netmask to 255.255.255.255
3. Set the broadcast address to 178.32.70.132 (the same as the VM's IP)
4. Add the following routing rules:
post-up route add 91.121.183.254 dev eth0 # (91.121.183.254 being the same as the ESXi' host IP address but with 101 changed to 254)
post-up route add default gw 91.121.183.254
post-down route del 91.121.183.254 dev eth0
post-down route del default gw 91.121.183.254

This works. However, I'd like to add not Linux, but pfSense, as a guest VM to the ESXi host in the same way. I can happily set up most of this in pfSense, but don't know if it's possible to add the routing rules in step 4 above.

I'm not clear how Windows works out the route by itself, but with Windows I just do steps 1 - 3 above and it works. I'd be great if this is possible as it'd allow me to use pfSense as my firewall of choice within VMware rather than having to use Linux.

Many thanks,

Colin

[Brightwire]

Sorry - should have said that for Windows I do steps 1 to 2 and then set the default gateway to 91.121.183.254 and it just works.

Thanks,

Colin

[wallabybob]

I don't have any experience with VMWARE but I have played with VirtualBox.

You don't really say what you do with the Linux guest system but you seem to mention only one interface on it so I assume its an end-point for communication. pfSense is normally used as a router/firewall hence has at least two communications interfaces. Do you really want to replace an end-system by a router? If so, what is the router going to route?

Or do you want a pfSense guest virtual machine to take over some of the routing functons I presume are currently performed by the VMWARE host? If so, you probably need to talk to whoever has administrative responsibility for the VMWARE host to get them to setup appropriate "plumbing" linking the different virtual machines that are going to use pfSense as the router/firewall.

Your ethernet interface commands specify a network mask of 255.255.255.255 so I guess they are emulating a point-to-point link and use PPP. Correct? (If I recall correctly, in PPP initialisation each end can exchange IP addresses so they know the address of the system at the other end of the link.)

[Brightwire]

Thank you for your help.

The pfSense VM would have two virtual NICs in it - one connected to the outside world (set up in the same way as I outlined above), and one for a virtual internal network. The pfSense VM would act as firewall for a bunch of other VMs which would be connected only to the virtual internal network.

The Linux example is really only there to say that I can set up a Linux VM to connect in the way described. I'd like to connect pfSense in exactly the same way (though with 2 virtual NICs not one - one connected to the external network, and one to the virtual internal network - I just wanted to keep the example simple because the issue is about getting pfSense connected to the external network).

Apologies if I'm being a bit unclear here. I think that the key bit of information I need is this:

Can I add a routing rule in pfSense to in exactly the same way as I can in Linux? In Linux I just add the following:

**    post-up route add 91.xx.yy.254 dev eth0
    post-up route add default gw 91.xx.yy.254**

to the /etc/networking/interfaces file, and I'd like to be able to do something similar in pfSense. Is this possible?

Very much appreciated.

Colin

P.S. I'm afraid I just don't know the answer to your question about emulating PPP - sorry.

[Brightwire]

Hi All,

I've looked into this a little further and can now be a little more succinct in what I want to do. I have an example of how to use FreeBSD in this configuration, and as pfSense is based on FreeBSD I wonder whether it's a massive step to do the following.

The instructions I have are to update the rc.conf file as follows:

_  ifconfig_em0="inet178.32.70.132 netmask 255.255.255.255 broadcast 178.32.70.132"
    static_routes="net1 net2"
    route_net1="-net 91.121.183.254/32 178.32.70.132"
    route_net2="default 91.121.183.254_

I did try to add an rc.conf file to do this, but it seems to conflict (pfSense I think removes this file on startup). Is there a way to achieve this in pfSense? I feel it's almost there and just need to be able to add this route, but am unsure how to go about it.

Many thanks,

Colin

[wallabybob]

To manage static routes, in web GUI use System -> Static Routes (pfSense 1.2.3) or System -> Routing (pfSense 2.0 BETA). Click on the "+" at the bottom to add a route.

[Brightwire]

Hi Wallabybob,

Many thanks for your help and suggestions. Without meaning to sound daft, how would I go about adding a route here to replace the default gateway? When I try to type this into the static route page (I'm using 1.2.3), I get an error which says "A valid destination network must be specified". Any ideas?

Is it possible to create an rc.conf file that is not wiped out by pfSense when it restarts?

Many thanks,

Colin

[wallabybob]

@Brightwire:

Without meaning to sound daft, how would I go about adding a route here to replace the default gateway? When I try to type this into the static route page (I'm using 1.2.3), I get an error which says "A valid destination network must be specified". Any ideas?

I would choose em0 as my pfSense WAN interface. On configuring the interface set the Type to Static, set the Static IP configuration IP address to 178.32.70.132/32 and the gateway to 91.121.183.254. I did that on my running pfSense system, changing the interface type from DHCP to Static. Unfortunately I was left with no default route (no default route displayed in output of shell command netstat -rn). Perhaps if I rebooted I would get the correct default route through 91.121.183.254, but I'm not prepared to take my service down any longer so I'll leave that as an exercise for you to try. Around the time I changed the WAN interface the following was reported in the system log:

Sep 30 22:24:48    php: /interfaces_wan.php: No default gateway detected, adding 91.121.183.254
Sep 30 22:24:52    php: : No default gateway detected, adding 91.121.183.254
Sep 30 22:24:57    check_reload_status: reloading filter
Sep 30 22:25:01    root: Your DDNS records have been updated to
Sep 30 22:25:04    php: : No default gateway detected, adding 91.121.183.254

If you can't get this to work in pfSense 1.2.3 try it in pfSense 2.0 BETA.

@Brightwire:

Is it possible to create an rc.conf file that is not wiped out by pfSense when it restarts?

pfSense uses FreeBSD but it isn't FreeBSD, particularly in startup. Even if you could create an rc.conf that wasn't wiped out by pfSense on startup it may not help.

[aspineux]

This is a post I wrote on my blog. This is one working solution.

http://blog.magiksys.net/node/1

Regards.

Alain