Two servers, two internet ips two local ips = No forwarding

kragebein

Hello!
Im not sure if this is in the right category. But in some way its multiwan :P

I am runing two servers (virtually, on the same host).
pfsense and ubuntu server.

I have two internet IPs from my provider, both pfsense and my ubuntu server have one each.

They are also both connected to the 192.168.1.x network.

pfsense have and 192.168.1.9
ubuntu have and 192.168.1.6

The pfsense internet IP will be my public ip, the IP that everyone knows about (trough dyn-dns), and i want the ubuntu server to be on a DMZ with the real ip that it already have.

My problem:
I cannot forward a port (like port 22 and 80) from pfsense(192.168.1.9) to 192.168.1.6 - the forwarding just doesnt work.
If i request a NAT IP from pfsense with dhcp, the forward will work, but then i loose my connectivity on the ubuntu server, and the real IP on that server no longer works..

My question is:
How can i fix this? I assume that there is some sort of routing problem..
This code is run on the ubuntu server:

route


Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     192.168.1.9     255.255.255.255 UGH   0      0        0 eth0
192.168.1.0     *               255.255.255.0   U     0      0        0 eth0
109.189.137.0   *               255.255.255.0   U     0      0        0 eth1
default         mo-ctrl01.cdi.n 0.0.0.0         UG    0      0        0 eth1


ip route


192.168.1.0 via 192.168.1.9 dev eth0
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.6
109.189.137.0/24 dev eth1  proto kernel  scope link  src 109.189.137.246
default via 109.189.137.1 dev eth1


eth0      Link encap:Ethernet  HWaddr ee:b1:bd:c9:dc:97
          inet addr:192.168.1.6  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::ecb1:bdff:fec9:dc97/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6294612 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6185807 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4682472784 (4.6 GB)  TX bytes:3587207842 (3.5 GB)
          Interrupt:32 Base address:0x4000

eth1      Link encap:Ethernet  HWaddr 9e:5a:43:0e:9c:4c
          inet addr:109.189.137.246  Bcast:109.189.137.255  Mask:255.255.255.0
          inet6 addr: fe80::9c5a:43ff:fe0e:9c4c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:872511 errors:0 dropped:0 overruns:0 frame:0
          TX packets:278209 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:84693247 (84.6 MB)  TX bytes:203377450 (203.3 MB)
          Interrupt:36 Base address:0x2100

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1235 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1235 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:173470 (173.4 KB)  TX bytes:173470 (173.4 KB)

Do I need to add someting on pfsense to be able to forward ports to this server, or do I need to change something on the ubuntu server itself?

cmb

Your default route on Ubuntu must point to the LAN IP.
http://doc.pfsense.org/index.php/Port_Forward_Troubleshooting

kragebein

@cmb:

Your default route on Ubuntu must point to the LAN IP.
http://doc.pfsense.org/index.php/Port_Forward_Troubleshooting

But then I'd loose the connectivety on the IP i have on the ubuntu server.

I'll try anyhow.

cmb

oh it's dual homed on public and private. Then you need proper policy routing on the Ubuntu host to send traffic back out the correct NIC. I know that's possible, don't know how you do that on Linux offhand.

hikeonpast

Try this: http://www.ubuntugeek.com/howto-add-permanent-static-routes-in-ubuntu.html

I'm debugging a similar configuration, though I'm using the load balancer.

cmb

@hikeonpast:

Try this: http://www.ubuntugeek.com/howto-add-permanent-static-routes-in-ubuntu.html

I'm debugging a similar configuration, though I'm using the load balancer.

Static routes aren't adequate to achieve that AFAIK, you need policy routing to route traffic into eth0 back out of eth0's gateway, and traffic into eth1 out eth1's gateway. Can't just add routes as you'd have to have two default routes which leaves you in the same scenario basically.

SeventhSon

If i request a NAT IP from pfsense with dhcp, the forward will work, but then i loose my connectivity on the ubuntu server, and the real IP on that server no longer works..

DHCP overrides your default gateway on your linux box.
Shouldn't it just work when you put back the default gateway after that (or don't change it at all?)?

And why is this in there:

192.168.1.0 192.168.1.9 255.255.255.255 UGH 0 0 0 eth0

this should be enough:

192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.6
109.189.137.0/24 dev eth1 proto kernel scope link src 109.189.137.246
default via 109.189.137.1 dev eth1

Or am I missing something here…