Routing problem - Newbee question

rumpelstilzchen

I am new to pfSense and I try to setup an openVPN tunnel. I have crazy problem with pfSense 2.1 BETA and openVPN. I defined a Server with the following:

UDP, tunnel mode and standard port

The LAN interface is 10.0.0.1, the local network is 10.0.0.0/8
The tunnel network is 10.0.1.0/26

I have a DNS Server with 10.1.0.5
I have a CentOS Server with 10.1.0.10

I can open the tunnel from my M$ Vista box without a problem. I can ping the DNS Server (10.1.0.5) successfully and also can open a VNC session from my Windows box through the tunnel.
But I CANNOT ping 10.1.0.10 through the tunnel!!

Both servers are in the same B-Network. What might be wrong here???

It only doesen't work through the tunnel. It works fine internally, e.g. I can ping 10.1.0.10 from the pfSense box and from any other box in the whole LAN network.

Any ideal what is missing here?

Rumpi

phil.davis

The LAN interface is 10.0.0.1, the local network is 10.0.0.0/8
The tunnel network is 10.0.1.0/26

The tunnel network is inside your LAN network. That can't work. 2 suggestions:

change the "/8" to a bigger number (smaller subnet). Unless you have a lot of devices on a single LAN, then use 10.1.0.0/24 and change your LAN interface to 10.1.0.1/24 and put all your devices/DHCP pool into the 10.1.0.2-254 range.
Move the tunnel network to some other range, away from anywhere that is likely to be used for the next LAN subnet. And make it a "/24" to keep it simple for mere mortals who look at a network diagram - 10.99.1.0/24 or whatever.

rumpelstilzchen

@phil.davis:

The LAN interface is 10.0.0.1, the local network is 10.0.0.0/8
The tunnel network is 10.0.1.0/26

The tunnel network is inside your LAN network. That can't work. 2 suggestions:

change the "/8" to a bigger number (smaller subnet). Unless you have a lot of devices on a single LAN, then use 10.1.0.0/24 and change your LAN interface to 10.1.0.1/24 and put all your devices/DHCP pool into the 10.1.0.2-254 range.

Move the tunnel network to some other range, away from anywhere that is likely to be used for the next LAN subnet. And make it a "/24" to keep it simple for mere mortals who look at a network diagram - 10.99.1.0/24 or whatever.

Hi Phil,

Thanks very much for your quick reply. I tried the following but still no luck:

LAN interface: 10.0.0.1, local network 10.0.0.0/9 (this means 10.0.0.1 - 10.127.255.254, correct?)
Tunnel network: 10.200.1.0/24

I can connect successfully and get 10.200.1.3 as address. But I can still only ping the two nameservers on 10.1.0.5 and 10.1.0.6. All other addresses get timeouts.

Does this mean I can only work with openVPN on C-networks and route from one C-network in another C-network?

Looks like I am missing some fundamental thing. Any help is greatly appreciated.

Rumpi

rumpelstilzchen

I tried a few more things but I cannot ping anything else then the DNS servers!!

I defined the following local networks:

10.0.0.0/24, 10.1.0.0/24, 10.2.1.0/24

I can access the pfSense webApplication through the tunnel on 10.0.0.1. I can ping the name servers which have 10.1.0.5 and 10.1.0.6 but cannot ping 10.2.1.197 which is a running server and can be pinged from one of the name servers.

I changed the local networks to 10.0.0.0/24, 10.2.1.0/24

As expected I cannot ping the two nameservers anymore but can access the web application. The hosts on the 10.2.1.0/24 subnet are not reachable, no ping, no vnc etc.

Any suggestions what could be wrong here??

Rumpi

phil.davis

LAN interface: 10.0.0.1, local network 10.0.0.0/9 (this means 10.0.0.1 - 10.127.255.254, correct?)
Tunnel network: 10.200.1.0/24

Yes, those subnet settings should work.
I am guessing that the things that do not respond to ping, cannot do so because of:
a) They have some firewall that blocks incoming from outside the local LAN; or
b) They do not have a default gateway set (so they can answer on the local LAN, but cannot route replies off the LAN); or
c) ???

marvosa

post your server1.conf and firewall rules from openvpn tab.

A network map will also be helpful.

rumpelstilzchen

Ok, I have CentOS 6.4 and Windows Server 2012 as operating systems. Most of the boxes are virtualized. We are currently testing Windows Server 2012 and Hyper-V 3 as host systems.

None of the boxes have currently iptables or a firewall or SELinux enabled.

The default gw is always set to 10.0.0.1 in the LAN

Currently I have the machines in the office simulating the datacenter situation. The LAN Adresses in the office are 172.16.63.0/24 and is connected to the WAN interface of the pfSense machine. So if I connect from my workstation in the office, I will be outside and connect through the WAN interface. To get easy access to the machines (also later in the datacenter) I need a tunnel (from the office it will be IPSEC), from notebook and homeoffice it will be OpenVPN. The internal LAN is and will be 10.0.0.0/9.

The exact segmentation of the internal LAN we do not know yet, but there will be a part for our internal use and a part for customers and a subnet for shared machines such as DNS Servers, Mail Gateways etc.

So currently we have one box with pfSense (will be a cluster in production). Its LAN gateway address is 10.0.0.1.
There will be a network 10.1.0.0/24 for shared servers (DNS etc.)
There will be a network 10.2.1.0/24 for internal servers
There will be a network 10.2.2.0/24 for demo and test servers
There will be a number of networks like 10.10.0.0/24, 10.10.1.0/24 etc for Customers

There is a tunnel network 10.200.1.0/24
There will be more tunnel networks on more VPN Servers like 10.200.2.0/24 with a different port

The idea behind that is, that there are administrators for internal machines and others for customers. They will use different VPNs. Some customers will have access to only 'their' LAN segments. Don't know yet, if it would be possible to define VLANs for that and then route them to the VLANs.

However interesting is the following routing table:

IPv4
Destination Gateway Flags Refs Use Mtu Netif Expire
default 172.16.63.1 UGS 0 169722 1500 igb1
8.8.8.8 172.16.63.1 UGHS 0 34 1500 igb1
10.0.0.0/9 link#1 U 0 232366 1500 igb0
10.0.0.1 link#1 UHS 0 58140 16384 lo0
10.1.0.5 10.0.0.1 UHS 0 6681 1500 igb0
10.1.0.6 10.0.0.1 UHS 0 2435 1500 igb0
10.200.1.0/24 10.200.1.1 UGS 0 3482 1500 ovpns1
10.200.1.1 link#8 UH 0 0 1500 ovpns1
127.0.0.1 link#5 UH 0 33 16384 lo0
172.16.0.0/16 link#2 U 0 33927 1500 igb1
172.16.63.120 link#2 UHS 0 0 16384 lo0

I am new to pfSense so I don't know where they are coming from. But I can ping the two name servers with the address 10.1.0.5 and 10.1.0.6 through the tunnel and also the gateway itself on 10.0.0.1. Basically thats all addresses apearing in the routing table.

So I am just wondering if there are routes missing or if I need to add some manually and if so, which ones?

Where can I find server1.conf?

The firewall rule was generated by the wizard and looks as follows:

VPN:
ID Proto Source Port Destination Port Gateway Queue Schedule Description
IPv4 * * * * * * none OpenVPN DCS Internes Admin VPN wizard

WAN:
ID Proto Source Port Destination Port Gateway Queue Schedule Description

Reserved/not assigned by IANA * * * * * * Block bogon networks
IPv4 UDP * * WAN address 1194 (OpenVPN) * none OpenVPN DCS Internes Admin VPN wizard

There are no other rules defined yet, no NAT etc.

Should I attach screen shots or where can I find the text representation of what is shown in the web application?

Thanks for your help.

Rumpi

marvosa

server1.conf is located in /var/etc/openvpn

rumpelstilzchen

Thanks for your help, I am not familiar with freeBSD but most of the linux commands seem to work. Here the content of my server1.conf:

dev ovpns1
dev-type tun
tun-ipv6
dev-node /dev/tun1
writepid /var/run/openvpn_server1.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp
cipher AES-256-CBC
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
client-connect /usr/local/sbin/openvpn.attributes.sh
client-disconnect /usr/local/sbin/openvpn.attributes.sh
local 172.16.63.120
tls-server
server 10.200.1.0 255.255.255.0
client-config-dir /var/etc/openvpn-csc
username-as-common-name
auth-user-pass-verify /var/etc/openvpn/server1.php via-env
tls-verify /var/etc/openvpn/server1.tls-verify.php
lport 1194
management /var/etc/openvpn/server1.sock unix
max-clients 10
push "route 10.2.1.0 255.255.255.0"
push "route 10.0.0.0 255.255.255.0"
push "route 10.1.0.0 255.255.255.0"
push "dhcp-option DOMAIN datacave.local"
push "dhcp-option DNS 10.1.0.5"
push "dhcp-option DNS 10.1.0.6"
push "dhcp-option DNS 8.8.8.8"
ca /var/etc/openvpn/server1.ca
cert /var/etc/openvpn/server1.cert
key /var/etc/openvpn/server1.key
dh /etc/dh-parameters.1024
tls-auth /var/etc/openvpn/server1.tls-auth 0
comp-lzo
persist-remote-ip
float
topology subnet

phil.davis

Exactly what is the LAN-side network?
You have 10.0.0.0/9 in your standard routing table, so I guess there is just 1 big flat LAN hanging off the pfSense LAN interface.
But then in your server conf file:

push "route 10.2.1.0 255.255.255.0"
push "route 10.0.0.0 255.255.255.0"
push "route 10.1.0.0 255.255.255.0"

You only push routes to little pieces of the 10.0.0.0/9
What LAN IPs do not ping from across the OpenVPN?

marvosa

Yes, this is not making sense.

You said your network is now 10.0.0.0/9, so why are you pushing routes to 10.2.1.0/24, 10.0.0.0/24 and 10.1.0.0/24? Those are all inside of 10.0.0.0/9… you've got something mixed up. Give us a network map, so we can see how you're connected and what you're trying to accomplish.

If your LAN is truly 10.0.0.0/9 then in your VPN config, under Tunnel Settings, your Local Network should only read 10.0.0.0/9…. does it?

If so, we should see this in your server1.conf -> push "route 10.0.0.0 255.128.0.0" but we don't.

Also, you are using split tunnel, so why are you pushing google's DNS to your clients? And your DNS servers are also inside your LAN, so I'm not sure why you're pushing those out either.

rumpelstilzchen

@phil.davis:

Exactly what is the LAN-side network?
You have 10.0.0.0/9 in your standard routing table, so I guess there is just 1 big flat LAN hanging off the pfSense LAN interface.
But then in your server conf file:
push "route 10.2.1.0 255.255.255.0"
push "route 10.0.0.0 255.255.255.0"
push "route 10.1.0.0 255.255.255.0"
You only push routes to little pieces of the 10.0.0.0/9
What LAN IPs do not ping from across the OpenVPN?

Yes as described above: 10.0.0.0/9 is the whole LAN which will be segmented later. Currently there are the following networks:

10.0.0.0/24 10.0.0.100-150 are used for DHCP (just for testing at the moment, later there will be no DHCP)
10.1.0.0/24 here are the currently two name servers which can be pinged
10.2.1.0/24 here are some servers, basically all virtual machines, windows and linux. NONE of these are reachable

So here again the details: From my Windows box I create a tunnel. I get a connection. ipconfig returns:

Windows-IP-Konfiguration

Ethernet-Adapter LAN-Verbindung 3:

Verbindungsspezifisches DNS-Suffix: datacave.local
IPv4-Adresse . . . . . . . . . . : 10.200.1.2
Subnetzmaske . . . . . . . . . . : 255.255.255.0
Standardgateway . . . . . . . . . :

I ping one of the nameservers:

C:\Users\tb>ping 10.1.0.5

Ping wird ausgeführt für 10.1.0.5 mit 32 Bytes Daten:
Antwort von 10.1.0.5: Bytes=32 Zeit<1ms TTL=63
Antwort von 10.1.0.5: Bytes=32 Zeit<1ms TTL=63
Antwort von 10.1.0.5: Bytes=32 Zeit<1ms TTL=63
Antwort von 10.1.0.5: Bytes=32 Zeit<1ms TTL=63

Ping-Statistik für 10.1.0.5:
Pakete: Gesendet = 4, Empfangen = 4, Verloren = 0 (0% Verlust),
Ca. Zeitangaben in Millisek.:
Minimum = 0ms, Maximum = 0ms, Mittelwert = 0ms

They answer successfully

I ping one of the servers in the 10.2.1.0/24 network. No answer!

C:\Users\tb>ping 10.2.1.199

Ping wird ausgeführt für 10.2.1.199 mit 32 Bytes Daten:
Zeitüberschreitung der Anforderung.
Zeitüberschreitung der Anforderung.
Zeitüberschreitung der Anforderung.
Zeitüberschreitung der Anforderung.

Ping-Statistik für 10.2.1.199:
Pakete: Gesendet = 4, Empfangen = 0, Verloren = 4 (100% Verlust),

rumpelstilzchen

@marvosa:

Yes, this is not making sense.

You said your network is now 10.0.0.0/9, so why are you pushing routes to 10.2.1.0/24, 10.0.0.0/24 and 10.1.0.0/24? Those are all inside of 10.0.0.0/9… you've got something mixed up. Give us a network map, so we can see how you're connected and what you're trying to accomplish.

If your LAN is truly 10.0.0.0/9 then in your VPN config, under Tunnel Settings, your Local Network should only read 10.0.0.0/9…. does it?

If so, we should see this in your server1.conf -> push "route 10.0.0.0 255.128.0.0" but we don't.

Also, you are using split tunnel, so why are you pushing google's DNS to your clients? And your DNS servers are also inside your LAN, so I'm not sure why you're pushing those out either.

Ok, looks like i misunderstand things and I need some help in the design of my network.

As already explained I have a big LAN, which is not yet partitionned finally. The idea is, that there are several subnets for server with special purpose. Then there are subnets for different customers, and there are subnets for ourselfes.

There should be different VPNs: Some Super-Admins can connect to the 'root' and see the whole network and can admin all machines there.
There should be other subnets, where only people can connect, who are allowed to see these machines.

Example:

A super admin should be able to connect to any machine in any subnet below 10.0.0.0/9

Lets say cutomer X has three virtual machines and some virtual desktops in a subnet 10.53.99.0/28. To access these machines, he should have a VPN, only showing him these 14 usable IP adresses.

The plan is to use openVPN for mobile/home office users and IPSec for network to network connections.

Is this not possible?

Rumpi

rumpelstilzchen

@marvosa:

Yes, this is not making sense.

You said your network is now 10.0.0.0/9, so why are you pushing routes to 10.2.1.0/24, 10.0.0.0/24 and 10.1.0.0/24? Those are all inside of 10.0.0.0/9… you've got something mixed up. Give us a network map, so we can see how you're connected and what you're trying to accomplish.

If your LAN is truly 10.0.0.0/9 then in your VPN config, under Tunnel Settings, your Local Network should only read 10.0.0.0/9…. does it?

If so, we should see this in your server1.conf -> push "route 10.0.0.0 255.128.0.0" but we don't.

Also, you are using split tunnel, so why are you pushing google's DNS to your clients? And your DNS servers are also inside your LAN, so I'm not sure why you're pushing those out either.

Ok, I made some changes. First forget about the google DNS server. It is a lab environment and at the beginning I had no working nameserver.

So here is the new configuration:

login as: root
Using keyboard-interactive authentication.
Password:
*** Welcome to pfSense 2.1-BETA1-pfSense (amd64) on dcsfire1 ***

WAN (wan) -> igb1 -> v4: 172.16.63.120/16
LAN (lan) -> igb0 -> v4: 10.0.0.1/9

Logout (SSH only) 8) Shell
Assign Interfaces 9) pfTop
Set interface(s) IP address 10) Filter Logs
Reset webConfigurator password 11) Restart webConfigurator
Reset to factory defaults 12) pfSense Developer Shell
Reboot system 13) Upgrade from console
Halt system 14) Disable Secure Shell (sshd)
Ping host 15) Restore recent configuration

Enter an option: 8

[2.1-BETA1][root@dcsfire1.datacave.biz]/var/etc/openvpn(51): cat server1.conf
dev ovpns1
dev-type tun
tun-ipv6
dev-node /dev/tun1
writepid /var/run/openvpn_server1.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp
cipher AES-256-CBC
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
client-connect /usr/local/sbin/openvpn.attributes.sh
client-disconnect /usr/local/sbin/openvpn.attributes.sh
local 172.16.63.120
tls-server
server 10.200.1.0 255.255.255.0
client-config-dir /var/etc/openvpn-csc
username-as-common-name
auth-user-pass-verify /var/etc/openvpn/server1.php via-env
tls-verify /var/etc/openvpn/server1.tls-verify.php
lport 1194
management /var/etc/openvpn/server1.sock unix
max-clients 10
push "route 10.0.0.0 255.128.0.0"
push "dhcp-option DOMAIN datacave.local"
push "dhcp-option DNS 10.1.0.5"
push "dhcp-option DNS 10.1.0.6"
ca /var/etc/openvpn/server1.ca
cert /var/etc/openvpn/server1.cert
key /var/etc/openvpn/server1.key
dh /etc/dh-parameters.1024
tls-auth /var/etc/openvpn/server1.tls-auth 0
comp-lzo
persist-remote-ip
float
topology subnet
[2.1-BETA1][root@dcsfire1.datacave.biz]/var/etc/openvpn(52):

So the whole big flat LAN should be accessible now. But after a restart of the openVPN server, it behaves still as before! I can ping 10.1.0.5 but NOT 10.2.1.199.

Rumpi

marvosa

What is the subnet mask of 10.2.1.199? And where is it located on the network?

rumpelstilzchen

Currently I only have two machines and a small switch. One of the machines is the IPFire Server.

The other box is a powerfull Win 2012 Server with Hyper-V 3 with 8 network ports. This machine is a host which will have many many VMs soon. Currently both of the networks are connected to a 'Virtual Switch' with 255.0.0.0 as subnet. The interface itself has 10.0.2.129. You can create many Virtual Switches and connect them to physical network ports which you connect then to a physical switch. Currently I have simply connected two cables to the physical switch. One is for the host computer and one is for the virtual switch.

Don't know, if this answers your question….

Rumpi

marvosa

Actually no. but we seem to get more new information with every post :)

You said you couldn't ping a device with an IP of 10.2.1.199…. I am asking what that is and where that device is located... (physical machine, vm, router, etc?) ... and what is the subnet mask configured on that device?

phil.davis

If 10.2.1.199 has a subnet mask 255.0.0.0 in it (/8 from the past) then it will think that the OpenVPN tunnel is part of the /8 and will expect it to be on the LAN, which it is not. Hopefully it is just simply fixing up the subnet mask on the unpingable machines to use /9.

rumpelstilzchen

@marvosa:

What is the subnet mask of 10.2.1.199? And where is it located on the network?

Hi Guys,

All Machines (the 'pingable' ones and those who do not answer) are CentOS boxes as VMs. The host is a Windows 2012 Server with 10.0.2.128/255.128.0.0 (I changed the netmask everywhere). Still no change in the situation. So I tried the ping and traceroute tools in pfSense. And thats showing the following results (see attached screenshots):

Is there any better way to see what is going on in the firewall? Are there any logs that contain information?

If I look at the openVPN log under system logs I only see successful connections:

May 21 08:20:14 openvpn[18150]: MAvpnAdmin/172.16.63.214:1194 send_push_reply(): safe_cap=940
May 21 08:20:12 openvpn[18150]: MAvpnAdmin/172.16.63.214:1194 MULTI_sva: pool returned IPv4=10.200.1.2, IPv6=(Not enabled)
May 21 08:20:12 openvpn[18150]: 172.16.63.214:1194 [MAvpnAdmin] Peer Connection Initiated with [AF_INET]172.16.63.214:1194
May 21 08:20:12 openvpn: user 'MAvpnAdmin' authenticated
May 21 07:24:16 openvpn[18150]: Initialization Sequence Completed
May 21 07:24:16 openvpn[18150]: UDPv4 link remote: [undef]
May 21 07:24:16 openvpn[18150]: UDPv4 link local (bound): [AF_INET]172.16.63.120:1194
May 21 07:24:16 openvpn[13606]: /usr/local/sbin/ovpn-linkup ovpns1 1500 1558 10.200.1.1 255.255.255.0 init
May 21 07:24:16 openvpn[13606]: /sbin/ifconfig ovpns1 10.200.1.1 10.200.1.1 mtu 1500 netmask 255.255.255.0 up
May 21 07:24:16 openvpn[13606]: do_ifconfig, tt->ipv6=1, tt->did_ifconfig_ipv6_setup=0
May 21 07:24:16 openvpn[13606]: TUN/TAP device /dev/tun1 opened
May 21 07:24:16 openvpn[13606]: TUN/TAP device ovpns1 exists previously, keep at program end
May 21 07:24:16 openvpn[13606]: Control Channel Authentication: using '/var/etc/openvpn/server1.tls-auth' as a OpenVPN static key file
May 21 07:24:16 openvpn[13606]: NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
May 21 07:24:16 openvpn[13606]: Could not retrieve default gateway from route socket:: No such process (errno=3)
May 21 07:24:16 openvpn[13606]: OpenVPN 2.3.1 amd64-portbld-freebsd8.3 [SSL (OpenSSL)] [LZO] [eurephia] [MH] [IPv6] built on May 6 2013
May 20 22:26:08 openvpn[16884]: tbVpnAdmin/172.16.63.102:1194 send_push_reply(): safe_cap=940
May 20 22:26:06 openvpn[16884]: tbVpnAdmin/172.16.63.102:1194 MULTI_sva: pool returned IPv4=10.200.1.2, IPv6=(Not enabled)
May 20 22:26:06 openvpn[16884]: 172.16.63.102:1194 [tbVpnAdmin] Peer Connection Initiated with [AF_INET]172.16.63.102:1194
May 20 22:26:06 openvpn: user 'tbVpnAdmin' authenticated

The firewall log may also be interesting:

block May 21 08:51:08 WAN 172.16.63.150:138 172.16.63.255:138 UDP
block May 21 08:50:00 WAN 172.16.63.111:138 172.16.63.255:138 UDP
block May 21 08:49:05 WAN 172.16.63.102:138 172.16.63.255:138 UDP
block May 21 08:48:04 WAN 172.16.63.204:138 172.16.63.255:138 UDP
block May 21 08:47:37 WAN 172.16.63.1:67 255.255.255.255:68 UDP
block May 21 08:47:37 WAN 0.0.0.0:68 255.255.255.255:67 UDP
block May 21 08:45:19 WAN 172.16.63.1:67 255.255.255.255:68 UDP
block May 21 08:45:19 WAN 0.0.0.0:68 255.255.255.255:67 UDP
block May 21 08:44:04 WAN 172.16.63.100:138 172.16.63.255:138 UDP
block May 21 08:44:04 WAN 172.16.63.100:138 172.16.63.255:138 UDP
block May 21 08:42:45 LAN 10.1.0.5:669 10.0.2.128:2049 TCP:RA
block May 21 08:42:43 WAN 172.16.63.214:137 172.16.63.255:137 UDP
block May 21 08:42:43 WAN 172.16.63.214:137 172.16.63.255:137 UDP
block May 21 08:42:42 WAN 172.16.63.214:137 172.16.63.255:137 UDP
block May 21 08:42:30 LAN 10.1.0.5:2990208737 10.0.2.128:2049 TCP:etatt
block May 21 08:42:21 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
block May 21 08:41:56 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
block May 21 08:41:43 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
block May 21 08:41:36 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
block May 21 08:41:33 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
block May 21 08:41:32 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
block May 21 08:41:31 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
block May 21 08:41:30 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
block May 21 08:41:30 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
block May 21 08:39:40 WAN 172.16.63.1:67 255.255.255.255:68 UDP

Just to summarize the IPs again:

pfSense box: WAN 172.16.63.120/16 (static address from our internal office LAN), Gateway 172.16.63.1 (a Fortinet Firewall)
LAN 10.0.0.0/8 (the LAN for all the datacenter servers), Gateway 10.0.0.1

Host1 (Windows 2012 server): 10.0.2.128/255.128.0.0 (This is one of the 8 NIC ports, connected to a physical switch)
10.0.2.129/255.128.0.0 (This is another NIC port, connected to the physical switch, used for the virtual Switch for Hyper-V 3. This switch then connects the virtual NICs of the VMs)

Now we have the following VM's all running on physical machine Host1:

10.1.0.5 ns1 DNS server running on CentOS 6.4
10.1.0.6 ns2 DNS server running on CentOS 6.4
10.2.1.193 - 199 several servers all running CentOS 6.4 working as web-, database- and application servers
10.2.1.129 - 135 several servers all running Windows 2012 woring as AD, RDS and other Windows servers

ALL OF THESE VMs are currently connected to the virtual switch described above with 10.0.2.129. However depending on the machines and applications, it is possible to create multiple virtual switches and assign different physical NICs to each switch. NIC teaming is also possible. I currently use none of these feature, it is currently as simple as possible.

Any more help would be greatly appreciated. Also the question if this could be a problem of the Beta release (I currently doubt that, but you never know). If so I would buy a LSI card and switch back to the official 2.0.3 release. I use the beta only for hw compatibility reasons.

Rumpi

Traceroute_DNS.jpg_thumb

Traceroute_WebServer.jpg_thumb

marvosa

pfSense box: WAN 172.16.63.120/16 (static address from our internal office LAN), Gateway 172.16.63.1 (a Fortinet Firewall)
LAN 10.0.0.0/8 (the LAN for all the datacenter servers), Gateway 10.0.0.1

Is this a typo? I thought this was changed to 10.0.0.0/9?

So, I'm not sure if you're specifically not answering the question or if I'm not being direct enough when I ask for the subnet mask. For instance, when you say:

10.1.0.5 ns1 DNS server running on CentOS 6.4
10.1.0.6 ns2 DNS server running on CentOS 6.4
10.2.1.193 - 199 several servers all running CentOS 6.4 working as web-, database- and application servers
10.2.1.129 - 135 several servers all running Windows 2012 woring as AD, RDS and other Windows servers

You still have not given us the masks for the servers you are trying to reach. You've given us the mask for the host machine, but not each guest. Double check the mask on each guest and report back.

It would also be helpful if you provided a network map, so we can see how things are physically connected. Also, where are you testing from?

Your firewall log is interesting. You shouldn't be getting blocks between 10.1.0.5 and 10.0.2.128 because they are on the same LAN… that traffic should not be hitting the firewall. Just another reason to double check connections and masks.