Routing problem - Newbee question



  • I am new to pfSense and I try to setup an openVPN tunnel. I have crazy problem with pfSense 2.1 BETA and openVPN. I defined a Server with the following:

    UDP, tunnel mode and standard port

    The LAN interface is 10.0.0.1,  the local network is 10.0.0.0/8
    The tunnel network is 10.0.1.0/26

    I have a DNS Server with 10.1.0.5
    I have a CentOS Server with 10.1.0.10

    I can open the tunnel from my M$ Vista box without a problem. I can ping the DNS Server (10.1.0.5) successfully and also can open a VNC session from my Windows box through the tunnel.
    But I CANNOT ping 10.1.0.10 through the tunnel!!

    Both servers are in the same B-Network. What might be wrong here???

    It only doesen't work through the tunnel. It works fine internally, e.g. I can ping 10.1.0.10 from the pfSense box and from any other box in the whole LAN network.

    Any ideal what is missing here?

    Rumpi



  • The LAN interface is 10.0.0.1,  the local network is 10.0.0.0/8
    The tunnel network is 10.0.1.0/26

    The tunnel network is inside your LAN network. That can't work. 2 suggestions:

    1. change the "/8" to a bigger number (smaller subnet). Unless you have a lot of devices on a single LAN, then use 10.1.0.0/24 and change your LAN interface to 10.1.0.1/24 and put all your devices/DHCP pool into the 10.1.0.2-254 range.
    2. Move the tunnel network to some other range, away from anywhere that is likely to be used for the next LAN subnet. And make it a "/24" to keep it simple for mere mortals who look at a network diagram - 10.99.1.0/24 or whatever.


  • @phil.davis:

    The LAN interface is 10.0.0.1,  the local network is 10.0.0.0/8
    The tunnel network is 10.0.1.0/26

    The tunnel network is inside your LAN network. That can't work. 2 suggestions:

    1. change the "/8" to a bigger number (smaller subnet). Unless you have a lot of devices on a single LAN, then use 10.1.0.0/24 and change your LAN interface to 10.1.0.1/24 and put all your devices/DHCP pool into the 10.1.0.2-254 range.
    2. Move the tunnel network to some other range, away from anywhere that is likely to be used for the next LAN subnet. And make it a "/24" to keep it simple for mere mortals who look at a network diagram - 10.99.1.0/24 or whatever.

    Hi Phil,

    Thanks very much for your quick reply. I tried the following but still no luck:

    LAN interface: 10.0.0.1, local network 10.0.0.0/9  (this means 10.0.0.1 - 10.127.255.254, correct?)
    Tunnel network: 10.200.1.0/24

    I can connect successfully and get 10.200.1.3 as address. But I can still only ping the two nameservers on 10.1.0.5 and 10.1.0.6. All other addresses get timeouts.

    Does this mean I can only work with openVPN on C-networks and route from one C-network in another C-network?

    Looks like I am missing some fundamental thing. Any help is greatly appreciated.

    Rumpi



  • I tried a few more things but I cannot ping anything else then the DNS servers!!

    I defined the following local networks:

    10.0.0.0/24, 10.1.0.0/24, 10.2.1.0/24

    I can access the pfSense webApplication through the tunnel on 10.0.0.1. I can ping the name servers which have 10.1.0.5 and 10.1.0.6 but cannot ping 10.2.1.197 which is a running server and can be pinged from one of the name servers.

    I changed the local networks to 10.0.0.0/24, 10.2.1.0/24

    As expected I cannot ping the two nameservers anymore but can access the web application. The hosts on the 10.2.1.0/24 subnet are not reachable, no ping, no vnc etc.

    Any suggestions what could be wrong here??

    Rumpi



  • LAN interface: 10.0.0.1, local network 10.0.0.0/9  (this means 10.0.0.1 - 10.127.255.254, correct?)
    Tunnel network: 10.200.1.0/24

    Yes, those subnet settings should work.
    I am guessing that the things that do not respond to ping, cannot do so because of:
    a) They have some firewall that blocks incoming from outside the local LAN; or
    b) They do not have a default gateway set (so they can answer on the local LAN, but cannot route replies off the LAN); or
    c) ???



  • post your server1.conf and firewall rules from openvpn tab.

    A network map will also be helpful.



  • Ok, I have CentOS 6.4 and Windows Server 2012 as operating systems. Most of the boxes are virtualized. We are currently testing Windows Server 2012 and Hyper-V 3 as host systems.

    None of the boxes have currently iptables or a firewall or SELinux enabled.

    The default gw is always set to 10.0.0.1 in the LAN

    Currently I have the machines in the office simulating the datacenter situation. The LAN Adresses in the office are 172.16.63.0/24 and is connected to the WAN interface of the pfSense machine. So if I connect from my workstation in the office, I will be outside and connect through the WAN interface. To get easy access to the machines (also later in the datacenter) I need a tunnel (from the office it will be IPSEC), from notebook and homeoffice it will be OpenVPN. The internal LAN is and will be 10.0.0.0/9.

    The exact segmentation of the internal LAN we do not know yet, but there will be a part for our internal use and a part for customers and a subnet for shared machines such as DNS Servers, Mail Gateways etc.

    So currently we have one box with pfSense (will be a cluster in production). Its LAN gateway address is 10.0.0.1.
    There will be a network 10.1.0.0/24 for shared servers (DNS etc.)
    There will be a network 10.2.1.0/24 for internal servers
    There will be a network 10.2.2.0/24 for demo and test servers
    There will be a number of networks like 10.10.0.0/24, 10.10.1.0/24 etc for Customers

    There is a tunnel network 10.200.1.0/24
    There will be more tunnel networks on more VPN Servers like 10.200.2.0/24 with a different port

    The idea behind that is, that there are administrators for internal machines and others for customers. They will use different VPNs. Some customers will have access to only 'their' LAN segments. Don't know yet, if it would be possible to define VLANs for that and then route them to the VLANs.

    However interesting is the following routing table:

    IPv4
    Destination Gateway Flags Refs Use Mtu Netif Expire
    default 172.16.63.1 UGS 0 169722 1500 igb1
    8.8.8.8 172.16.63.1 UGHS 0 34 1500 igb1
    10.0.0.0/9 link#1 U 0 232366 1500 igb0
    10.0.0.1 link#1 UHS 0 58140 16384 lo0
    10.1.0.5 10.0.0.1 UHS 0 6681 1500 igb0
    10.1.0.6 10.0.0.1 UHS 0 2435 1500 igb0
    10.200.1.0/24 10.200.1.1 UGS 0 3482 1500 ovpns1
    10.200.1.1 link#8 UH 0 0 1500 ovpns1
    127.0.0.1 link#5 UH 0 33 16384 lo0
    172.16.0.0/16 link#2 U 0 33927 1500 igb1
    172.16.63.120 link#2 UHS 0 0 16384 lo0

    I am new to pfSense so I don't know where they are coming from. But I can ping the two name servers with the address 10.1.0.5 and 10.1.0.6 through the tunnel and also the gateway itself on 10.0.0.1. Basically thats all addresses apearing in the routing table.

    So I am just wondering if there are routes missing or if I need to add some manually and if so, which ones?

    Where can I find server1.conf?

    The firewall rule was generated by the wizard and looks as follows:

    VPN:
    ID Proto Source Port Destination Port Gateway Queue Schedule Description
    IPv4 * * * * * * none   OpenVPN DCS Internes Admin VPN wizard

    WAN:
    ID Proto Source Port Destination Port Gateway Queue Schedule Description

    • Reserved/not assigned by IANA * * * * * * Block bogon networks
      IPv4 UDP * * WAN address 1194 (OpenVPN) * none   OpenVPN DCS Internes Admin VPN wizard

    There are no other rules defined yet, no NAT etc.

    Should I attach screen shots or where can I find the text representation of what is shown in the web application?

    Thanks for your help.

    Rumpi



  • server1.conf is located in /var/etc/openvpn



  • Thanks for your help, I am not familiar with freeBSD but most of the linux commands seem to work. Here the content of my server1.conf:

    dev ovpns1
    dev-type tun
    tun-ipv6
    dev-node /dev/tun1
    writepid /var/run/openvpn_server1.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp
    cipher AES-256-CBC
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    client-connect /usr/local/sbin/openvpn.attributes.sh
    client-disconnect /usr/local/sbin/openvpn.attributes.sh
    local 172.16.63.120
    tls-server
    server 10.200.1.0 255.255.255.0
    client-config-dir /var/etc/openvpn-csc
    username-as-common-name
    auth-user-pass-verify /var/etc/openvpn/server1.php via-env
    tls-verify /var/etc/openvpn/server1.tls-verify.php
    lport 1194
    management /var/etc/openvpn/server1.sock unix
    max-clients 10
    push "route 10.2.1.0 255.255.255.0"
    push "route 10.0.0.0 255.255.255.0"
    push "route 10.1.0.0 255.255.255.0"
    push "dhcp-option DOMAIN datacave.local"
    push "dhcp-option DNS 10.1.0.5"
    push "dhcp-option DNS 10.1.0.6"
    push "dhcp-option DNS 8.8.8.8"
    ca /var/etc/openvpn/server1.ca
    cert /var/etc/openvpn/server1.cert
    key /var/etc/openvpn/server1.key
    dh /etc/dh-parameters.1024
    tls-auth /var/etc/openvpn/server1.tls-auth 0
    comp-lzo
    persist-remote-ip
    float
    topology subnet



  • Exactly what is the LAN-side network?
    You have 10.0.0.0/9 in your standard routing table, so I guess there is just 1 big flat LAN hanging off the pfSense LAN interface.
    But then in your server conf file:

    push "route 10.2.1.0 255.255.255.0"
    push "route 10.0.0.0 255.255.255.0"
    push "route 10.1.0.0 255.255.255.0"
    

    You only push routes to little pieces of the 10.0.0.0/9
    What LAN IPs do not ping from across the OpenVPN?



  • Yes, this is not making sense.

    You said your network is now 10.0.0.0/9, so why are you pushing routes to 10.2.1.0/24, 10.0.0.0/24 and 10.1.0.0/24?  Those are all inside of 10.0.0.0/9… you've got something mixed up.  Give us a network map, so we can see how you're connected and what you're trying to accomplish.

    If your LAN is truly 10.0.0.0/9 then in your VPN config, under Tunnel Settings, your Local Network should only read 10.0.0.0/9…. does it?

    If so, we should see this in your server1.conf -> push "route 10.0.0.0 255.128.0.0" but we don't.

    Also, you are using split tunnel, so why are you pushing google's DNS to your clients?  And your DNS servers are also inside your LAN, so I'm not sure why you're pushing those out either.



  • @phil.davis:

    Exactly what is the LAN-side network?
    You have 10.0.0.0/9 in your standard routing table, so I guess there is just 1 big flat LAN hanging off the pfSense LAN interface.
    But then in your server conf file:

    push "route 10.2.1.0 255.255.255.0"
    push "route 10.0.0.0 255.255.255.0"
    push "route 10.1.0.0 255.255.255.0"
    

    You only push routes to little pieces of the 10.0.0.0/9
    What LAN IPs do not ping from across the OpenVPN?

    Yes as described above: 10.0.0.0/9 is the whole LAN which will be segmented later. Currently there are the following networks:

    10.0.0.0/24  10.0.0.100-150 are used for DHCP (just for testing at the moment, later there will be no DHCP)
    10.1.0.0/24  here are the currently two name servers which can be pinged
    10.2.1.0/24  here are some servers, basically all virtual machines, windows and linux. NONE of these are reachable

    So here again the details: From my Windows box I create a tunnel. I get a connection. ipconfig returns:

    Windows-IP-Konfiguration

    Ethernet-Adapter LAN-Verbindung 3:

    Verbindungsspezifisches DNS-Suffix: datacave.local
      IPv4-Adresse  . . . . . . . . . . : 10.200.1.2
      Subnetzmaske  . . . . . . . . . . : 255.255.255.0
      Standardgateway . . . . . . . . . :

    I ping one of the nameservers:

    C:\Users\tb>ping 10.1.0.5

    Ping wird ausgeführt für 10.1.0.5 mit 32 Bytes Daten:
    Antwort von 10.1.0.5: Bytes=32 Zeit<1ms TTL=63
    Antwort von 10.1.0.5: Bytes=32 Zeit<1ms TTL=63
    Antwort von 10.1.0.5: Bytes=32 Zeit<1ms TTL=63
    Antwort von 10.1.0.5: Bytes=32 Zeit<1ms TTL=63

    Ping-Statistik für 10.1.0.5:
       Pakete: Gesendet = 4, Empfangen = 4, Verloren = 0 (0% Verlust),
    Ca. Zeitangaben in Millisek.:
       Minimum = 0ms, Maximum = 0ms, Mittelwert = 0ms

    They answer successfully

    I ping one of the servers in the 10.2.1.0/24 network. No answer!

    C:\Users\tb>ping 10.2.1.199

    Ping wird ausgeführt für 10.2.1.199 mit 32 Bytes Daten:
    Zeitüberschreitung der Anforderung.
    Zeitüberschreitung der Anforderung.
    Zeitüberschreitung der Anforderung.
    Zeitüberschreitung der Anforderung.

    Ping-Statistik für 10.2.1.199:
       Pakete: Gesendet = 4, Empfangen = 0, Verloren = 4 (100% Verlust),



  • @marvosa:

    Yes, this is not making sense.

    You said your network is now 10.0.0.0/9, so why are you pushing routes to 10.2.1.0/24, 10.0.0.0/24 and 10.1.0.0/24?  Those are all inside of 10.0.0.0/9… you've got something mixed up.  Give us a network map, so we can see how you're connected and what you're trying to accomplish.

    If your LAN is truly 10.0.0.0/9 then in your VPN config, under Tunnel Settings, your Local Network should only read 10.0.0.0/9…. does it?

    If so, we should see this in your server1.conf -> push "route 10.0.0.0 255.128.0.0" but we don't.

    Also, you are using split tunnel, so why are you pushing google's DNS to your clients?  And your DNS servers are also inside your LAN, so I'm not sure why you're pushing those out either.

    Ok, looks like i misunderstand things and I need some help in the design of my network.

    As already explained I have a big LAN, which is not yet partitionned finally. The idea is, that there are several subnets for server with special purpose. Then there are subnets for different customers, and there are subnets for ourselfes.

    There should be different VPNs: Some Super-Admins can connect to the 'root' and see the whole network and can admin all machines there.
    There should be other subnets, where only people can connect, who are allowed to see these machines.

    Example:

    A super admin should be able to connect to any machine in any subnet below 10.0.0.0/9

    Lets say cutomer X has three virtual machines and some virtual desktops in a subnet 10.53.99.0/28. To access these machines, he should have a VPN, only showing him these 14 usable IP adresses.

    The plan is to use openVPN for mobile/home office users and IPSec for network to network connections.

    Is this not possible?

    Rumpi



  • @marvosa:

    Yes, this is not making sense.

    You said your network is now 10.0.0.0/9, so why are you pushing routes to 10.2.1.0/24, 10.0.0.0/24 and 10.1.0.0/24?  Those are all inside of 10.0.0.0/9… you've got something mixed up.  Give us a network map, so we can see how you're connected and what you're trying to accomplish.

    If your LAN is truly 10.0.0.0/9 then in your VPN config, under Tunnel Settings, your Local Network should only read 10.0.0.0/9…. does it?

    If so, we should see this in your server1.conf -> push "route 10.0.0.0 255.128.0.0" but we don't.

    Also, you are using split tunnel, so why are you pushing google's DNS to your clients?  And your DNS servers are also inside your LAN, so I'm not sure why you're pushing those out either.

    Ok, I made some changes. First forget about the google DNS server. It is a lab environment and at the beginning I had no working nameserver.

    So here is the new configuration:

    login as: root
    Using keyboard-interactive authentication.
    Password:
    *** Welcome to pfSense 2.1-BETA1-pfSense (amd64) on dcsfire1 ***

    WAN (wan)      -> igb1      -> v4: 172.16.63.120/16
    LAN (lan)      -> igb0      -> v4: 10.0.0.1/9

    1. Logout (SSH only)                  8) Shell
    2. Assign Interfaces                  9) pfTop
    3. Set interface(s) IP address      10) Filter Logs
    4. Reset webConfigurator password    11) Restart webConfigurator
    5. Reset to factory defaults        12) pfSense Developer Shell
    6. Reboot system                    13) Upgrade from console
    7. Halt system                      14) Disable Secure Shell (sshd)
    8. Ping host                        15) Restore recent configuration

    Enter an option: 8

    [2.1-BETA1][root@dcsfire1.datacave.biz]/var/etc/openvpn(51): cat server1.conf
    dev ovpns1
    dev-type tun
    tun-ipv6
    dev-node /dev/tun1
    writepid /var/run/openvpn_server1.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp
    cipher AES-256-CBC
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    client-connect /usr/local/sbin/openvpn.attributes.sh
    client-disconnect /usr/local/sbin/openvpn.attributes.sh
    local 172.16.63.120
    tls-server
    server 10.200.1.0 255.255.255.0
    client-config-dir /var/etc/openvpn-csc
    username-as-common-name
    auth-user-pass-verify /var/etc/openvpn/server1.php via-env
    tls-verify /var/etc/openvpn/server1.tls-verify.php
    lport 1194
    management /var/etc/openvpn/server1.sock unix
    max-clients 10
    push "route 10.0.0.0 255.128.0.0"
    push "dhcp-option DOMAIN datacave.local"
    push "dhcp-option DNS 10.1.0.5"
    push "dhcp-option DNS 10.1.0.6"
    ca /var/etc/openvpn/server1.ca
    cert /var/etc/openvpn/server1.cert
    key /var/etc/openvpn/server1.key
    dh /etc/dh-parameters.1024
    tls-auth /var/etc/openvpn/server1.tls-auth 0
    comp-lzo
    persist-remote-ip
    float
    topology subnet
    [2.1-BETA1][root@dcsfire1.datacave.biz]/var/etc/openvpn(52):

    So the whole big flat LAN should be accessible now. But after a restart of the openVPN server, it behaves still as before! I can ping 10.1.0.5 but NOT 10.2.1.199.

    Rumpi



  • What is the subnet mask of 10.2.1.199?  And where is it located on the network?



  • Currently I only have two machines and a small switch. One of the machines is the IPFire Server.

    The other box is a powerfull Win 2012 Server with Hyper-V 3 with 8 network ports. This machine is a host which will have many many VMs soon. Currently both of the networks are connected to a 'Virtual Switch' with 255.0.0.0 as subnet. The interface itself has 10.0.2.129. You can create many Virtual Switches and connect them to physical network ports which you connect then to a physical switch. Currently I have simply connected two cables to the physical switch. One is for the host computer and one is for the virtual switch.

    Don't know, if this answers your question….

    Rumpi



  • Actually no.  but we seem to get more new information with every post :)

    You said you couldn't ping a device with an IP of 10.2.1.199….  I am asking what that is and where that device is located... (physical machine, vm, router, etc?) ... and what is the subnet mask configured on that device?



  • If 10.2.1.199 has a subnet mask 255.0.0.0 in it (/8 from the past) then it will think that the OpenVPN tunnel is part of the /8 and will expect it to be on the LAN,  which it is not. Hopefully it is just simply fixing up the subnet mask on the unpingable machines to use /9.



  • @marvosa:

    What is the subnet mask of 10.2.1.199?  And where is it located on the network?

    Hi Guys,

    All Machines (the 'pingable' ones and those who do not answer) are CentOS boxes as VMs. The host is a Windows 2012 Server with 10.0.2.128/255.128.0.0 (I changed the netmask everywhere). Still no change in the situation. So I tried the ping and traceroute tools in pfSense. And thats showing the following results  (see attached screenshots):

    Is there any better way to see what is going on in the firewall? Are there any logs that contain information?

    If I look at the openVPN log under system logs I only see successful connections:

    May 21 08:20:14 openvpn[18150]: MAvpnAdmin/172.16.63.214:1194 send_push_reply(): safe_cap=940
    May 21 08:20:12 openvpn[18150]: MAvpnAdmin/172.16.63.214:1194 MULTI_sva: pool returned IPv4=10.200.1.2, IPv6=(Not enabled)
    May 21 08:20:12 openvpn[18150]: 172.16.63.214:1194 [MAvpnAdmin] Peer Connection Initiated with [AF_INET]172.16.63.214:1194
    May 21 08:20:12 openvpn: user 'MAvpnAdmin' authenticated
    May 21 07:24:16 openvpn[18150]: Initialization Sequence Completed
    May 21 07:24:16 openvpn[18150]: UDPv4 link remote: [undef]
    May 21 07:24:16 openvpn[18150]: UDPv4 link local (bound): [AF_INET]172.16.63.120:1194
    May 21 07:24:16 openvpn[13606]: /usr/local/sbin/ovpn-linkup ovpns1 1500 1558 10.200.1.1 255.255.255.0 init
    May 21 07:24:16 openvpn[13606]: /sbin/ifconfig ovpns1 10.200.1.1 10.200.1.1 mtu 1500 netmask 255.255.255.0 up
    May 21 07:24:16 openvpn[13606]: do_ifconfig, tt->ipv6=1, tt->did_ifconfig_ipv6_setup=0
    May 21 07:24:16 openvpn[13606]: TUN/TAP device /dev/tun1 opened
    May 21 07:24:16 openvpn[13606]: TUN/TAP device ovpns1 exists previously, keep at program end
    May 21 07:24:16 openvpn[13606]: Control Channel Authentication: using '/var/etc/openvpn/server1.tls-auth' as a OpenVPN static key file
    May 21 07:24:16 openvpn[13606]: NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
    May 21 07:24:16 openvpn[13606]: Could not retrieve default gateway from route socket:: No such process (errno=3)
    May 21 07:24:16 openvpn[13606]: OpenVPN 2.3.1 amd64-portbld-freebsd8.3 [SSL (OpenSSL)] [LZO] [eurephia] [MH] [IPv6] built on May 6 2013
    May 20 22:26:08 openvpn[16884]: tbVpnAdmin/172.16.63.102:1194 send_push_reply(): safe_cap=940
    May 20 22:26:06 openvpn[16884]: tbVpnAdmin/172.16.63.102:1194 MULTI_sva: pool returned IPv4=10.200.1.2, IPv6=(Not enabled)
    May 20 22:26:06 openvpn[16884]: 172.16.63.102:1194 [tbVpnAdmin] Peer Connection Initiated with [AF_INET]172.16.63.102:1194
    May 20 22:26:06 openvpn: user 'tbVpnAdmin' authenticated

    The firewall log may also be interesting:

    block May 21 08:51:08 WAN 172.16.63.150:138 172.16.63.255:138 UDP
    block May 21 08:50:00 WAN 172.16.63.111:138 172.16.63.255:138 UDP
    block May 21 08:49:05 WAN 172.16.63.102:138 172.16.63.255:138 UDP
    block May 21 08:48:04 WAN 172.16.63.204:138 172.16.63.255:138 UDP
    block May 21 08:47:37 WAN 172.16.63.1:67 255.255.255.255:68 UDP
    block May 21 08:47:37 WAN 0.0.0.0:68 255.255.255.255:67 UDP
    block May 21 08:45:19 WAN 172.16.63.1:67 255.255.255.255:68 UDP
    block May 21 08:45:19 WAN 0.0.0.0:68 255.255.255.255:67 UDP
    block May 21 08:44:04 WAN 172.16.63.100:138 172.16.63.255:138 UDP
    block May 21 08:44:04 WAN 172.16.63.100:138 172.16.63.255:138 UDP
    block May 21 08:42:45 LAN 10.1.0.5:669 10.0.2.128:2049 TCP:RA
    block May 21 08:42:43 WAN 172.16.63.214:137 172.16.63.255:137 UDP
    block May 21 08:42:43 WAN 172.16.63.214:137 172.16.63.255:137 UDP
    block May 21 08:42:42 WAN 172.16.63.214:137 172.16.63.255:137 UDP
    block May 21 08:42:30 LAN 10.1.0.5:2990208737 10.0.2.128:2049 TCP:etatt
    block May 21 08:42:21 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
    block May 21 08:41:56 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
    block May 21 08:41:43 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
    block May 21 08:41:36 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
    block May 21 08:41:33 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
    block May 21 08:41:32 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
    block May 21 08:41:31 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
    block May 21 08:41:30 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
    block May 21 08:41:30 LAN 10.1.0.5:2973431521 10.0.2.128:2049 TCP:etatt
    block May 21 08:39:40 WAN 172.16.63.1:67 255.255.255.255:68 UDP

    Just to summarize the IPs again:

    pfSense box: WAN 172.16.63.120/16  (static address from our internal office LAN), Gateway 172.16.63.1 (a Fortinet Firewall)
                      LAN  10.0.0.0/8  (the LAN for all the datacenter servers), Gateway 10.0.0.1

    Host1 (Windows 2012 server):  10.0.2.128/255.128.0.0  (This is one of the 8 NIC ports, connected to a physical switch)
                                              10.0.2.129/255.128.0.0  (This is another NIC port, connected to the physical switch, used for the virtual Switch for Hyper-V 3. This switch then connects the virtual NICs of the VMs)

    Now we have the following VM's all running on physical machine Host1:

    10.1.0.5    ns1  DNS server running on CentOS 6.4
    10.1.0.6    ns2  DNS server running on CentOS 6.4
    10.2.1.193 - 199  several servers all running CentOS 6.4 working as web-, database- and application servers
    10.2.1.129 - 135  several servers all running Windows 2012 woring as AD, RDS and other Windows servers

    ALL OF THESE VMs are currently connected to the virtual switch described above with 10.0.2.129. However depending on the machines and applications, it is possible to create multiple virtual switches and assign different physical NICs to each switch. NIC teaming is also possible. I currently use none of these feature, it is currently as simple as possible.

    Any more help would be greatly appreciated.  Also the question if this could be a problem of the Beta release (I currently doubt that, but you never know). If so I would buy a LSI card and switch back to the official 2.0.3 release. I use the beta only for hw compatibility reasons.

    Rumpi






  • pfSense box: WAN 172.16.63.120/16  (static address from our internal office LAN), Gateway 172.16.63.1 (a Fortinet Firewall)
                      LAN  10.0.0.0/8  (the LAN for all the datacenter servers), Gateway 10.0.0.1

    Is this a typo?  I thought this was changed to 10.0.0.0/9?

    So, I'm not sure if you're specifically not answering the question or if I'm not being direct enough when I ask for the subnet mask.  For instance, when you say:

    10.1.0.5    ns1  DNS server running on CentOS 6.4
    10.1.0.6    ns2  DNS server running on CentOS 6.4
    10.2.1.193 - 199  several servers all running CentOS 6.4 working as web-, database- and application servers
    10.2.1.129 - 135  several servers all running Windows 2012 woring as AD, RDS and other Windows servers

    You still have not given us the masks for the servers you are trying to reach.  You've given us the mask for the host machine, but not each guest.  Double check the mask on each guest and report back.

    It would also be helpful if you provided a network map, so we can see how things are physically connected.  Also, where are you testing from?

    Your firewall log is interesting.  You shouldn't be getting blocks between 10.1.0.5 and 10.0.2.128 because they are on the same LAN… that traffic should not be hitting the firewall.  Just another reason to double check connections and masks.



  • @marvosa:

    pfSense box: WAN 172.16.63.120/16   (static address from our internal office LAN), Gateway 172.16.63.1 (a Fortinet Firewall)
                       LAN  10.0.0.0/8  (the LAN for all the datacenter servers), Gateway 10.0.0.1

    Is this a typo?  I thought this was changed to 10.0.0.0/9?

    Sorry this is a typo, I changed all to 255.128.0.0

    @marvosa:

    So, I'm not sure if you're specifically not answering the question or if I'm not being direct enough when I ask for the subnet mask.  For instance, when you say:

    10.1.0.5     ns1   DNS server running on CentOS 6.4
    10.1.0.6     ns2   DNS server running on CentOS 6.4
    10.2.1.193 - 199  several servers all running CentOS 6.4 working as web-, database- and application servers
    10.2.1.129 - 135  several servers all running Windows 2012 woring as AD, RDS and other Windows servers

    You still have not given us the masks for the servers you are trying to reach.   You've given us the mask for the host machine, but not each guest.  Double check the mask on each guest and report back.

    It would also be helpful if you provided a network map, so we can see how things are physically connected.  Also, where are you testing from?

    Your firewall log is interesting.  You shouldn't be getting blocks between 10.1.0.5 and 10.0.2.128 because they are on the same LAN… that traffic should not be hitting the firewall.  Just another reason to double check connections and masks.

    Sorry, I currently don't have a map, I need a tool for that like Nagios. But I haven't installed yet.

    BUT I THINK YOU GOT IT! At least I found one machine with 255.0.0.0 instead of 255.128.0.0 running CentOS. I changed that and it is answering now on pings!! I have a Windows box setup correctly which does not anser. I will have to check the other machines, check local firewalls on windows and reboot all the boxes and see if it is working. I will give feedback later.  Hopefully one major problem is solved.

    First of all thanks a lot for your help! Coming back soon with the results.

    Rumpi



  • Hi Guys, seems that the problem is solved. The main problem was to have the tunnel network inside the LAN which does not work. Reducing the LAN network and placing the tunnel network outside was the main fix. All other problems where a result of not having consequently changed all netmasks to the reduced LAN network on my guest operating systems. The two name servers had 255.255.0.0 as netmask, thats why they answered ping requests through the tunnel. The other boxes still had 255.0.0.0. All have now 255.128.0.0 and everything works fine.

    So many thanks again to all who helped me! Great work!

    Rumpi


Locked