PfSense can ping all but one specific IP address in range

eeverglades · Apr 22, 2013, 12:12 PM

Hi,

will try to give resume of my setup:
1. I have a production network range 192.168.22.x
2. We use a Veeam backup server for VMWare. The Veeam server IP is 192.168.22.20. This server can create a 'sandbox' to test virtual machines.
3. I create virtual machine in a sandbox, and make at static IP address mapping from production network into the VM in the sandbox. The IP is 192.168.22.15 on production network (I have tryed other IP for the VM aswell, but all results are the same)
4. I can ping and remote to the VM in the sandbox from every server/PC in the IP range 192.168.22.x.
5. The only place I can NOT ping the VM in the sandbox from is the pfSense firewall.
5.1. I use the correct interface to ping from
5.2. the firewall can ping the Veeam server on 192.168.22.20 and other servers/PC's in the range, but not the VM in sandbox on 192.168.22.15.

I have never seen this occur before, so any usefull input will be much appreciated.

I need to get firewall to ping/connect to the 192.168.22.15 IP, because I need remote access the VM in the sandbox. The firewall is preventing me from achieving this.

stephenw10 · Apr 22, 2013, 1:06 PM

I assume there is nothing in the firewall logs? Or the system logs?
Possibly something to do with the MAC translation from the real to virtual NIC in VMware. Just speculation.

If there's nothing in the logs it's probably a routing problem. Possible the VM doesn't have a route back to the pfSense box. Maybe the wrong subnet mask?

Run a packet capture on the LAN interface to see if pings are leaving and returning at all.

Steve

marvosa · Apr 22, 2013, 2:04 PM

Give us a network map with IP and mask info.

wallabybob · Apr 22, 2013, 7:10 PM

@eeverglades:

5. The only place I can NOT ping the VM in the sandbox from is the pfSense firewall.

What does ping report when you attempt that? (The ping report is almost always more informative than "can not ping".)

rakeshvijayan · Apr 23, 2013, 7:35 AM

Let me know my friend did you setup the vmbox network adapter to bridge connection to your ethernet card or wireless.by default the vmbox it will access Nat connection so you cant ping form the out side connection …

check your both ipaddress range in vmbox and out side is match to ping both connection and post here network configuration here we cant assume with out clue dear friend

eeverglades · Apr 23, 2013, 7:43 AM

Wau, quite an active forum. Did not expect so many replies so soon.

First a confession: the IP range is not 192.168.20.x/24, but 192.168.88.x/24

So important IP address in that range are:
192.168.88.15 = VM in sandbox
192.168.88.20 = Veeam server (backup server)
192.168.88.254 = pfSense firewall interface

When I try packet capture from firewall with full detail setting for IP 192.168.88.15, and then ping the VM I get:
07:41:47.733598 ARP, Request who-has 192.168.88.15 tell 192.168.88.254, length 28
07:41:48.752283 ARP, Request who-has 192.168.88.15 tell 192.168.88.254, length 28
07:41:49.772138 ARP, Request who-has 192.168.88.15 tell 192.168.88.254, length 28

When I try the same capture for the Veeam backup server I get:
07:47:44.423325 IP 192.168.88.254 > 192.168.88.20: ICMP echo request, id 29645, seq 0, length 64
07:47:44.423642 IP 192.168.88.20 > 192.168.88.254: ICMP echo reply, id 29645, seq 0, length 64
07:47:45.440352 IP 192.168.88.254 > 192.168.88.20: ICMP echo request, id 29645, seq 1, length 64
07:47:45.440588 IP 192.168.88.20 > 192.168.88.254: ICMP echo reply, id 29645, seq 1, length 64
07:47:46.460201 IP 192.168.88.254 > 192.168.88.20: ICMP echo request, id 29645, seq 2, length 64
07:47:46.460534 IP 192.168.88.20 > 192.168.88.254: ICMP echo reply, id 29645, seq 2, length 64

And as mentioned before, I can ping and RDP to the VM in the sandbox from any server/PC in the 192.168.88.x range.

So I'm ??!??!??!????

eeverglades · Apr 23, 2013, 7:49 AM

Thank you for the info rakeshvijayan.

The pfSense firewall is Virtual, so it has no physical Ethernetcard.

All the other servers which can ping the VM in the sandbox are also virtual.

I'm unable to ping from the 'inside' interface of the firewall.

Hope I understood your post correctly, and my reply therefore is relevant.

rakeshvijayan · Apr 23, 2013, 8:03 AM

how you configure pf ethernet card what ever may be there must be a virtual interface in actual os of wmware to connect to virtual machine check there . if it whole are correct post a Image of you pf interface here we need to check the configuration that made there ….my static ip configuration is so with cidr 24

eeverglades · Apr 23, 2013, 8:51 AM

I have not mentioned that the 192.168.88.x/24 is also using Vlan 88.

I tought that maybe this could be an issue, but after checking the Vswitch setup, I still can't find an error.

I have also tryed with and without Vlan 88 set on the firewall interface on pfSense, but still no go.

I have attached a Vsphere setup picture.

The firewall is the one called 254.domain.lan.

A server is the one called 102.domain.lan.

102 can ping VM, but 254 can not.

stephenw10 · Apr 23, 2013, 8:55 AM

How many interfaces do you have on the pfSense VM? Looks like just one.

The fact that when you try to ping from it it is ARPing and getting no reply is not good.

Steve

wallabybob · Apr 23, 2013, 8:57 AM

@eeverglades:

When I try packet capture from firewall with full detail setting for IP 192.168.88.15, and then ping the VM I get:
07:41:47.733598 ARP, Request who-has 192.168.88.15 tell 192.168.88.254, length 28

This is pfSense trying to discover the MAC address of the system with IP address 192.168.88.15. That there is no reply suggests to me one or more of:
1. The "plumbing" linking VMs doesn't include a system with IP address 192.168.88.15
2. Such a system is configured to ignore ARPs.

I suggest you do a packet capture in that VM to see if the ARP Requests are reaching it and it is responding.

eeverglades · Apr 23, 2013, 9:04 AM

Yes, it is highly likely that the ARP requests are not reaching the VM from the pfSense, but why?

The VM accepts ping and RDP from all other devices in the IP range, so it is not an issues with the VM setup.

It must be some sort of network issue…...

The backupserver hosting the VM is physical, so all the traffic goes through a physical NIC. But why can the server access the VM, while the pfSense cant, when their setup seems identical?

They are in the same IP range, and both are connected to Vlan 88. The backupserver is also connected to Vlan 88.

stephenw10 · Apr 23, 2013, 9:41 AM

3. It is responding to ARPs but the pfSense box is not seeing the response.

Perhaps the other VMs have already cached the MAC/IP of the server. Is the pfSense box the most recent VM?

Can you ARP for that IP from any other machine?

Steve

eeverglades · Apr 23, 2013, 10:21 AM

Hi Steve,

Yes, all machines in the range can ping the VM in the sandbox.

No, the pfSense has been in production for over 1 year.

I'm not an network expert, since it's almost 10 years since i've studied ARP etc., and have forgotten all about it.

I've tryed different IP's for static mapping to the VM in the sandbox, and all the servers can find the VM right away. But pfSense won't.

wallabybob · Apr 23, 2013, 10:40 AM

@eeverglades:

I've tryed different IP's for static mapping to the VM in the sandbox, and all the servers can find the VM right away. But pfSense won't.

I think if you want more specific help you will need to provide much more detail on your configuration. In particular, how pfSense is supposed to communicate with the "problem" VM. I don't know vSphere but I consider it suspicious that your previously posted vSphere configuration screenshot doesn't show the problem VM on the same VLAN as the pfSense x.x.x.254 interface.

eeverglades · Apr 23, 2013, 11:20 AM

Ok, i've tryed to make a drawing using paint (yes good old paint :) )

Does this give you guys any possible soulutions or ideas for tools for problemsolving?

stephenw10 · Apr 23, 2013, 1:19 PM

Hmm. I would try creating a different server in the sandbox and see if the results are any different.

You haven't shown any VLANs on the diagram, I assume everything there is in the same VLAN?

Check the MAC of the sandbox server against the real NIC and anything else in the chain. .20 .22 and .15 are presumably using the same physical NIC. There may be more than one device using the same MAC which is causing pfSense a problem. Do you have any other FreeBSD boxes to test with?

Steve

rakeshvijayan · Apr 24, 2013, 5:20 AM

FORM YOU PICTURE SHOW THAT YOU CONFIGURED IPS IN SAME RANGE NO NATING IS DOING THERE . MY SUGGESTION IS TRY TO REMOVE THE TICK FROM Block private networks Block bogon networks FROM THE INTERFACE . THIS MAY SOLVE YOU PROBLEM

eeverglades · Apr 29, 2013, 12:42 PM

Been on a long weekend vacation…...

Rake, good suggestion, but unfortunately the boxes are unticked :(

Stephen, the MAC's of the sandbox proxy and real NIC are different.

I have checked the ARP table on the pfSense, and the IP of the sandbox proxy is in the table, although the IP of the VM in the sandbox is not.

Note: I am NOT able to ping the ip of the sandbox proxy from the pfSense firewall either. All other servers in the range can ping the sandbox proxy IP.

I am ever so close to jumping out the window (don't worry, only a 2 feet drop). This problem is just not logical.........

johnpoz · Apr 29, 2013, 1:01 PM

So if pfsense is not on the same segment??

The firewall is the one called 254.domain.lan.
A server is the one called 102.domain.lan.
102 can ping VM, but 254 can not.

As mentioned above by wallabybob where is this 88.15 box connected to that vswitch? If that vswitch is the 192.168.88.0/24 ??

Show us this box your trying to ping on your vsphere setup. And its ipconfig /all – I am guessing its a windows box?