PfSense can ping all but one specific IP address in range
-
So if pfsense is not on the same segment??
The firewall is the one called 254.domain.lan.
A server is the one called 102.domain.lan.
102 can ping VM, but 254 can not.As mentioned above by wallabybob where is this 88.15 box connected to that vswitch? If that vswitch is the 192.168.88.0/24 ??
Show us this box your trying to ping on your vsphere setup. And its ipconfig /all – I am guessing its a windows box?
-
Hi John,
thank you for joining.
If you check the drawing I made on top of page 2 in this tread, you might get the overview you need.
The sandbox proxy is created by the backupserver, and is a linux as far as I know. I have no linux experience.
Let me know if there is any other info you might need.
-
You didn't say whether you have any other FreeBSD machines on your network?
One possibility is that FreeBSD, and hence pfSense, adhere strictly to the rules regarding IPs, routing, subnets etc. Other OSs not so much. Hence it's possible to have a setup that works from Windows and not FreeBSD.Steve
-
No, there are no other FreeBSD machines in that IP range. So maybe this could be relevant. You have any suggestions as to how I might check this? A complete novice with FreeBSD.
I will just point out once more that pfSense can ping the backup server at .20 and all other machines in the range. Only the proxy/VM are 'unpingable'
-
What do you mean by proxy for sandbox? is that just another esxi host? Or other vm, then show us its vwswitch setup, like you did with the esxi host containing the pfsense and 102 VMs
-
You have any suggestions as to how I might check this?
No not really. You could setup a FreeBSD (or some derivative of it) machine and see how that behaves but that's not a quick and easy test. To be honest I doubt that this is the cause but I thought I'd mention it since we seem to be running out of options. The only time I've heard of it was a Windows box that was using a gateway outside of its subnet something that pfSense refused to do in the same situation.
Steve
-
can we see the ipconfig /all from the 102 box that you say can ping this 88.15 box
-
To Johnpoz (the ipconfig /all of the 102, which can ping VM):
Ethernet adapter DOMAIN.LAN:
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Broadcom BCM5716C NetXtreme II GigE (NDIS
VBD Client) #36
Physical Address. . . . . . . . . : 78-2B-CB-49-76-5F
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::31dd:2adc:9c3a:9b1e%13(Preferred)
IPv4 Address. . . . . . . . . . . : 192.168.88.10(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.88.254
DHCPv6 IAID . . . . . . . . . . . : 309865419
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-16-C8-11-52-78-2B-CB-49-76-60The default gateway is the pfSense, and all this works perfectly. The pfSense can also ping 102.
The proxy for the sandbox is created by the backupserver, and is not part of the VMWare enviroment. Please view drawing on top of tread page 2.
-
I expected the above to show 192.168.88.102 not .10 is that just a typo?
Steve
-
Hi Steve,
you are absoluty right. Wrong server ipconfig. This is the config for 102:
Ethernet adapter Domain.lan:Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Intel(R) PRO/1000 MT Network Connection #
2
Physical Address. . . . . . . . . : 00-50-56-96-42-07
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::c52d:5bbd:1909:a310%14(Preferred)
IPv4 Address. . . . . . . . . . . : 192.168.88.102(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.88.254
DHCPv6 IAID . . . . . . . . . . . : 318787670
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-18-36-15-5A-00-50-56-96-42-06 -
I saw your paint drawing.. And it makes no sense to what you mean by Proxy??
What proxy software? Do you mean its doing NAT? How does it proxy?
If you saying 88.102 can talk to 88.15 how does it do it via the proxy?
From your drawing there is no difference between the interface connected to the vswitch for your 88.102 box and pfsense at 88.102
Ok just took a look real quick!!
http://www.veeam.com/vmware-esx-backup.html
And its doing NAT into your sandbox, clearly states the stuff in the sandbox are isolated and you connect to them via a masqueraded IP
http://forums.veeam.com/viewtopic.php?f=24&t=9329&start=15#p40131
Q: How is it possible to access temporary VMs in the isolated network from production network, if VMs in both networks have the same IP addresses?
A: Each temporary VM is assigned so called "masquerade address" from selected masquerade network (part of virtual lab settings). Routing table on Veeam Backup server is automatically updated, and proxy appliance IP address in the production network is assigned as gateway for masquerade network. Acting as gateway, the proxy appliance performs address translation and substitutes masquerade IP address with real IP address in the isolated network. Although this sounds pretty complex, all happens transparently for you as a user.So what was the NAT setup you created when you installed this veem backup - it says you create that scheme when you setup.
-
Yes, correct. The masqrade IP range isolates the enviroment in the sandbox from production, and the masqrade IP range is only accessible from the Veeam server itself. In my case the masqrade range is 188.x.
I have a server .88.101 in production. I then recreate this server in the sandbox, which means it gets the masqrade IP of 188.101 to the world outside of the sandbox.
To be able to access the VM which is created in the sandbox from the production enviroment, you can map an unused IP adress in the production through to a VM in the Isolated enviroment.
Ergo I have mapped IP 88.15 in production to 88.101 in sandbox, which means that when I try to ping 88.15, I am actually trying to reach the VM in the sandbox, which has an IP of 88.101.
Hope you understanding my attempt at explaining.
To sum up the pinging status:
pfSense: .88.254
production server 102: .88.102
Veeam server: .88.20
Proxy gateway 'inside' of veeam server: .88.22
Masqrade IP for VM in sandbox: .188.101
Production IP for VM in sandbox: .88.15source: Dest: Result:
.88.20 .188.101 OK (Veeam server is only server to be able to ping masqrade IP, since it is the only server that knows of the .188.x IP range)
.88.102 .88.22 OK (server 102 can ping proxy gw)
.88.102 .88.15 OK (server 102 can ping VM in sandbox)
.88.254 .88.20 OK (pfSense can ping Veeam server)
.88.254 .88.22 NEG (pfSense can NOT ping proxy)
.88.254 .88.16 NEG (pfSense can NOT ping VM in sandbox) -
So from a quick breeze over of the UG for the veem backup 6.5, I don't think your understanding how it works to be honest.
So these labs or how you call them sandboxes? Are created on VM HOST, so your saying in your drawing that this esx host is your backup server?
Or is the LAB(sandbox) on your same ESXi host as your pfsense box? Or s different host?
While I agree your never going to get to this isolated VM in your lab/sandbox if you can not talk to the proxy - understanding your environment is a requirement in helping you find your problem.
–
44 | Veeam Backup & Replication | VMware Environments | USER GUIDE | REV 5To enable communication between the outer world and VMs in the virtual lab, Veeam Backup & Replication uses a proxy appliance that is created and registered in the folder and resource pool of the virtual lab. The proxy appliance is a VM that acts as a gateway routing requests from the production network to the isolated network.
To connect to isolated networks, Veeam Backup & Replication adds to the proxy appliance a vNIC adapter for each network. Each vNIC adapter gets an IP address from the network to which it is connected, which is typically the same as the IP address of a default gateway in the corresponding production network.
–So from this above statement - and your pic of your vswitch, where was this proxy/router VM connected to the vswitch that your pfsense 88.254 interface connected? If you proxy VM and sandbox are on a different ESX host, lets see the vswitch setup there where you proxy and VM in your lab is connected.
so if you have
esxi host (production) <88.3> --- 192.168.88.0/24 --- <88.20> other esxi host (veem backup with labs and vms)
Lets see the vm network from this other esxi host with the 88.20 address in your real world physical network.
-
Whilst this is clearly quite complex it doesn't really explain why a VM in the same host as the pfSense VM is able to reach the server VM in the lab/sandbox whilst pfSense is not.
However there must be a load of settings that could be wrong or at least not what you thought they were. Why is it not responding to ARP requests from the pfSense box? I would seem that the proxy is either not responding or sending responses to wrong place.
Perhaps because the pfSense box is at the end of the IP range, .254. :-\Steve
-
"Whilst this is clearly quite complex it doesn't really explain why a VM in the same host as the pfSense VM is able to reach the server VM in the lab/sandbox whilst pfSense is not."
While I agree with the comment that its odd that pfsense can ping the host that it seems the proxy (I don't like that word in this setup - more of a router) is on. But not the "proxy" and that the other host connected to the same vm switch as pfsense 88.254 address can
.88.254 .88.20 OK (pfSense can ping Veeam server)
.88.254 .88.22 NEG (pfSense can NOT ping proxy)But without understanding this setup more, its hard to know what the issue is. Maybe there is a firewall? Its clearly more complex that just a bunch of boxes connected to same switch in the same segment.
Maybe this 88.102 box has another interface? Would love to help the OP figure out his problem, but does not seem like we are getting a full picture of the setup. And from how the OP is explaining it - its quite possible they don't fully understand it ;)
Can we get a full picture of the VM networking setup from both his esxi host where pfsense sits and this veem exi host?? Or are these vms and labs on the 1 esxi Host??
-
Hi John,
you are absoluty correct. Eggs all over my face :) I messed up the Veeam explaination. The isolated eviroment is created on an esxi, not on the Veeam backup server.
I have migrated the relevant servers to the same esxi host, to give a better overview. I have tryed to take screenshots of the relevant setup.
Virtal lab 2 is the proxy/router created for the isolated enviroment.
Hope it helps….. And thank you for your patience.....
From what I see in the attachments, there is absolutly no reason why 102(server) can ping the virtual Lab 2 at 88.22, and 254(pfsense) can not.
 -
Can we get a full picture please? I can not make any sense of what your posting because its bits and pieces.
In your first posting
You did not show those virtual labs like you do in this time
So where is the 88.20 IP?? Is that proxy/router vm or is that IP of your veem physical box?
So why can pfsense not ping the interface of the proxy at 88.22 from 88.254 while interfaces are connected in the same switch? Well looks like you have a vlan tag of 88 on that switch? How is pfsense setup with vlans? etc.. Why are you using vlan tags? I don't see the point of it in this setup?? What IP do you have for the vlan you created on pfsense of 88, what interface did you put that vlan? etc. etc.
Can we get the network configuration of that VM proxy appliance that shows 2 interfaces one in the vswitch connected to pfsense interface and other in..
Here is a question for you can pfsense at 88.254 ping 88.102? Or 102 ping 254?
So in your first posting you only show 2 virtual machines on that switch, it say 2 right there in your picture. Now it says 4? So were they just added?? Also lets say you wanted to use vlan 88, great – you sure would not put vlan 88 tag on both sides your vm proxy??
From the switches yous how above both having vlan 88 tag, you have 1 leg of your proxy in a vswitch port group with vlan 88 tag, and the other leg where pfsense is and 88.102 box also having a vlan 88 tag?? That vm proxy is routing between networks from my quick breeze through of the UG.. So why would there be vlan 88 on both of its interfaces connected to 2 different network ip ranges?
-
88.20 is the physical Veeam box.
How is pfsense setup with vlans?
I have tryed both with and without Vlan 88 configured on the .88.254 interface (em1) of pfSense. Makes no difference to pinging at all.Can we get the network configuration of that VM proxy appliance that shows 2 interfaces one in the vswitch connected to pfsense interface and other in..
I have attached the config of the proxy. Don't know if it's enough for you. The interface in the proxy config with IP .88.254 is the inside of the isolated network. Maybe that could be the problem?Here is a question for you can pfsense at 88.254 ping 88.102? Or 102 ping 254?
Yes, i've noted several times that all the machines in the range can ping eachother. Only pfSense can not ping .22(proxy) or .15(VM)Now it says 4? So were they just added??
I posted just before: "I have migrated the relevant servers to the same esxi host, to give a better overview." Therefore all the machines I find relevant are in the same picture now.So why would there be vlan 88 on both of its interfaces connected to 2 different network ip ranges?
Hope I understand your long question correct. All these machines are in the .88.x/24 range.
 -
"config with IP .88.254 is the inside of the isolated network. Maybe that could be the problem?"
Yah think ;)
Why would the proxy answer pings to 88.254 when to him that is HIS address. I am going to look at the UG again, I don't think your setup is correct having the same 88.0/24 on both sides of the proxy VM.. Makes no sense! And can have all kinds of issues!!!
So you arp for 88.22, which is basically hey who has 88.22 tell 88.254
Well to that proxy VM, oh tell 88.254 – sure I will tell myself ;)
-
Very good. We seem to get closer to the target.
It's a bit of a roundabout with the .88.254 address of the pfSense.
It is the gateway for the production network, and therefore when you create the isolated enviroment and restore copies of the production network servers in that enviroment, all these isolated servers are still configured with the .88.254 GW. That is why Veeam normally recommends to use the same GW IP in the isolated enviroment, as in the production enviroment.But the problem I facing is that I want to access the isolated enviroment using the production GW, which is not possible since the production GW and isolated network GW have the same IP.
I've seen Veeam mention somewhere that there is a solution for the isolated network to get Internet access, and I would guess that that solution would also solve the problem.
Maybe by just giving the isolated GW a new available IP in the productionrange, and manually changing the GW IP of the servers in the isolated enviroment to that GW.
And maybe adding a default route to the proxy pointing to the production GW.Going to do some reading. GREAT to finally be making some progresssssss.
