cannot access Proxmox VMs after switching to pfSense

[kal800]

Hi,

I've just replaced my Orbi Pro router with pfSense installed on Xeon based server. I have not made any changes in my network configuration - everything remains the same. There are no VLANs, every host is in the same subnet. I've got WAN interface connected to cable modem in bridge mode, LAN interface is connected to the switch where all remaining devices are plugged in, and Orbi router as an Access Point for wireless usage. Everything works fine, well almost. I am able to access Proxmox host, but I cannot access any VM guest. When I try to ping them, I'm having:

PING 192.168.1.111 (192.168.1.111): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
ping: sendto: No route to host
Request timeout for icmp_seq 2
ping: sendto: Host is down
Request timeout for icmp_seq 3
ping: sendto: Host is down
Request timeout for icmp_seq 4
ping: sendto: Host is down
Request timeout for icmp_seq 5

So, it is clear for me that the firewall blocks the traffic between LAN hosts and Virtual Machines. I believe that the cause could be that several IPs are on the same physical interface of Proxmox server, but I don't have a clue where to seek solution.

Could you please give me some advise?

Kal800

[viragomann]

@kal800
How are the VMs connected to the LAN? Are they bridged to the Proxmox NIC or natted?

Can you ping a VM from pfSense?

Are the network settings correct on the VMs (default gateway)?

[kal800]

@viragomann said in cannot access Proxmox VMs after switching to pfSense:

@kal800
How are the VMs connected to the LAN? Are they bridged to the Proxmox NIC or natted?

bridge=vmbr0 - that is the network interface setup

Can you ping a VM from pfSense?

no, I cannot - the host is down

Are the network settings correct on the VMs (default gateway)?

yes, IP and netmask and default gw are correct on VMs - they worked before after all

[viragomann]

@kal800
I'd suspect that there is a network issue.

If you ping a VM from pfSense, the system does an ARP request for the IP and if there is a reply the MAC and IP are inserted into the ARP table (Diagnostic > ARP).
I assume, it fails.

However, do an investigation of the ARP communication to see if it works.
In Diagnostic > Packet Capture select LAN interface and ARP protocol, set the detail level to full and start the capture.
Then try a ping to a VM.
Stop the capture and check, what you got.

[kal800]

@viragomann

I changed the network device type from "virtio" to Intel E1000 type, and it started to work.

It is strange, because those IPs were visible on ARP tables before - when it did not work.

Except Windows VM that uses E1000 as well, but who cares about Windows host after all ;)

[viragomann]

@kal800
So you installed pfSense on Proxmox?
Would be worth to mention.

[kal800]

@viragomann

no, it is installed on bare metal. I have just changed network interface type on each of VMs config.

[viragomann]

@kal800
I'm wondering, what the switching to pfSense has changed then regarding the VMs network settings. Weird.

[stephenw10]

Mmm, weird indeed. That should make no difference.
Kind of 'feels' like something was cached and changing the NIC type cleared it. Guessing though.

[kal800]

@stephenw10

Well, I've just switched to virtio again, rebooted all of them, and it works... weird indeed.