No connection to virtualized pfSense after add and reboot a 2nd WAN interface
-
Hello, I use pfSense 2.60 in a virtualized (Proxmox) environment. I use at the moment 1 Management and one WAN (dynamic IPv4 from ISP) interface.
All works fine.I've got now a new FTTH fiber connection from another ISP with a dynamic IPv6 ip address (PPPoE).
Goal:
Use both WAN interfaces and use load balancing over pfSense (is this possible with IPv4 and IPv6?).Steps for the 2nd WAN interface:
- new Linux Bridge Port on Proxmox
- add virtIO bridge Network interface with VLAN 22 on pfSense
Config on pfSense:
- add new Interface IPv4 (PPPoE) / IPv6 (DHCP6)
If I then restart the pfSense then I've no connection to the pfSense (from Management). Only If I remove the new virtIO bridge in Proxmox and make a reboot then I've access again.
If I make no restart I can use the new 2nd WAN interface. The interface comes online and I got the right IPv6 from the ISP.
I can also make a Speedtest from pfSense and can observe that both interfaces were used.But if I restart the pfSense then I've no connection.
I don't understand this behavior. -
@denbir In virtual environments the names of the virtual adapters can get messed up. Did you try VNCing from Proxmox after the restart and see the output?
-
@nightlyshark VNCing? Do you mean the native Console from Proxmox? Yes, the adapter looks the right one.
-
@denbir Maybe try it in steps? Eg, first create the virtio iface -> reboot. Assign it to a PfSense WAN2 iface->reboot. DHCP, a-reboot. This is what I had to do in order to setup the virtio ifaces, only from a TrueNAS host (behyve hypervisor).
-
After I add the new virtIO Network Device bridge (net1) on the pfSense VM and make a reboot then is no connection possible.
The new created interface is also on the pfSense disabled but is assigned.
-
@denbir That iface (net1) seems to have a VLAN tag configured, hypervisor side. Maybe try without the VLAN tag? I believe that needs to be configured pfsense-side (vtnet (net1) <- VLAN 22 (configured on the iface for vtnet) <- PPPoE (configured on the iface for VLAN 22).
-
@nightlyshark Looks better. I've removed the VLAN tag on the Hypervisor and after a reboot the connection could be again established.
Also with an enabled interface. -
@denbir Configuring the VLAN tag 22 on the hypervisor, means only packets with a VLAN tag of 22 pass to the net1->virtio from the hypervisor vswitch. Just in case you ever need to configure it like this.
-
@nightlyshark OK but VLAN 22 is necessary from the ISP. That is why is configured on the Hypervisor. I'll try this on the pfSense direct to configure VLAN 22 for the 2nd WAN port.
-
@denbir Just had a thought, did you set vmbr2 as VLAN-aware?
-
-
@nightlyshark Yes, VLAN aware was already configured.
-
@denbir I don't know enough about proxmox to be of help there, but there must be a way to untag the VLAN 22 packets before they reach pfsense, which would only do PPPoE without being aware of a VLAN.
Or just pass-through an adapter as hardware to pfsense and connect from there? In case your hardware allows you to, of course. This would also greatly improve pfsense performance, as it uses hardware capabilities of network adapters heavily.
-
@denbir Could it be an MTU of proxmox bridge vs MTU of net1 on pfsense issue?
-
@denbir I think the VLAN needs to also be configured (in proxmox) on the physical adapter that connects to the FTTH modem as a tagged VLAN, in order to create a trunk between the FTTH modem and pfsense because the path is:
FTTH ether -> Trunk for VLAN 22
Physical adapter on server running proxmox (VLAN aware?)
proxmox bridge for net1 (yes VLAN aware, as you said)
PfSense VirtIO iface (configure a VLAN with tag 22 on it)If the chain is broken in any adapter, all VLAN 22 packets are likely to be dropped.
-
@nightlyshark Pass-through is not possible because in the Server is only one hardware NIC with 4 integrated ports. If I understand this correct then can I use only dedicated Hardware for pass-through.
-
@nightlyshark I don't understand this behavior regarding the VLAN for 2nd WAN interface. I use in Proxmox with 3 interfaces.
Interface 1 for Management
Interface 2 for 1st Wan port and the new 2nd interface for 2nd Wan port with VLAN 22.
The 2nd Wan interface config should independent from the Management interface but this is not so. -
This post is deleted! -
@nightlyshark Sorry, this info was wrong. I've tested the config again and already after I add the new 2nd wan interface on the pfSense VM (with or without VLAN) after this step and and a reboot then is the pfSense not available.
-
@denbir Sorry for the delay. Did you enable the VLAN tag on the physical NIC that connects FTTH with proxmox?
