• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

No connection to virtualized pfSense after add and reboot a 2nd WAN interface

Scheduled Pinned Locked Moved Routing and Multi WAN
33 Posts 3 Posters 5.6k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • D
    DenBir @NightlyShark
    last edited by Feb 21, 2023, 11:09 AM

    @nightlyshark Looks better. I've removed the VLAN tag on the Hypervisor and after a reboot the connection could be again established.
    Also with an enabled interface.

    N 1 Reply Last reply Feb 21, 2023, 3:45 PM Reply Quote 1
    • N
      NightlyShark @DenBir
      last edited by Feb 21, 2023, 3:45 PM

      @denbir Configuring the VLAN tag 22 on the hypervisor, means only packets with a VLAN tag of 22 pass to the net1->virtio from the hypervisor vswitch. Just in case you ever need to configure it like this.

      D 2 Replies Last reply Feb 21, 2023, 5:29 PM Reply Quote 0
      • D
        DenBir @NightlyShark
        last edited by Feb 21, 2023, 5:29 PM

        @nightlyshark OK but VLAN 22 is necessary from the ISP. That is why is configured on the Hypervisor. I'll try this on the pfSense direct to configure VLAN 22 for the 2nd WAN port.

        N 2 Replies Last reply Feb 21, 2023, 5:50 PM Reply Quote 0
        • N
          NightlyShark @DenBir
          last edited by NightlyShark Feb 21, 2023, 5:56 PM Feb 21, 2023, 5:50 PM

          @denbir Just had a thought, did you set vmbr2 as VLAN-aware?

          1 Reply Last reply Reply Quote 0
          • N
            NightlyShark @DenBir
            last edited by Feb 21, 2023, 5:50 PM

            @denbir
            77606a5c-f236-412d-bcf4-c10d7c94425b-image.png

            D 1 Reply Last reply Feb 21, 2023, 5:56 PM Reply Quote 0
            • D
              DenBir @NightlyShark
              last edited by Feb 21, 2023, 5:56 PM

              @nightlyshark Yes, VLAN aware was already configured.

              N 3 Replies Last reply Feb 21, 2023, 6:02 PM Reply Quote 0
              • N
                NightlyShark @DenBir
                last edited by Feb 21, 2023, 6:02 PM

                @denbir I don't know enough about proxmox to be of help there, but there must be a way to untag the VLAN 22 packets before they reach pfsense, which would only do PPPoE without being aware of a VLAN.

                Or just pass-through an adapter as hardware to pfsense and connect from there? In case your hardware allows you to, of course. This would also greatly improve pfsense performance, as it uses hardware capabilities of network adapters heavily.

                D 1 Reply Last reply Feb 21, 2023, 6:39 PM Reply Quote 0
                • N
                  NightlyShark @DenBir
                  last edited by Feb 21, 2023, 6:03 PM

                  @denbir Could it be an MTU of proxmox bridge vs MTU of net1 on pfsense issue?

                  1 Reply Last reply Reply Quote 0
                  • N
                    NightlyShark @DenBir
                    last edited by Feb 21, 2023, 6:12 PM

                    @denbir I think the VLAN needs to also be configured (in proxmox) on the physical adapter that connects to the FTTH modem as a tagged VLAN, in order to create a trunk between the FTTH modem and pfsense because the path is:
                    FTTH ether -> Trunk for VLAN 22
                    Physical adapter on server running proxmox (VLAN aware?)
                    proxmox bridge for net1 (yes VLAN aware, as you said)
                    PfSense VirtIO iface (configure a VLAN with tag 22 on it)

                    If the chain is broken in any adapter, all VLAN 22 packets are likely to be dropped.

                    D 2 Replies Last reply Feb 21, 2023, 6:49 PM Reply Quote 0
                    • D
                      DenBir @NightlyShark
                      last edited by Feb 21, 2023, 6:39 PM

                      @nightlyshark Pass-through is not possible because in the Server is only one hardware NIC with 4 integrated ports. If I understand this correct then can I use only dedicated Hardware for pass-through.

                      G 1 Reply Last reply Feb 24, 2023, 12:06 PM Reply Quote 0
                      • D
                        DenBir @NightlyShark
                        last edited by Feb 21, 2023, 6:49 PM

                        @nightlyshark I don't understand this behavior regarding the VLAN for 2nd WAN interface. I use in Proxmox with 3 interfaces.
                        Interface 1 for Management
                        Interface 2 for 1st Wan port and the new 2nd interface for 2nd Wan port with VLAN 22.
                        The 2nd Wan interface config should independent from the Management interface but this is not so.

                        1 Reply Last reply Reply Quote 0
                        • D
                          DenBir @NightlyShark
                          last edited by Feb 21, 2023, 7:00 PM

                          This post is deleted!
                          1 Reply Last reply Reply Quote 0
                          • D
                            DenBir @NightlyShark
                            last edited by Feb 21, 2023, 7:29 PM

                            @nightlyshark Sorry, this info was wrong. I've tested the config again and already after I add the new 2nd wan interface on the pfSense VM (with or without VLAN) after this step and and a reboot then is the pfSense not available.

                            N 1 Reply Last reply Feb 21, 2023, 9:12 PM Reply Quote 0
                            • N
                              NightlyShark @DenBir
                              last edited by Feb 21, 2023, 9:12 PM

                              @denbir Sorry for the delay. Did you enable the VLAN tag on the physical NIC that connects FTTH with proxmox?

                              D 1 Reply Last reply Feb 21, 2023, 9:40 PM Reply Quote 0
                              • D
                                DenBir @NightlyShark
                                last edited by Feb 21, 2023, 9:40 PM

                                @nightlyshark No worries. I am so happy that you support my challange :-).
                                The FTTH is direct connected with the physical port enp3s0 of the proxmox server. I've configured then a Linux Bridge vmbr2 and enabled VLAN aware.
                                17ba40ab-a716-490a-b3db-ea3cfdd4404f-image.png

                                I see no option to set direct on the physical port a VLAN tag.
                                67371b13-59d1-446b-b654-c0cd573ddb07-image.png

                                I think this is not the problem because also if I add the new Network Device on the VM with or without a VLAN tag then is the pfSense after a reboot not reachable.
                                Why is here a dependency between the Management and the 2nd WAN? I don't understand this ...
                                Proxmox and other VMs on Proxmox are reachable only the pfSense is then not reachable.
                                e44fdf56-ca26-45e2-8f9f-a04ac0143894-image.png

                                N 3 Replies Last reply Feb 21, 2023, 9:51 PM Reply Quote 0
                                • N
                                  NightlyShark @DenBir
                                  last edited by Feb 21, 2023, 9:51 PM

                                  @denbir I think you need to tag VLAN 22 on the enp3s0 iface for the traffic to trunk, not just vmbr2-net1. I think you can do it from CLI. (Unless you do not have local access to proxmox and are in danger of getting locked-out)

                                  1 Reply Last reply Reply Quote 0
                                  • N
                                    NightlyShark @DenBir
                                    last edited by Feb 21, 2023, 9:53 PM

                                    @denbir I think those bridges are not designed to handle ingress VLAN traffic, only inter-paravirtual and host VLANs.

                                    1 Reply Last reply Reply Quote 0
                                    • N
                                      NightlyShark @DenBir
                                      last edited by NightlyShark Feb 26, 2023, 9:25 AM Feb 21, 2023, 9:55 PM

                                      @denbir I have a virtualized pfsense install on TrueNAS (bhyve hypervisor), and to have acceptable performance I had to add another NIC for TrueNAS itself and just passed the HP Broadcomm 4-port 1Gig-NIC directly to PfSense. Next step, slept like a baby.

                                      1 Reply Last reply Reply Quote 0
                                      • G
                                        Gblenn @DenBir
                                        last edited by Feb 24, 2023, 12:06 PM

                                        @denbir said in No connection to virtualized pfSense after add and reboot a 2nd WAN interface:

                                        @nightlyshark Pass-through is not possible because in the Server is only one hardware NIC with 4 integrated ports. If I understand this correct then can I use only dedicated Hardware for pass-through.

                                        My i350-T4 cards show up with each individual interface as a separate entity in the Devices list under PCI Devices in Hardware. They show up as e.g. 0000:01:00.1, .2, .3 and .4 respectively.
                                        Just make sure the box that reads "All Functions" is not ticked, otherwise the entire card will be allocated.

                                        Also, I believe that the port numbering seen by pfsense, like vtnet0, vtnet1 and so on, depends on which order you happen to add them in Proxmox. It's not the vmbr number that is showing up, it's net0, net1 etc. So if vmbr2 is added before vmbr0, vmbr2 will be vtnet0... At least that's what I think I'm seeing. I think that is what I am seeing in your picture earlier. LAN is vtnet1 which you gave a tag in Proxmox. With that setup, I think it you should put it on vmbr0. Perhaps you are mistaking the order of the ports on the card??

                                        I would go in to the pfsense Console on Proxmox and make sure you have the three interfaces assigned as intended, and reassign if needed. One way forward could be to only assign the LAN and WAN as you had it set up before. Then the newly added interface will be possible to add from the UI before you finally save and reboot.

                                        Wrt VLAN tags being set in Proxmox, that port will act the as an untagged port in a switch. Only VLAN 22 traffic will pass but pfsense will not be aware of the VLAN tag, since it's being stripped...

                                        N 1 Reply Last reply Feb 25, 2023, 12:19 AM Reply Quote 0
                                        • N
                                          NightlyShark @Gblenn
                                          last edited by Feb 25, 2023, 12:19 AM

                                          @gblenn Some specific NICs are registered as a single IOMMU domain and thus it is not possible to pass them on in hardware. My HPE Broadcom 4-Port Gigabit NIC had the same problem. It can either be passed through with all 4 ports or not at all. That is a hardware limitation and cannot be worked around.

                                          G 1 Reply Last reply Feb 25, 2023, 9:21 AM Reply Quote 0
                                          26 out of 33
                                          • First post
                                            26/33
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                            This community forum collects and processes your personal information.
                                            consent.not_received