Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    What is the VMM Network selection for a LAN and WAN NIC

    Scheduled Pinned Locked Moved Virtualization
    33 Posts 3 Posters 4.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • E
      eiger3970 0 @viragomann
      last edited by eiger3970 0

      @viragomann Before a host reboot:

      ubuntu@ubuntu:/etc/netplan$ cat 01-network-manager-all.yaml
      # Let NetworkManager manage all devices on this system
      network:
        version: 2
        renderer: NetworkManager
        ethernets:
          enp2s0:
            dhcp4: yes 
          enp3s0:
            dhcp4: yes
        bridges:
          br0:
            dhcp4: yes
            interfaces:
              - enp3s0
              - vtnet0
          br1:
            dhcp4: no
            addresses: [192.168.1.120/24]
            gateway4: 192.168.1.1
            nameservers:
              addresses: [8.8.8.8,8.8.8.4]
            interfaces:
              - enp2s0
              - vtnet0
      ubuntu@ubuntu:/etc/netplan$ sudo netplan try
      
      ** (process:22120): WARNING **: 14:50:09.654: `gateway4` has been deprecated, use default routes instead.
      See the 'Default routes' section of the documentation for more details.
      /etc/netplan/01-network-manager-all.yaml:24:11: Error in network definition: br1: interface 'vtnet0' is not defined
              - vtnet0
                ^
      ubuntu@ubuntu:/etc/netplan$ ip -c a
      1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
          link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
          inet 127.0.0.1/8 scope host lo
             valid_lft forever preferred_lft forever
          inet6 ::1/128 scope host 
             valid_lft forever preferred_lft forever
      2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
          link/ether 1c:61:b4:6d:38:4f brd ff:ff:ff:ff:ff:ff
          inet6 fe80::31ca:9227:dcb3:d09e/64 scope link noprefixroute 
             valid_lft forever preferred_lft forever
      3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
          link/ether a8:a1:59:6e:1f:8b brd ff:ff:ff:ff:ff:ff
          inet 100.76.25.213/10 brd 100.127.255.255 scope global dynamic noprefixroute enp3s0
             valid_lft 206sec preferred_lft 206sec
          inet6 2406:2d40:4100:8fb2:a33c:3599:7805:669/64 scope global temporary dynamic 
             valid_lft 240sec preferred_lft 90sec
          inet6 2406:2d40:4100:8fb2:14a9:5501:ce7c:4421/64 scope global dynamic mngtmpaddr noprefixroute 
             valid_lft 240sec preferred_lft 90sec
          inet6 fe80::5be3:7f2a:2da2:5285/64 scope link noprefixroute 
             valid_lft forever preferred_lft forever
      4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
          link/ether 52:54:00:88:b4:b4 brd ff:ff:ff:ff:ff:ff
          inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
             valid_lft forever preferred_lft forever
      

      After reboot, no Internet.

      ubuntu@ubuntu:~$ ip -c a
      1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
          link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
          inet 127.0.0.1/8 scope host lo
             valid_lft forever preferred_lft forever
          inet6 ::1/128 scope host 
             valid_lft forever preferred_lft forever
      2: enp2s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
          link/ether 1c:61:b4:6d:38:4f brd ff:ff:ff:ff:ff:ff
      3: enp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
          link/ether a8:a1:59:6e:1f:8b brd ff:ff:ff:ff:ff:ff
      4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
          link/ether 52:54:00:88:b4:b4 brd ff:ff:ff:ff:ff:ff
          inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
             valid_lft forever preferred_lft forever
      

      No Internet as I think the Host is looking for the Guest pfSense router.
      However, this is the next issue the VMM won't add 1 or 2 bridges.
      Starting the Guest Error starting domain: Cannot get interface MTU on 'br0': No such device.

      V 1 Reply Last reply Reply Quote 0
      • V
        viragomann @eiger3970 0
        last edited by

        @eiger3970-0 said in What is the VMM Network selection for a LAN and WAN NIC:

        However, this is the next issue the VMM won't add 1 or 2 bridges.
        Starting the Guest Error starting domain: Cannot get interface MTU on 'br0': No such device.

        No surprise at all. There is no br0 configured.

        Obviously you have no proper bridge configuration now. None of your NICs is assigned to the only one bridge which is called virbr0.

        E 2 Replies Last reply Reply Quote 1
        • E
          eiger3970 0 @viragomann
          last edited by

          This post is deleted!
          1 Reply Last reply Reply Quote 0
          • E
            eiger3970 0 @viragomann
            last edited by

            @viragomann I've realised the bridges I create NIC0-br0 and NIC1-br1, disappear after rebooting the machine?
            However virbr0 appears (probably from KVM) and stays after a reboot.

            The main issue now seems to be the Ubuntu 22.04 machine can't ping the vm router on 192.168.1.1.
            I can't quite figure out the host machine's static ip, even though I thought I set it in /etc/netplan/01-network-manager-all.yaml.

            E 1 Reply Last reply Reply Quote 0
            • E
              eiger3970 0 @eiger3970 0
              last edited by eiger3970 0

              @eiger3970-0
              After a few Ubuntu 22.04 reboots, the br0 and br1 appear permanent.
              However, whilst br0 connectes to enp3s0, br1 won't stay connected to enp2s0?
              br0 automatically connects to vnet1, after I start up the vm.
              Confusing?

              ubuntu@ubuntu:/etc/netplan$ ip -c a
              1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
                  link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
                  inet 127.0.0.1/8 scope host lo
                     valid_lft forever preferred_lft forever
                  inet6 ::1/128 scope host 
                     valid_lft forever preferred_lft forever
              2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
                  link/ether 1c:61:b4:6d:38:4f brd ff:ff:ff:ff:ff:ff
                  inet 192.168.1.120/24 brd 192.168.1.255 scope global noprefixroute enp2s0
                     valid_lft forever preferred_lft forever
                  inet6 fe80::31ca:9227:dcb3:d09e/64 scope link noprefixroute 
                     valid_lft forever preferred_lft forever
              3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
                  link/ether a8:a1:59:6e:1f:8b brd ff:ff:ff:ff:ff:ff
              4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
                  link/ether 52:54:00:88:b4:b4 brd ff:ff:ff:ff:ff:ff
                  inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
                     valid_lft forever preferred_lft forever
              5: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
                  link/ether d2:da:46:a2:b2:3e brd ff:ff:ff:ff:ff:ff
                  inet 192.168.1.120/24 brd 192.168.1.255 scope global noprefixroute br1
                     valid_lft forever preferred_lft forever
                  inet6 fe80::d0da:46ff:fea2:b23e/64 scope link 
                     valid_lft forever preferred_lft forever
              6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
                  link/ether 0a:d4:78:1f:cd:41 brd ff:ff:ff:ff:ff:ff
                  inet 100.76.25.213/10 brd 100.127.255.255 scope global dynamic noprefixroute br0
                     valid_lft 267sec preferred_lft 267sec
                  inet6 2406:2d40:4100:8fb2:20eb:c8df:ba1e:9e3e/64 scope global temporary dynamic 
                     valid_lft 202sec preferred_lft 52sec
                  inet6 2406:2d40:4100:8fb2:8d4:78ff:fe1f:cd41/64 scope global dynamic mngtmpaddr 
                     valid_lft 202sec preferred_lft 52sec
                  inet6 fe80::8d4:78ff:fe1f:cd41/64 scope link 
                     valid_lft forever preferred_lft forever
              7: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
                  link/ether fe:54:00:33:3c:4b brd ff:ff:ff:ff:ff:ff
                  inet6 fe80::fc54:ff:fe33:3c4b/64 scope link 
                     valid_lft forever preferred_lft forever
              8: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br1 state UNKNOWN group default qlen 1000
                  link/ether fe:54:00:50:81:3f brd ff:ff:ff:ff:ff:ff
                  inet6 fe80::fc54:ff:fe50:813f/64 scope link 
                     valid_lft forever preferred_lft forever
              ubuntu@ubuntu:/etc/netplan$ bridge link
              3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100 
              7: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100 
              8: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br1 state forwarding priority 32 cost 100 
              ubuntu@ubuntu:/etc/netplan$ ubuntu@ubuntu:/etc/netplan$ cat 01-network-manager-all.yaml
              # Let NetworkManager manage all devices on this system
              network:
                version: 2
                renderer: NetworkManager
                ethernets:
                  enp2s0:
                    dhcp4: true 
                  enp3s0:
                    dhcp4: true
                bridges:
                  br0:
                    dhcp4: true
                    interfaces:
                      - enp3s0
                  br1:
                    dhcp4: false
                    addresses: [192.168.1.120/24]
                    gateway4: 192.168.1.1
                    nameservers:
                      addresses: [8.8.8.8,8.8.8.4]
                    interfaces:
                      - enp2s0
              
              V 1 Reply Last reply Reply Quote 0
              • V
                viragomann @eiger3970 0
                last edited by

                @eiger3970-0
                I'd kick the Network Manager away and do it with ifup / ifdown scripts.
                This is a server, the settings need to be done only once.

                I only use NM on the desktop and notebook, because you can easily make a VPN connection with it.

                E 1 Reply Last reply Reply Quote 1
                • E
                  eiger3970 0 @viragomann
                  last edited by

                  @viragomann Yes, I've updated the bridge connection via ip link commands and have the NIC0-br0 and NIC1-br1.
                  However, this Ubuntu 22.04 machine 192.168.1.120 and the guest vm pfSense 192.168.1.1 won't ping each other?

                  ubuntu@ubuntu:/etc/netplan$ ip -c a
                  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
                      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
                      inet 127.0.0.1/8 scope host lo
                         valid_lft forever preferred_lft forever
                      inet6 ::1/128 scope host 
                         valid_lft forever preferred_lft forever
                  2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br1 state UP group default qlen 1000
                      link/ether 1c:61:b4:6d:38:4f brd ff:ff:ff:ff:ff:ff
                      inet6 fe80::31ca:9227:dcb3:d09e/64 scope link noprefixroute 
                         valid_lft forever preferred_lft forever
                  3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
                      link/ether a8:a1:59:6e:1f:8b brd ff:ff:ff:ff:ff:ff
                  4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
                      link/ether 52:54:00:88:b4:b4 brd ff:ff:ff:ff:ff:ff
                      inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
                         valid_lft forever preferred_lft forever
                  7: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
                      link/ether fe:54:00:33:3c:4b brd ff:ff:ff:ff:ff:ff
                      inet6 fe80::fc54:ff:fe33:3c4b/64 scope link 
                         valid_lft forever preferred_lft forever
                  8: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
                      link/ether fe:54:00:50:81:3f brd ff:ff:ff:ff:ff:ff
                      inet6 fe80::fc54:ff:fe50:813f/64 scope link 
                         valid_lft forever preferred_lft forever
                  9: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
                      link/ether 0a:d4:78:1f:cd:41 brd ff:ff:ff:ff:ff:ff
                      inet 100.76.25.213/10 brd 100.127.255.255 scope global dynamic noprefixroute br0
                         valid_lft 294sec preferred_lft 294sec
                      inet6 2406:2d40:4100:8fb2:19c5:376e:1317:8ae1/64 scope global temporary dynamic 
                         valid_lft 197sec preferred_lft 47sec
                      inet6 2406:2d40:4100:8fb2:8d4:78ff:fe1f:cd41/64 scope global dynamic mngtmpaddr 
                         valid_lft 197sec preferred_lft 47sec
                      inet6 fe80::8d4:78ff:fe1f:cd41/64 scope link 
                         valid_lft forever preferred_lft forever
                  10: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
                      link/ether d2:da:46:a2:b2:3e brd ff:ff:ff:ff:ff:ff
                      inet 192.168.1.120/24 brd 192.168.1.255 scope global noprefixroute br1
                         valid_lft forever preferred_lft forever
                      inet6 fe80::d0da:46ff:fea2:b23e/64 scope link 
                         valid_lft forever preferred_lft forever
                  ubuntu@ubuntu:/etc/netplan$ bridge link
                  2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br1 state forwarding priority 32 cost 4 
                  3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100 
                  ubuntu@ubuntu:/etc/netplan$ cat 01-network-manager-all.yaml
                  # Let NetworkManager manage all devices on this system
                  network:
                    version: 2
                    renderer: NetworkManager
                    ethernets:
                      enp2s0:
                        dhcp4: true 
                      enp3s0:
                        dhcp4: true
                    bridges:
                      br0:
                        dhcp4: true
                        interfaces:
                          - enp3s0
                      br1:
                        dhcp4: false
                        addresses: [192.168.1.120/24]
                        gateway4: 192.168.1.1
                        nameservers:
                          addresses: [8.8.8.8,8.8.8.4]
                        interfaces:
                          - enp2s0
                  ubuntu@ubuntu:/etc/netplan$ ip r
                  default via 100.64.0.1 dev br0 proto dhcp metric 425 
                  default via 192.168.1.1 dev br1 proto static metric 20426 
                  34.120.255.244 dev br0 proto dhcp scope link metric 425 
                  100.64.0.0/10 dev br0 proto kernel scope link src 100.76.25.213 metric 425 
                  169.254.0.0/16 dev virbr0 scope link metric 1000 linkdown 
                  192.168.1.0/24 dev br1 proto kernel scope link src 192.168.1.120 metric 426 
                  192.168.100.1 dev br0 proto dhcp scope link metric 425 
                  192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown 
                  
                  V 1 Reply Last reply Reply Quote 0
                  • V
                    viragomann @eiger3970 0
                    last edited by

                    @eiger3970-0
                    The "ip a" output doesn't look plausible to me. If a single NIC is slave in a bridge, I'd expect that the bridge takes over the MAC from the NIC.

                    Also there should not be any IP settings on the NIC port. However, your enp2s0 shows IPv6 settings.

                    Also I cannot see that any of your virtual interfaces is connected to one of the bridges. So it's expected that pinging doesn't work.
                    Did you set the bridge name in VMM?

                    E 1 Reply Last reply Reply Quote 1
                    • E
                      eiger3970 0 @viragomann
                      last edited by

                      @viragomann Maybe I'm missing the connection, but not sure what to do? I connect NIC0-br0 and NIC1-br1. Maybe I need something like NIC0-br0-vtnet0 and NIC1-br1-vtnet1?

                      I create bridges for the VMM, in this order.
                      1: Ubuntu 22.04 Host machine:
                      sudo ip link set dev enp3s0 master br0
                      sudo ip link set dev enp2s0 master br1

                      2: Ubuntu 22.04 Host machine:
                      /etc/netplan/01-network-manager-all.yaml

                      # Let NetworkManager manage all devices on this system
                      network:
                        version: 2
                        renderer: NetworkManager
                        ethernets:
                          enp2s0:
                            dhcp4: true 
                          enp3s0:
                            dhcp4: true
                        bridges:
                          br0:
                            dhcp4: true
                            interfaces:
                              - enp3s0
                          br1:
                            dhcp4: false
                            addresses: [192.168.1.120/24]
                            gateway4: 192.168.1.1
                            nameservers:
                              addresses: [8.8.8.8,8.8.8.4]
                            interfaces:
                              - enp2s0
                      

                      3: Virtual Machine Manager 4.0.0:
                      Virtual Network Interface > Network Source: Bridge device... > Device name: br0.
                      Add Hardware > Virtual Network Interface > Network Source: Bridge device... > Device name: br1.

                      V 1 Reply Last reply Reply Quote 0
                      • V
                        viragomann @eiger3970 0
                        last edited by

                        @eiger3970-0 said in What is the VMM Network selection for a LAN and WAN NIC:

                        3: Virtual Machine Manager 4.0.0:
                        Virtual Network Interface > Network Source: Bridge device... > Device name: br0.
                        Add Hardware > Virtual Network Interface > Network Source: Bridge device... > Device name: br1.

                        That's how I did it as well. But running "ip a", my virtual NICs show the bridges they're connected to:

                        f5965430-28ed-4c59-a64a-7439ae25e5d5-image.png

                        However, as already mentioned, I don't use the Network Manager.

                        Do you use the host also for other purposes? If it's only for virtualization I'd consider to install Proxmox or alike on it.

                        E 1 Reply Last reply Reply Quote 1
                        • E
                          eiger3970 0 @viragomann
                          last edited by

                          @viragomann I just use the host machine as a personal desktop.
                          I previously ran Proxmox which ran nicely, however had to run Ubuntu as a vm.
                          My hope is to run Ubuntu with better performance not being a vm any longer.

                          E 1 Reply Last reply Reply Quote 0
                          • E
                            eiger3970 0 @eiger3970 0
                            last edited by eiger3970 0

                            @eiger3970-0 I've reinstalled the Ubuntu 22.04, VMM 4.0.0 and VM pfSense.
                            Ubuntu 192.168.1.120 pings the Internet (very slow connection though), vm router and LAN devices.
                            VM router 192.168.1.1 pings Ubuntu and LAN devices, however not the Internet.
                            This tells me Internet is running via the Ubuntu 22.04 iptables, rather than via the vm router.
                            Somehow I think I'm missing bridging the WAN interface NIC0-br0-vm router?
                            Here's my latest configuration:

                            root@ubuntu:/etc/netplan# cat 50-cloud-init.yaml
                            network:
                              version: 2
                              renderer: networkd
                            
                              ethernets:
                                enp2s0:
                                  dhcp4: false
                                  dhcp6: false
                                enp3s0:
                                  dhcp4: false
                                  dhcp6: false
                            
                              bridges:
                                br0:
                                  interfaces: [enp3s0]
                                  dhcp4: true
                                  dhcp6: true
                                br1:
                                  interfaces: [enp2s0]
                                  addresses: [192.168.1.120/24]
                                  # gateway4 is deprecated, use routes instead.
                                  routes:
                                  - to: default
                                    via: 192.168.1.1
                                    metric: 100
                                    on-link: true
                                  mtu: 1500
                                  nameservers:
                                    addresses: [8.8.8.8]
                                  parameters:
                                    stp: true
                                    forward-delay: 4
                                  dhcp4: no
                                  dhcp6: no        
                            
                            root@ubuntu:/etc/netplan# ip -c a
                            1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
                                link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
                                inet 127.0.0.1/8 scope host lo
                                   valid_lft forever preferred_lft forever
                                inet6 ::1/128 scope host 
                                   valid_lft forever preferred_lft forever
                            2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br1 state UP group default qlen 1000
                                link/ether 1c:61:b4:6d:38:4f brd ff:ff:ff:ff:ff:ff
                            3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
                                link/ether a8:a1:59:6e:1f:8b brd ff:ff:ff:ff:ff:ff
                            4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
                                link/ether 52:54:00:a9:81:30 brd ff:ff:ff:ff:ff:ff
                                inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
                                   valid_lft forever preferred_lft forever
                            8: vnet3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
                                link/ether fe:54:00:d4:d4:20 brd ff:ff:ff:ff:ff:ff
                                inet6 fe80::fc54:ff:fed4:d420/64 scope link 
                                   valid_lft forever preferred_lft forever
                            9: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
                                link/ether e2:73:dd:ac:ba:e0 brd ff:ff:ff:ff:ff:ff
                                inet 100.76.25.213/10 metric 100 brd 100.127.255.255 scope global dynamic br0
                                   valid_lft 187sec preferred_lft 187sec
                                inet6 2406:2d40:4100:8fb2:e073:ddff:feac:bae0/64 scope global dynamic mngtmpaddr noprefixroute 
                                   valid_lft 233sec preferred_lft 83sec
                                inet6 fe80::e073:ddff:feac:bae0/64 scope link 
                                   valid_lft forever preferred_lft forever
                            10: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
                                link/ether ce:08:2f:30:4a:71 brd ff:ff:ff:ff:ff:ff
                                inet 192.168.1.120/24 brd 192.168.1.255 scope global br1
                                   valid_lft forever preferred_lft forever
                                inet6 fe80::cc08:2fff:fe30:4a71/64 scope link 
                                   valid_lft forever preferred_lft forever
                            11: vnet4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br1 state UNKNOWN group default qlen 1000
                                link/ether fe:54:00:95:1e:e5 brd ff:ff:ff:ff:ff:ff
                                inet6 fe80::fc54:ff:fe95:1ee5/64 scope link 
                                   valid_lft forever preferred_lft forever
                            
                            root@ubuntu:/etc/netplan# ip r
                            default via 192.168.1.1 dev br1 proto static metric 100 onlink 
                            default via 100.64.0.1 dev br0 proto dhcp src 100.76.25.213 metric 100 
                            1.1.1.1 via 100.64.0.1 dev br0 proto dhcp src 100.76.25.213 metric 100 
                            8.8.8.8 via 100.64.0.1 dev br0 proto dhcp src 100.76.25.213 metric 100 
                            34.120.255.244 dev br0 proto dhcp scope link src 100.76.25.213 metric 100 
                            100.64.0.0/10 dev br0 proto kernel scope link src 100.76.25.213 metric 100 
                            100.64.0.1 dev br0 proto dhcp scope link src 100.76.25.213 metric 100 
                            169.254.0.0/16 dev virbr0 scope link metric 1000 linkdown 
                            192.168.1.0/24 dev br1 proto kernel scope link src 192.168.1.120 
                            192.168.100.1 dev br0 proto dhcp scope link src 100.76.25.213 metric 100 
                            192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown 
                            
                            root@ubuntu:/etc/netplan# bridge link
                            2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br1 state forwarding priority 32 cost 4 
                            3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 4 
                            8: vnet3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100 
                            11: vnet4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br1 state forwarding priority 32 cost 100 
                            
                            V 1 Reply Last reply Reply Quote 0
                            • V
                              viragomann @eiger3970 0
                              last edited by

                              @eiger3970-0 said in What is the VMM Network selection for a LAN and WAN NIC:

                              buntu 192.168.1.120 pings the Internet (very slow connection though)

                              It has two routes for upstream connection. The one via the pfSense VM might fail.

                              VM router 192.168.1.1 pings Ubuntu and LAN devices, however not the Internet.

                              What is its WAN configuration and default gateway?

                              E 1 Reply Last reply Reply Quote 1
                              • E
                                eiger3970 0 @viragomann
                                last edited by eiger3970 0

                                @viragomann
                                What are the 2 routes for upstream connection, I don't clearly see them?

                                I think these are the LAN and WAN details:
                                Ubuntu 22.04 Desktop > Browser > 192.168.1.1 > Enter > pfSense > Lobby > Dashboard > Interfaces > LAN 10Gbase-T <full-duplex> 192.168.1.1 track6.
                                Ubuntu 22.04 Desktop > Browser > 192.168.1.1 > Enter > pfSense > Lobby > Dashboard > Interfaces > WAN 10Gbase-T <full-duplex> 100.76.25.213 2406.2d40:4100:8fb2:5054:ff:fed4:d420

                                I think these are the Gateways details:
                                Ubuntu 22.04 Desktop > Browser > 192.168.1.1 > Enter > pfSense > Lobby > Dashboard > Gateways > WAN_DHCP6 fe80::200:5eff:fe00:101
                                Ubuntu 22.04 Desktop > Browser > 192.168.1.1 > Enter > pfSense > Lobby > Dashboard > Gateways > WAN_DHCP 100.64.0.1.

                                These are all default settings that appeared in pfSense after I configured Ubuntu 22.04 Desktop's Netplan and turned off NetworkManager as per this tutorial.

                                V 1 Reply Last reply Reply Quote 0
                                • V
                                  viragomann @eiger3970 0
                                  last edited by

                                  @eiger3970-0

                                  default via 192.168.1.1 dev br1 proto static metric 100 onlink
                                  default via 100.64.0.1 dev br0 proto dhcp src 100.76.25.213 metric 100

                                  You have the first one set manually. The latter is due to the DHCP client enabled on br0. So the bridge gets a config from the ISP router presumably.

                                  If you want to go out via pfSense disable the DHCP client on br0 and enable it on pfSense WAN, assuming this is connected to br0.

                                  E 1 Reply Last reply Reply Quote 1
                                  • E
                                    eiger3970 0 @viragomann
                                    last edited by

                                    @viragomann Thank you for the clarification.
                                    This is what I've been trying to do, to have traffic via the vm router.
                                    Any suggestions, as if I disable the br0's DHCP, it must need some address for a static IP?
                                    How do I check the vm router is connected to br0?
                                    How do I connect the vm router to br0?
                                    How do I check the vm router's WAN is on DHCP? The vm router's Interfaces: [WAN] IPv4 Configuration Type, does say it's on DHCP, so this should be ok?

                                    E 1 Reply Last reply Reply Quote 0
                                    • E
                                      eiger3970 0 @eiger3970 0
                                      last edited by

                                      @eiger3970-0 I've configured Ubuntu 22.04 to use renderer networkd rather than NetworkManager.
                                      I can now route the Ubuntu host via the vm router for Internet and LAN access.

                                      However, only some Internet loads and ping 8.8.8.8 has 100% packet loss?

                                      I think the pfSense configuration needs some details I'm missing.

                                      Any paid support to finalise my configuration please?
                                      I've set LAN and WAN to static IPs, but will probably fail once the ISP changes the public IP.

                                      1 Reply Last reply Reply Quote 0
                                      • P
                                        Patch @eiger3970 0
                                        last edited by Patch

                                        @eiger3970-0 said in What is the VMM Network selection for a LAN and WAN NIC:

                                        This updated diagram might help for clarity?

                                        That diagram looks wrong to me. Specifically
                                        Wan side would normally be
                                        ISP <-> Modem (Bridged router) <-> Nic0/eth0/snp3s0 <-> br0 <-> pfSenseVM (and no other VM or hypervisor)

                                        Lan side would normally be
                                        pfSenseVM <-> br1 <-> other VMs, Hypervisor, and Nic1/eth1/enp2s0 <-> external switch <-> Wired Lan devices, Wifi AP

                                        So in your diagram I don't like br0 connection to VM x and VM y

                                        Nor do I like the non specific connection of the Hypervisor to the pfsense router

                                        • The hypervisor network access should be explicitly via the Lan / br1. In Proxmox that is set by assigning an IP address to the Vbr1. Less sure how to do it directly in Ubuntu 22.04.
                                        • The hypervisor also has VM access via the VM console and Qemu but that is separate to network access
                                        E 1 Reply Last reply Reply Quote 1
                                        • E
                                          eiger3970 0 @Patch
                                          last edited by

                                          @Patch thanks, your reply seems right.
                                          The setup is now working, but I haven’t got my head 100% around it yet.
                                          I’m figuring out how to make a clear and correct topology for easy understanding and replication.

                                          E 1 Reply Last reply Reply Quote 0
                                          • E
                                            eiger3970 0 @eiger3970 0
                                            last edited by eiger3970 0

                                            @eiger3970-0 I have created a new topology map. topology map.

                                            However, I am unclear the the packet routes?
                                            As per ip route, line 2 is the default route of this host machine Ubuntu 22.04 LTS Desktop.
                                            100.64.0.1 came from a DHCP, but where is the DHCP from, the VM router or the ISP?

                                            ubuntu@ubuntu:/etc/netplan$ ip -c a
                                            1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
                                                link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
                                                inet 127.0.0.1/8 scope host lo
                                                   valid_lft forever preferred_lft forever
                                                inet6 ::1/128 scope host 
                                                   valid_lft forever preferred_lft forever
                                            2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master local-br1.1 state UP group default qlen 1000
                                                link/ether 46:9e:01:27:db:0d brd ff:ff:ff:ff:ff:ff permaddr 1c:61:b4:6d:38:4f
                                            3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
                                                link/ether a8:a1:59:6e:1f:8b brd ff:ff:ff:ff:ff:ff
                                                inet 100.76.25.213/10 brd 100.127.255.255 scope global dynamic noprefixroute enp3s0
                                                   valid_lft 168sec preferred_lft 168sec
                                                inet6 2406:2d40:4100:8fb2:7b54:ac94:277b:140f/64 scope global temporary deprecated dynamic 
                                                   valid_lft 234sec preferred_lft 0sec
                                                inet6 2406:2d40:4100:8fb2:f995:e028:b158:4865/64 scope global dynamic mngtmpaddr noprefixroute 
                                                   valid_lft 234sec preferred_lft 84sec
                                                inet6 fe80::8cc1:c5d0:5e3e:28b1/64 scope link noprefixroute 
                                                   valid_lft forever preferred_lft forever
                                            4: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
                                                link/ether 52:54:00:94:0d:bb brd ff:ff:ff:ff:ff:ff
                                                inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
                                                   valid_lft forever preferred_lft forever
                                            5: local-br1.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
                                                link/ether 2e:f6:bf:42:0c:f8 brd ff:ff:ff:ff:ff:ff
                                                inet 192.168.1.120/24 brd 192.168.1.255 scope global noprefixroute local-br1.1
                                                   valid_lft forever preferred_lft forever
                                            6: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master local-br1.1 state UNKNOWN group default qlen 1000
                                                link/ether fe:54:00:3d:53:41 brd ff:ff:ff:ff:ff:ff
                                                inet6 fe80::fc54:ff:fe3d:5341/64 scope link 
                                                   valid_lft forever preferred_lft forever
                                            7: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master virbr0 state UNKNOWN group default qlen 1000
                                                link/ether fe:54:00:a3:f6:c4 brd ff:ff:ff:ff:ff:ff
                                                inet6 fe80::fc54:ff:fea3:f6c4/64 scope link 
                                                   valid_lft forever preferred_lft forever
                                            16: vnet3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master local-br1.1 state UNKNOWN group default qlen 1000
                                                link/ether fe:54:00:8d:31:06 brd ff:ff:ff:ff:ff:ff
                                                inet6 fe80::fc54:ff:fe8d:3106/64 scope link 
                                                   valid_lft forever preferred_lft forever
                                            17: vnet4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master local-br1.1 state UNKNOWN group default qlen 1000
                                                link/ether fe:54:00:f7:f9:e5 brd ff:ff:ff:ff:ff:ff
                                                inet6 fe80::fc54:ff:fef7:f9e5/64 scope link 
                                                   valid_lft forever preferred_lft forever
                                            18: vnet5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master local-br1.1 state UNKNOWN group default qlen 1000
                                                link/ether fe:54:00:b6:35:37 brd ff:ff:ff:ff:ff:ff
                                                inet6 fe80::fc54:ff:feb6:3537/64 scope link 
                                                   valid_lft forever preferred_lft forever
                                            
                                            ubuntu@ubuntu:/etc/netplan$ arp -n
                                            Address                  HWtype  HWaddress           Flags Mask            Iface
                                            192.168.1.160            ether   3c:2a:f4:60:b5:7b   C                     local-br1.1
                                            192.168.1.1              ether   52:54:00:3d:53:41   C                     local-br1.1
                                            192.168.122.149          ether   52:54:00:a3:f6:c4   C                     virbr0
                                            192.168.1.102            ether   52:54:00:8d:31:06   C                     local-br1.1
                                            192.168.1.100            ether   1c:ca:e3:77:c5:53   C                     local-br1.1
                                            100.64.0.1               ether   00:00:5e:00:01:01   C                     enp3s0
                                            
                                            ubuntu@ubuntu:/etc/netplan$ ip route
                                            default via 100.64.0.1 dev enp3s0 proto dhcp metric 102 
                                            34.120.255.244 dev enp3s0 proto dhcp scope link metric 102 
                                            100.64.0.0/10 dev enp3s0 proto kernel scope link src 100.76.25.213 metric 102 
                                            169.254.0.0/16 dev virbr0 scope link metric 1000 
                                            192.168.1.0/24 dev local-br1.1 proto kernel scope link src 192.168.1.120 metric 425 
                                            192.168.100.1 dev enp3s0 proto dhcp scope link metric 102 
                                            192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 
                                            
                                            E 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.