Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense not enabling port

    Scheduled Pinned Locked Moved General pfSense Questions
    145 Posts 4 Posters 10.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • georgelzaG
      georgelza @Gblenn
      last edited by

      @Gblenn let me try.
      it was set to 10 GBS FDX.

      switched it to Auto, still same.

      G

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        It has to be auto at both ends to use that. Otherwise it should link at 10G fixed on both ends.

        Probably need to query the module to make sure it at least sees signal on the fiber. I don't have anything to test that with but maybe: ethtool --show-module enp4s0f0

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          Just to confirm this is Proxmox installed on the same Topton hardware that pfSense couldn't get link on?

          G 1 Reply Last reply Reply Quote 0
          • G
            Gblenn @stephenw10
            last edited by Gblenn

            @stephenw10 said in pfSense not enabling port:

            Just to confirm this is Proxmox installed on the same Topton hardware that pfSense couldn't get link on?

            No it's a separate machine, thinking it should work in Proxmox at least...

            It has to be auto at both ends to use that. Otherwise it should link at 10G fixed on both ends.

            Probably need to query the module to make sure it at least sees signal on the fiber. I don't have anything to test that >with but maybe: ethtool --show-module enp4s0f0

            It did list this at the end as per the post further up:

                Link detected: yes

            And Proxmox log was showing interface going up and down...

            Oct 31 16:46:17 pmox1 kernel: ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
            Oct 31 16:46:17 pmox1 kernel: vmbr30: port 1(enp4s0f0) entered disabled state
            Oct 31 16:46:17 pmox1 kernel: ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
            Oct 31 16:46:17 pmox1 kernel: vmbr30: port 1(enp4s0f0) entered blocking state
            Oct 31 16:46:17 pmox1 kernel: vmbr30: port 1(enp4s0f0) entered forwarding state
            Oct 31 16:46:18 pmox1 kernel: ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
            Oct 31 16:46:18 pmox1 kernel: ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
            Oct 31 16:46:18 pmox1 kernel: ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
            Oct 31 16:46:18 pmox1 kernel: ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX

            Perhaps time to try a different module, or set of modules / fiber...

            georgelzaG 1 Reply Last reply Reply Quote 0
            • georgelzaG
              georgelza @Gblenn
              last edited by

              @Gblenn

              I've already swopped the fiber out.

              And tried the pfSense which was 2 modules there... and that fiber. then ran into issues and thought if we think this is pfSense not liking it lets me jst confirm things work, in a more compatible environment, so installed transceivers into my 2nd Topton that runs pmox. patched that into my Unifi Pro Max port 26 which SFP+ (SFP1), port 25 is (SFP0).

              and ye. we not getting much further.

              My unifi switch is reporting the correct SFP+ transceivers so can't image there being a problem with them, and well, they brand brand new.

              G

              G 1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Ok but it's the same model Topton device? Same NIC?

                Yeah, seeing the NIC logged as changing link state like that sure looks like it should be working....

                georgelzaG 1 Reply Last reply Reply Quote 1
                • G
                  Gblenn @georgelza
                  last edited by

                  @georgelza Just realized one more thing that you might try in Proxmox...

                  What do you see from this command?
                  dmesg | grep ixgbe

                  georgelzaG 1 Reply Last reply Reply Quote 0
                  • georgelzaG
                    georgelza @Gblenn
                    last edited by

                    @Gblenn said in pfSense not enabling port:

                    dmesg | grep ixgbe

                    [175483.486743] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175484.110776] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175484.214719] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175485.462768] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175485.670734] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175485.879762] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175485.982711] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175486.398837] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175486.502809] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175486.606757] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175486.710853] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175487.230749] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175487.438681] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175489.830832] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175489.934714] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175498.150577] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175498.255548] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175502.414503] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175502.518579] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
                    G 1 Reply Last reply Reply Quote 0
                    • georgelzaG
                      georgelza @stephenw10
                      last edited by

                      @stephenw10

                      pretty much yes, cpu and memory and nvme size differs, thats all.

                      G

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Link flapping every few seconds like that sure looks like a low level link negotiation issue.

                        georgelzaG 1 Reply Last reply Reply Quote 0
                        • georgelzaG
                          georgelza @stephenw10
                          last edited by

                          @stephenw10

                          so thesis the setting on the Unifi switch atm.

                          6fa94705-d818-4af8-b992-ed77e9255aaf-image.png

                          27a922a6-920a-432c-ad96-157bb979e4af-image.png

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            I would try setting both ends to 10G FDX fixed since it seems to be struggling to negotiate.

                            georgelzaG 1 Reply Last reply Reply Quote 0
                            • georgelzaG
                              georgelza @stephenw10
                              last edited by

                              @stephenw10

                              well first noticed is the native VLAN was wrong... fixed that bit did not change anything. ok so setting to 10GbE FDX on switch is easy, how do I accomplish the same on proxmox ?

                              G

                              G 1 Reply Last reply Reply Quote 0
                              • G
                                Gblenn @georgelza
                                last edited by

                                @georgelza I thought it was set to manual already, otherwise something like

                                ethtool -s enp4s0f0 speed 10000 duplex full autoneg off

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Yup at least as test just set it manually at the CLI. That won't survive a reboot of course.

                                  georgelzaG 1 Reply Last reply Reply Quote 0
                                  • georgelzaG
                                    georgelza @stephenw10
                                    last edited by

                                    @stephenw10
                                    well it seems I can now ping 172.16.30.1, from both the pmox shell, I also created a ubuntu guest vm, i can from inside here ping the 30.1 also

                                    so we def making progress.

                                    i've down a ifconfig <interface> down and a back up, not picking up a ip yet.

                                    G

                                    G 1 Reply Last reply Reply Quote 0
                                    • G
                                      Gblenn @georgelza
                                      last edited by

                                      @georgelza said in pfSense not enabling port:

                                      @Gblenn said in pfSense not enabling port:

                                      dmesg | grep ixgbe

                                      [175483.486743] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175484.110776] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175484.214719] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175485.462768] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175485.670734] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175485.879762] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175485.982711] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175486.398837] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175486.502809] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175486.606757] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175486.710853] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175487.230749] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175487.438681] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175489.830832] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175489.934714] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175498.150577] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175498.255548] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[175502.414503] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[175502.518579] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down

                                      It would be interesting to see what it sais before it start flapping like that... Of there is anything that could provide a clue whether there is some module inocmpatibility for example. Some cards check the module to see if it is in the list of "supported devices", which it will announce here if that is the case...

                                      I wonder if you remove the module, issue dmesg | grep ixgbe again.
                                      Then plug it in and immediately run dmesg | grep ixgbe

                                      1 Reply Last reply Reply Quote 0
                                      • G
                                        Gblenn @georgelza
                                        last edited by Gblenn

                                        @georgelza said in pfSense not enabling port:

                                        @stephenw10
                                        well it seems I can now ping 172.16.30.1, from both the pmox shell, I also created a ubuntu guest vm, i can from inside here ping the 30.1 also

                                        so we def making progress.

                                        i've down a ifconfig <interface> down and a back up, not picking up a ip yet.

                                        G

                                        Proxmox should be able to ping 172.16.30.1, since that is allowed according to your rules, and it is pfsense responding on that IP. You are likely pinging from 172.16.10 which is the Proxmox host, and not the ix0 interface you are trying to fix.
                                        From the VM however, if you are able to ping, you should be good to go.

                                        What does dmesg | grep ixgbe on Proxmox host show you now?

                                        georgelzaG 1 Reply Last reply Reply Quote 0
                                        • georgelzaG
                                          georgelza @Gblenn
                                          last edited by georgelza

                                          @Gblenn

                                          [178207.262591] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[178207.470686] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[178207.574588] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[178207.782573] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[178207.991549] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[178208.094751] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[178208.198712] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[178209.342607] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[178209.446782] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[178210.902621] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[178211.006577] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[178212.462795] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[178212.566761] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[178215.478660] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[178215.790564] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[178218.702566] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[178218.910620] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[178219.118550] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[178219.222534] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Down
[178219.430550] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX

                                          this is the guest vm created

                                          ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether bc:24:11:c0:e8:1e brd ff:ff:ff:ff:ff:ff
    inet 172.16.10.201/24 metric 100 brd 172.16.10.255 scope global dynamic ens18
       valid_lft 5516sec preferred_lft 5516sec
    inet6 fe80::be24:11ff:fec0:e81e/64 scope link
       valid_lft forever preferred_lft forever
4: ens19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether bc:24:11:8f:4d:2a brd ff:ff:ff:ff:ff:ff
    inet6 fe80::be24:11ff:fe8f:4d2a/64 scope link
       valid_lft forever preferred_lft forever

                                          and then the pmox shell

                                          root@pmox1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether a8:b8:e0:02:a3:71 brd ff:ff:ff:ff:ff:ff
3: enp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a8:b8:e0:02:a3:72 brd ff:ff:ff:ff:ff:ff
4: enp5s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a8:b8:e0:02:a3:73 brd ff:ff:ff:ff:ff:ff
5: enp6s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a8:b8:e0:02:a3:74 brd ff:ff:ff:ff:ff:ff
6: enp4s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr30 state DOWN group default qlen 1000
    link/ether a8:b8:e0:05:f0:91 brd ff:ff:ff:ff:ff:ff
7: enp4s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a8:b8:e0:05:f0:92 brd ff:ff:ff:ff:ff:ff
8: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether a8:b8:e0:02:a3:71 brd ff:ff:ff:ff:ff:ff
    inet 172.16.10.51/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::aab8:e0ff:fe02:a371/64 scope link 
       valid_lft forever preferred_lft forever
10: vmbr30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether a8:b8:e0:05:f0:91 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::aab8:e0ff:fe05:f091/64 scope link 
       valid_lft forever preferred_lft forever
15: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr100i0 state UNKNOWN group default qlen 1000
    link/ether fe:6b:ee:5d:ac:58 brd ff:ff:ff:ff:ff:ff
16: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ba:54:5b:a0:3b:26 brd ff:ff:ff:ff:ff:ff
17: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether de:8e:79:f1:ec:bd brd ff:ff:ff:ff:ff:ff
18: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000
    link/ether ba:54:5b:a0:3b:26 brd ff:ff:ff:ff:ff:ff
19: tap100i1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr100i1 state UNKNOWN group default qlen 1000
    link/ether 46:27:5f:a9:be:cc brd ff:ff:ff:ff:ff:ff
20: fwbr100i1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether b6:76:d9:f3:3c:fe brd ff:ff:ff:ff:ff:ff
21: fwpr100p1@fwln100i1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr30 state UP group default qlen 1000
    link/ether 96:3e:4f:c9:f2:84 brd ff:ff:ff:ff:ff:ff
22: fwln100i1@fwpr100p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i1 state UP group default qlen 1000
    link/ether b6:76:d9:f3:3c:fe brd ff:ff:ff:ff:ff:ff
                                          G georgelzaG 3 Replies Last reply Reply Quote 0
                                          • G
                                            Gblenn @georgelza
                                            last edited by

                                            Ok this is new and positive...

                                            10: vmbr30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
                                            link/ether a8:b8:e0:05:f0:91 brd ff:ff:ff:ff:ff:ff

                                            However it is strange the network device still sais NO-CARRIER...

                                            6: enp4s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr30 state DOWN group default qlen 1000
                                            link/ether a8:b8:e0:05:f0:91 brd ff:ff:ff:ff:ff:ff

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.