Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Does not connect to switch after switch reset

    Scheduled Pinned Locked Moved General pfSense Questions
    35 Posts 3 Posters 1.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • rikinn55R
      rikinn55 @bingo600
      last edited by

      @bingo600
      I did already update the switch firmware.
      The Qotom is 2.5gb and is the reason for the TP Link purchase.
      Replication: Here is what I did.
      Disconnect from TP Link to the Ubiquity 1gb switch = same issue.
      Reboot the Qotom get a link.
      Disconnect from Ubiquity and attach to Netgear dumb switch = same issue.
      Reboot the Qotom get a link.
      Disconnect and plug into the TP Link = same issue.
      Turn off the Qotom.
      When the Qotom is not running the port show connections to switch
      So I disconnect from TP Link port 1 and attached to port 6 lights linked up.
      Disconnected from TP Link port 6 attached to Ubiquity ports light up.
      Reattach to TP-Link and boot the Qotom
      Note: Keep in mind the Qotom has 5 ports and only ports 1 (WAN) and ports 2 (LAN) are being monitored by PfSense. Ports 3, 4, and 5 are not even set in PfSense. With that in mind after the Qotom was booted 1:
      Disconnected LAN and pluged into Qotom port 3 linked up with TP Link.
      Disconnect port 3 and plugged back to LAN no link established.
      Disconnected LAN plugged into Qotom port 4 linked up with TP Link
      Disconnected port 4 plugged into LAN no link.
      My conclusion is that the problem is not hardware. I believe the last test I did proved that it is something in the PfSense settings, but what.
      How can I get a print out of the settings in PfSense so I can share them?

      Rikinn

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        So they are Intel igc(4) NICs?

        Have you set a link speed as I described above?

        rikinn55R 1 Reply Last reply Reply Quote 0
        • rikinn55R
          rikinn55 @stephenw10
          last edited by

          @stephenw10
          I forgot to do that. I will do it and get back on what happens.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            I'm asking if you have already set anything because that could affect the link negotiation. By default most NICs should link OK.

            Also please confirm what NICs the firewall is using because there are several 2.5G NIC types it could be.

            rikinn55R 1 Reply Last reply Reply Quote 0
            • rikinn55R
              rikinn55 @stephenw10
              last edited by

              @stephenw10
              All nic's are auto negotiate. None are set to static link speed. The Qotom has all Intel nics. I was very careful on that when I was looking for a unit. I had already experience problems when not an Intel nic.

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Ok, so they are all Intel igc(4)?

                Are you running 2.6?

                rikinn55R 1 Reply Last reply Reply Quote 0
                • rikinn55R
                  rikinn55 @stephenw10
                  last edited by

                  @stephenw10
                  PfSense Scale
                  23.01-RELEASE (amd64)
                  built on Fri Feb 10 20:06:33 UTC 2023
                  FreeBSD 14.0-CURRENT

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    What NICs are they? What is returned by: pciconf -lv?

                    rikinn55R 1 Reply Last reply Reply Quote 0
                    • rikinn55R
                      rikinn55 @stephenw10
                      last edited by stephenw10

                      @stephenw10 said in Does not connect to switch after switch reset:

                      pciconf -lv

                      hostb0@pci0:0:0:0:	class=0x060000 rev=0x06 hdr=0x00 vendor=0x8086 device=0x31f0 subvendor=0x0000 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Gemini Lake Host Bridge'
    class      = bridge
    subclass   = HOST-PCI
vgapci0@pci0:0:2:0:	class=0x030000 rev=0x06 hdr=0x00 vendor=0x8086 device=0x3185 subvendor=0x0000 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'GeminiLake [UHD Graphics 600]'
    class      = display
    subclass   = VGA
none0@pci0:0:15:0:	class=0x078000 rev=0x06 hdr=0x00 vendor=0x8086 device=0x319a subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Celeron/Pentium Silver Processor Trusted Execution Engine Interface'
    class      = simple comms
ahci0@pci0:0:18:0:	class=0x010601 rev=0x06 hdr=0x00 vendor=0x8086 device=0x31e3 subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Celeron/Pentium Silver Processor SATA Controller'
    class      = mass storage
    subclass   = SATA
pcib1@pci0:0:19:0:	class=0x060400 rev=0xf6 hdr=0x01 vendor=0x8086 device=0x31d8 subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Gemini Lake PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
pcib2@pci0:0:19:1:	class=0x060400 rev=0xf6 hdr=0x01 vendor=0x8086 device=0x31d9 subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Gemini Lake PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
pcib3@pci0:0:19:2:	class=0x060400 rev=0xf6 hdr=0x01 vendor=0x8086 device=0x31da subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Gemini Lake PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
pcib4@pci0:0:19:3:	class=0x060400 rev=0xf6 hdr=0x01 vendor=0x8086 device=0x31db subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Gemini Lake PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
pcib5@pci0:0:20:0:	class=0x060400 rev=0xf6 hdr=0x01 vendor=0x8086 device=0x31d6 subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Gemini Lake PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
pcib6@pci0:0:20:1:	class=0x060400 rev=0xf6 hdr=0x01 vendor=0x8086 device=0x31d7 subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Gemini Lake PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
xhci0@pci0:0:21:0:	class=0x0c0330 rev=0x06 hdr=0x00 vendor=0x8086 device=0x31a8 subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Celeron/Pentium Silver Processor USB 3.0 xHCI Controller'
    class      = serial bus
    subclass   = USB
isab0@pci0:0:31:0:	class=0x060100 rev=0x06 hdr=0x00 vendor=0x8086 device=0x31e8 subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Celeron/Pentium Silver Processor LPC Controller'
    class      = bridge
    subclass   = PCI-ISA
ichsmb0@pci0:0:31:1:	class=0x0c0500 rev=0x06 hdr=0x00 vendor=0x8086 device=0x31d4 subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Celeron/Pentium Silver Processor Gaussian Mixture Model'
    class      = serial bus
    subclass   = SMBus
igc0@pci0:1:0:0:	class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc1@pci0:2:0:0:	class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc2@pci0:3:0:0:	class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc3@pci0:4:0:0:	class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc4@pci0:5:0:0:	class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Ok, so 5 igb NICs. But you say it has 8 ports total? If it has some sprt of internal switch that could be a problem.

                        rikinn55R 1 Reply Last reply Reply Quote 0
                        • rikinn55R
                          rikinn55 @stephenw10
                          last edited by

                          @stephenw10
                          The Tp link has 8 and the qotom has 5

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator @rikinn55
                            last edited by

                            @rikinn55 said in Does not connect to switch after switch reset:

                            Qotom Mini PC w/8 ports

                            OK, that was just a typo then?

                            So what's logged when it disconnects and then re-links?

                            What does ifconfig -vvvm igc0 show? Assuming igc0 is one of the ports that's flapping?

                            1 Reply Last reply Reply Quote 0
                            • rikinn55R
                              rikinn55
                              last edited by

                              @stephenw10
                              PfS_Gen_Log.txt
                              igc1 flags=8863UP,BROADCAST,RUNNING.txt

                              Hope these are the two items you want. If not please let me know where to find what your looking for.

                              Thanks

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                Hmm, OK so all the igc ports behave the same?

                                Before the reboot in that log we can see igc flapping every 5s or so.

                                After rebooting we see it flap once during the boot process then it stays up for ~2mins. Does it remain linked correctly after that?

                                rikinn55R 2 Replies Last reply Reply Quote 0
                                • rikinn55R
                                  rikinn55 @stephenw10
                                  last edited by

                                  @stephenw10
                                  Yes it stays up every time. Seems a reboot of the Qotom is what is required. I tried reboot the TP Link but that failed getting it to relink.

                                  1 Reply Last reply Reply Quote 0
                                  • rikinn55R
                                    rikinn55 @stephenw10
                                    last edited by

                                    @stephenw10
                                    Didn't answer the first question. Keep in mind PfS isn't doing anything with ports 2-4. So if I pulg and unplug from them I always get a link right away. I have not set PfS to use any of those ports as LAN yet.

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      Ok, well first try re-assigning LAN as a different NIC and make sure that behaves the same.

                                      The only time I've anything like this was when setting igc to 'autoselect' rather than 'default' in the config after having linked at 1G.

                                      I note I do have hardware checksum off-loading disabled on test devices here. I wouldn't expect that to make any difference to linking but wort trying.

                                      You can check the sysctls for igc1 like:

                                      23.01-RELEASE][root@4100.stevew.lan]/root: sysctl hw.igc
hw.igc.max_interrupt_rate: 8000
hw.igc.eee_setting: 1
hw.igc.rx_process_limit: 100
hw.igc.sbp: 1
hw.igc.smart_pwr_down: 0
hw.igc.rx_abs_int_delay: 66
hw.igc.tx_abs_int_delay: 66
hw.igc.rx_int_delay: 0
hw.igc.tx_int_delay: 66
hw.igc.disable_crc_stripping: 0
[23.01-RELEASE][root@4100.stevew.lan]/root: sysctl dev.igc.0
dev.igc.0.wake: 0
dev.igc.0.interrupts.rx_desc_min_thresh: 0
dev.igc.0.interrupts.asserts: 4030
dev.igc.0.mac_stats.tso_txd: 0
dev.igc.0.mac_stats.tx_frames_1024_1522: 0
dev.igc.0.mac_stats.tx_frames_512_1023: 0
dev.igc.0.mac_stats.tx_frames_256_511: 3163
dev.igc.0.mac_stats.tx_frames_128_255: 0
dev.igc.0.mac_stats.tx_frames_65_127: 0
dev.igc.0.mac_stats.tx_frames_64: 22
dev.igc.0.mac_stats.mcast_pkts_txd: 22
dev.igc.0.mac_stats.bcast_pkts_txd: 3163
dev.igc.0.mac_stats.good_pkts_txd: 3185
dev.igc.0.mac_stats.total_pkts_txd: 3185
dev.igc.0.mac_stats.good_octets_txd: 1108458
dev.igc.0.mac_stats.good_octets_recvd: 0
dev.igc.0.mac_stats.rx_frames_1024_1522: 0
dev.igc.0.mac_stats.rx_frames_512_1023: 0
dev.igc.0.mac_stats.rx_frames_256_511: 0
dev.igc.0.mac_stats.rx_frames_128_255: 0
dev.igc.0.mac_stats.rx_frames_65_127: 0
dev.igc.0.mac_stats.rx_frames_64: 0
dev.igc.0.mac_stats.mcast_pkts_recvd: 0
dev.igc.0.mac_stats.bcast_pkts_recvd: 0
dev.igc.0.mac_stats.good_pkts_recvd: 0
dev.igc.0.mac_stats.total_pkts_recvd: 0
dev.igc.0.mac_stats.xoff_txd: 0
dev.igc.0.mac_stats.xoff_recvd: 0
dev.igc.0.mac_stats.xon_txd: 0
dev.igc.0.mac_stats.xon_recvd: 0
dev.igc.0.mac_stats.alignment_errs: 0
dev.igc.0.mac_stats.crc_errs: 0
dev.igc.0.mac_stats.recv_errs: 0
dev.igc.0.mac_stats.recv_jabber: 0
dev.igc.0.mac_stats.recv_oversize: 0
dev.igc.0.mac_stats.recv_fragmented: 0
dev.igc.0.mac_stats.recv_undersize: 0
dev.igc.0.mac_stats.recv_no_buff: 0
dev.igc.0.mac_stats.missed_packets: 0
dev.igc.0.mac_stats.defer_count: 0
dev.igc.0.mac_stats.sequence_errors: 0
dev.igc.0.mac_stats.symbol_errors: 0
dev.igc.0.mac_stats.collision_count: 0
dev.igc.0.mac_stats.late_coll: 0
dev.igc.0.mac_stats.multiple_coll: 0
dev.igc.0.mac_stats.single_coll: 0
dev.igc.0.mac_stats.excess_coll: 0
dev.igc.0.queue_rx_1.rx_irq: 0
dev.igc.0.queue_rx_1.rxd_tail: 128
dev.igc.0.queue_rx_1.rxd_head: 0
dev.igc.0.queue_rx_0.rx_irq: 0
dev.igc.0.queue_rx_0.rxd_tail: 128
dev.igc.0.queue_rx_0.rxd_head: 0
dev.igc.0.queue_tx_1.tx_irq: 0
dev.igc.0.queue_tx_1.txd_tail: 0
dev.igc.0.queue_tx_1.txd_head: 0
dev.igc.0.queue_tx_0.tx_irq: 0
dev.igc.0.queue_tx_0.txd_tail: 222
dev.igc.0.queue_tx_0.txd_head: 222
dev.igc.0.fc_low_water: 32752
dev.igc.0.fc_high_water: 32768
dev.igc.0.rx_control: 71335966
dev.igc.0.device_control: 1478231617
dev.igc.0.watchdog_timeouts: 0
dev.igc.0.rx_overruns: 0
dev.igc.0.link_irq: 66
dev.igc.0.dropped: 0
dev.igc.0.eee_control: 1
dev.igc.0.itr: 488
dev.igc.0.tx_abs_int_delay: 66
dev.igc.0.rx_abs_int_delay: 66
dev.igc.0.tx_int_delay: 66
dev.igc.0.rx_int_delay: 0
dev.igc.0.rs_dump: 0
dev.igc.0.reg_dump: General Registers
	CTRL	 581c0641
	STATUS	 40780683
	CTRL_EXIT	 10000040

Interrupt Registers
	ICR	 00000000

RX Registers
	RCTL	 0440801e
	RDLEN	 00004000
	RDH	 00000000
	RDT	 00000080
	RXDCTL	 02040808
	RDBAL	 02392000
	RDBAH	 00000000

TX Registers
	TCTL	 a503f0fa
	TDBAL	 02376000
	TDBAH	 00000000
	TDLEN	 00004000
	TDH	 000000de
	TDT	 000000de
	TXDCTL	 0201011f
	TDFH	 00000000
	TDFT	 00000000
	TDFHS	 00000000
	TDFPC	 00000000


dev.igc.0.fc: 3
dev.igc.0.debug: -1
dev.igc.0.nvm: -1
dev.igc.0.iflib.rxq1.rxq_fl0.buf_size: 2048
dev.igc.0.iflib.rxq1.rxq_fl0.credits: 128
dev.igc.0.iflib.rxq1.rxq_fl0.cidx: 0
dev.igc.0.iflib.rxq1.rxq_fl0.pidx: 128
dev.igc.0.iflib.rxq1.cpu: 1
dev.igc.0.iflib.rxq0.rxq_fl0.buf_size: 2048
dev.igc.0.iflib.rxq0.rxq_fl0.credits: 128
dev.igc.0.iflib.rxq0.rxq_fl0.cidx: 0
dev.igc.0.iflib.rxq0.rxq_fl0.pidx: 128
dev.igc.0.iflib.rxq0.cpu: 0
dev.igc.0.iflib.txq1.r_abdications: 0
dev.igc.0.iflib.txq1.r_restarts: 0
dev.igc.0.iflib.txq1.r_stalls: 0
dev.igc.0.iflib.txq1.r_starts: 0
dev.igc.0.iflib.txq1.r_drops: 0
dev.igc.0.iflib.txq1.r_enqueues: 0
dev.igc.0.iflib.txq1.ring_state: pidx_head: 0000 pidx_tail: 0000 cidx: 0000 state: IDLE
dev.igc.0.iflib.txq1.txq_cleaned: 0
dev.igc.0.iflib.txq1.txq_processed: 0
dev.igc.0.iflib.txq1.txq_in_use: 0
dev.igc.0.iflib.txq1.txq_cidx_processed: 0
dev.igc.0.iflib.txq1.txq_cidx: 0
dev.igc.0.iflib.txq1.txq_pidx: 0
dev.igc.0.iflib.txq1.no_tx_dma_setup: 0
dev.igc.0.iflib.txq1.txd_encap_efbig: 0
dev.igc.0.iflib.txq1.tx_map_failed: 0
dev.igc.0.iflib.txq1.no_desc_avail: 0
dev.igc.0.iflib.txq1.mbuf_defrag_failed: 0
dev.igc.0.iflib.txq1.m_pullups: 0
dev.igc.0.iflib.txq1.mbuf_defrag: 0
dev.igc.0.iflib.txq1.cpu: 1
dev.igc.0.iflib.txq0.r_abdications: 0
dev.igc.0.iflib.txq0.r_restarts: 0
dev.igc.0.iflib.txq0.r_stalls: 0
dev.igc.0.iflib.txq0.r_starts: 3185
dev.igc.0.iflib.txq0.r_drops: 0
dev.igc.0.iflib.txq0.r_enqueues: 3185
dev.igc.0.iflib.txq0.ring_state: pidx_head: 1137 pidx_tail: 1137 cidx: 1137 state: IDLE
dev.igc.0.iflib.txq0.txq_cleaned: 6322
dev.igc.0.iflib.txq0.txq_processed: 6362
dev.igc.0.iflib.txq0.txq_in_use: 44
dev.igc.0.iflib.txq0.txq_cidx_processed: 218
dev.igc.0.iflib.txq0.txq_cidx: 178
dev.igc.0.iflib.txq0.txq_pidx: 222
dev.igc.0.iflib.txq0.no_tx_dma_setup: 0
dev.igc.0.iflib.txq0.txd_encap_efbig: 0
dev.igc.0.iflib.txq0.tx_map_failed: 0
dev.igc.0.iflib.txq0.no_desc_avail: 0
dev.igc.0.iflib.txq0.mbuf_defrag_failed: 0
dev.igc.0.iflib.txq0.m_pullups: 7
dev.igc.0.iflib.txq0.mbuf_defrag: 0
dev.igc.0.iflib.txq0.cpu: 0
dev.igc.0.iflib.override_nrxds: 0
dev.igc.0.iflib.override_ntxds: 0
dev.igc.0.iflib.use_logical_cores: 0
dev.igc.0.iflib.separate_txrx: 0
dev.igc.0.iflib.core_offset: 0
dev.igc.0.iflib.tx_abdicate: 0
dev.igc.0.iflib.rx_budget: 0
dev.igc.0.iflib.disable_msix: 0
dev.igc.0.iflib.override_qs_enable: 0
dev.igc.0.iflib.override_nrxqs: 0
dev.igc.0.iflib.override_ntxqs: 0
dev.igc.0.iflib.driver_version: 1
dev.igc.0.%parent: pci4
dev.igc.0.%pnpinfo: vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000 class=0x020000
dev.igc.0.%location: slot=0 function=0 dbsf=pci0:4:0:0 handle=\_SB_.PCI0.PEX3.PXSX
dev.igc.0.%driver: igc
dev.igc.0.%desc: Intel(R) Ethernet Controller I225-V
                                      rikinn55R 1 Reply Last reply Reply Quote 0
                                      • rikinn55R
                                        rikinn55 @stephenw10
                                        last edited by

                                        @stephenw10
                                        Here are the sysctls:
                                        sysctl hw.igc.txt

                                        Now I did a test by changing PfS to use igc.2 in place of igc.1 attached are resulting files.
                                        After reboot.txt
                                        After unplug igc.2 and plug back in.txt

                                        As you will notice I experienced the same issue.

                                        Thanks for all your help.

                                        1 Reply Last reply Reply Quote 0
                                        • stephenw10S
                                          stephenw10 Netgate Administrator
                                          last edited by

                                          Hmm, looks near identical to any igc port I have here.

                                          It does that when connected to anything? A host directly for example?

                                          rikinn55R 2 Replies Last reply Reply Quote 0
                                          • rikinn55R
                                            rikinn55 @stephenw10
                                            last edited by

                                            @stephenw10
                                            I will check that in the morning.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.