Hmm, I'm not aware of that specifically. Only that the modules will generally run at their 'native speeds only because the rate cannot be set. For most dual speed modules that means 10G but some will select 1G and run fine. The actual restriction as noted by Intel is shown here: https://docs.netgate.com/pfsense/en/latest/solutions/netgate-6100/io-ports.html#sfp-ethernet-ports

In answering my own question, I discovered that the continuous # symbols was not a rebuild and so I restarted and checked the bios. I changed from Legacy/UEFI to UEFI and the system restarted without a hitch! A+ pfSense!!!

For future readers: problem solved. For more information, see this post: https://forum.netgate.com/topic/200073/verification-needed-disabling-bypasses-on-cloudgenix-ion-3000/5

OK, complete and total replication. Ran the routine on two more ION 3000 devices, verified the disablement of bypasses by bridging all eight ports on the expansion board into a single network, connecting multiple devices to those ports, and accessing one device from another, including making the remotely accessed device access the Internet. Found a small refinement: you can actually install pfSense on the device (no need to resort to installing on another device), as long as you remember to use the right settings. Anyway, here's the refined procedure: One. Turn off the device (completely, as in, disconnect the power cable) and open up the case (there are four screws holding the top in place, one on each side and two in the back; once you remove the screws, the top slides off). Two. Disconnect the SATA SSD and remove the CF card. Three. Reposition a jumper on the expansion board and do some OpenWrt magic. For details on this step, see my post on the OpenWrt forum: https://forum.openwrt.org/t/report-openwrt-on-cloudgenix-ion-3000/246014 Briefly, you prepare a bootable device with OpenWrt (I used a CF card, but there's no reason you can't use a USB stick), boot the device from it, install three additional packages, make a slight edit to /boot/grub/grub.cfg to configure the disablement of bypasses, reboot the device (which should disable the bypasses), let the boot sequence run, then shut the device down. Four. Remove OpenWrt media, reconnect the SATA SSD, connect a USB stick with pfSense installer, and install pfSense. Remember the settings: UFS file system, entire drive, MS-DOS partitions (the layout suggested by the installer should work). Once the install is complete, shut the device down, disconnect the installer media, and boot the device. It should boot from the SSD, and bypasses should be disabled, so you can assign interfaces and as usual. Remember, the port map is weird: = Expansion board = ====== Onboard controllers ====== igb6 igb7 igb8 igb9 igb10 igb11 igb12 igb13 igb0 igb1 igb2 igb3 igb4 igb5 so by default, the rightmost onboard port (igb1) will be LAN and the one to the left of it (igb0), WAN. Five. To get the little LCD screen working, install lcdproc and configure it to use parallel port (/dev/lpt0) with 2x20 screen size and the Watchguard Firebox with SDEC driver.

@stephenw10 Thanks again. I just need it to install pfsense. I will be using a HDMI computer monitor. Then the webgui will take over.

@stephenw10 Thank you...that worked. [image: 1770652615230-e81450eb-1851-4a69-9496-5861e970441e-image.png]

@stephenw10 Whoops! Should be Xeon E-2174G (4c/8t 3.8/4.7GHz)

@stephenw10 It could be a combination of a bad patch cable AND a faulty igc3 port on the 6100. After swapping cables and ports, igc0 sees the LAN and igc1 sees the ISP/WAN. However, switching the working cable from icg1 to igc3, no link is detected. @patient0 my WAN connection is pure DHCP. I'll open a ticket with Netgate. Thanks, Tim

@Waqar.UK the X550 is supported well, yes.

Nice.

@stephenw10 - This is nice. Thank you very much. We will do the upgrade tonight.

@susamlicubuk The two rightmost pairs of ports have bypasses on them. You need to disable those bypasses. The steps below assume that you have BIOS in factory configuration. If you already made changes, adjust accordingly. Turn off the FW-7551. Set console speed on your computer to 9600 bps and connect your computer to the FW-7551 using its console port. Start the console software on your computer. Turn on the FW-7551. When you see the BIOS prompt asking you to hit Esc or Del, hit Tab. (This is a common occurrence when dealing with BIOS on the console; Tab tends to work best.) Next, enter the password (usually, it's either Kilimanjaro1 or Cheyenne1). This will get you into BIOS. In BIOS, go to Advanced. Under Advanced, select Watchdog mode and set it to Disabled. Next, still under Advanced, select Generation 2 LAN Bypass Configuration. On the screen that appears next, set LAN3-4 Runtime Bypass and LAN5-6 Runtime Bypass to Disabled. As to the two matching System Off Bypass options, you can leave them enabled (so when the device is off, data will pass through it, as if it were a mere keystone) or disable them (so when the device is off, there will be no data traffic through it). Next (optionally), hit Esc to return to Advanced and select Serial Port Console Redirection. On the screen that follows, select Console Redirection Settings. On the screen that follows, select Bits per second and set it to your desired value (since pfSense operates at 115200 bps by default, it makes sense to set BIOS to the same data transfer rate). Hit Esc two or three times to return to the main menu. Then, navigate to Save & Exit and choose Save Changes and Exit. This should do it.

@joy786 said in High latency to gateway and ISP when multiple switches are connected – possible pfSense overload?: What’s strange is that everything becomes stable as soon as I disconnect any one of my access switches. It doesn’t matter which switch I unplug — the problem immediately disappears. Current behavior Ping to pfSense gateway becomes unstable and increases significantly Ping to public IP / first ISP hop also increases pfSense CPU usage goes high during the issue As soon as one switch is disconnected: CPU drops Ping stabilizes This sounds like you are introducing then removing a Layer 2 loop.

A x4 PCIe 3.0 slot will pass 32Gbps. That shouldn't be a restriction.

The module is present in the public 26.03 snapshot. Do you have the loader line present to load it at boot?

Hmm, interesting. That seems fixable.....

@sr-it Sometimes you need to update the NVM firmware. I flashed my cards, and I’ve had zero problems since. Google it — the first link should be a “*Sense” forum post; read it carefully to see exactly what needs to be done to update the firmware.

@stephenw10 actually I don't know what I need since it's not available yet. I called Google to ask when and got told that a year could come and go between entrenchment and availability. So it's neither here nor there at the moment. I got the idea from reading online that their fiber converter box things have twisted pair jacks, presumably 10 gigabit.

Yes I would expect it to link at 10G to that switch.

Yes, Windows 11 auto-detected and installed Intel driver for the card. I'll need to do a test install with CE when I can find some extra time...