@tedquade Thanks for that, extremely helpful. I did not anticipate replacing a zfs mirror would not be a documented and supported procedure. I can't risk router being offline, so I've got a replacement unit coming today and I'll try that recovery when the current unit is offline.

@stephenw10 thanks for your answer!

MiniPCIe can carry PCI, USB and/or SATA, or some combination. It would be unusual to see an mPCIe slot that only carried USB but entirely possible. That said you would be better off with an external Ethernet connected access point if you can possibly do that. Steve

@jgallagherco Phew!

@stephenw10 Hello, Thank you for reaching me. I re-installed using memstick, and it gave me same situation. I tried 2 more times USB booting the result was same, so I let it be for 30+ minutes. Luckly it is up after 30+ minutes automatically. My config file has wiped out, but I'm happy. Thank you so much.

Thank you for explaining it Steve.

@PLBarton Do you still have this patch? I recently installed a i225-LM PCIe 4x card and the operation seems wonky and I wonder if it is related to x1 vs x4 lane support?

Getting old does indeed suck. We'd need to see more of the crash to know more there. Just the panic string is too generic to tell us much.

@stephenw10 I believe I came across that post while I was looking. Yep I think my buddy will need to see about a new modem. I would want a different provider myself. I have Spectrum and I got a modem with only an ethernet port and phone jacks so I can do what I want with my service.

@Setarcos Thanks for this! I will have to check it out.

Well, the solution I came to today is it's better to just a clean install of Proxmox, since, the pfSense VM is so important and its resilience in restoring config. Once the IOMMU is messed up, just reinstall. So, I got 64GB memory coming in for it on Tuesday to replace the 32GB, so I'll do it then...one hour max.

Done with the install media update / upgrade - pfSense Plus 23.05.1-RELEASE (arm) on pfsense *** Just lucky I had the right USB cable and a USB drive I could sacrifice.

@prabath-anuradha I mean I still think 10,000 is a huge amount, I mean even if you have a 10 gigabit WAN connection, if they are all using it concurrently then that's 1 megabit per second per connection, that's almost nothing and IMO would make the VPN not very useful. How fast is the WAN connection going to be at this site? I think 1,000 might be more reasonable, still a lot even on a 1541 I think, but more reasonable to achieve.

@stephenw10 no, some of these ports are generaly not active ... no LED

pfSense will run in 1G without an issue. If you add hungry packages you will need more. I would try it and see. You won't need more than 8G to get up and running.

@stephenw10 Agreed, but previously the QAT module itself was shipped as part of CE.

If it has 6 ports you may be using the wrong NIC. Try running ifconfig at the command line and see which NICs show as UP and linked.

Nice!

Sounds like the two interfaces are detected in a different order at boot. Check the MAC addresses still match after reboot. It is possible there is no MAC provided with a USB device and it randonly assigns one which causes even more issues. But, yes, you should try to avoid USB Ethernet if at all possible. Steve

Is that when booting the installer or first boot after installing? It looks like it's not seeing any interfaces available to assign. Do you see any detected in the boot logs? Steve