@epiclper There will be perhaps three things you could try out to gain the throughput a bit more. But with 870 MBit/s plus TCP overheat you will normally reaching the range of 900 + something MBit/s and this with a older 4 core CPU!!! First point: Install the last firmware 4.19.0.1 according to this HowTo. APU Bios upgrade PC Engines APU BIOS depot Set up in the /boot/loader.conf.local the following entries; hint.p4tcc.0.disabled=1 hint.acpi_throttle.0.disabled=1 hint.acpi_perf.0.disabled=1 Now your cpu will be not running anymore between 600MHz and 1000MHz, it is able to "run" from 1000MHz to 1400MHz, you should watch out the entire CPU temperature too please! Second point: Since pfSense version 2.6 the entire WAN load will be pulled over several queues, if you are not nailed to the 1 CPU core usage using PPPoE, you will be benefitting from the 1 queue = 1 CPU core. That means in theoretic more queues = more throughput. There are three different numbers for the queues as I know it; queue amount queue length queue size Third point The mbufsize can be tuned also, not even needed but also nice to know. If you are size them up you could get a gain from, with point of view towards to the throughput. A tip from me, if you are installing a fresh pfSense 2.6 please install it and then test it out without any packages installed and configured, your rules should be in place for sure, but no packages please installed. So you will see the entire throughput and you see then also what packages are narrow down the entire speed later! I was setting up at the installation using ZFS and size up the swap partition to 4 GB, since that I am not using 60% -90% of my onboard soldered ram, I am using 39% ram and ~35% swap, so it free me a bit of ram for more headspace. A side note, all available tunings can be single solve the problem, but often it is a together working game play of them, and to find out the bets option you must perhaps do some more tests in either different configuration to get the most out for you.

update to this thread: I've moved to an Intel X520-DA2 dual port NIC and I'm getting much better performance. I had to do some tuning. But I'm now getting about 7-8Gbps to my ISP's iperf3 server which seems reasonable for 3 hops away. I get about the same routing across subnets (vLANS) through pfSense. I'm also not processor or thread limited any more. At this point, I'll consider that a 'mostly win' - seems like a massive improvement from where I was. Assuming this box stays stable, I'll purchase support from Netgate since this will be my first time not running on Netgate hardware (outside of some VMs). Thanks everyone who chimed in here.

Unlikely IMO. Hardware errors are usually more random.

Did you restart the Starlink box? Is the 2100 pulling an IP address on it's WAN?

The Realtek driver and loader values should survive across a minor upgrade like that. The fact the Intel NICs are lost certainly isn't expected. It sounds like something low level if a power cycle brings them back. When they are lost do you see any errors in the boot log when the driver tries to attach? If they are not detected at all that seems like a PCIe error somewhere. Steve

Hmm, that setting it to WAN would have reset the default route. It may have lost it's default route somehow. But that would have broken the connection for everything.

@gertjan said in Starlink and pfSense: Such a scheme would need a specially build DHCP client on the (Starlink) router, and its behavior should be simulated on pfSense. I agree. If we knew what it actually requires we should be able to do it. See: https://forum.netgate.com/topic/176450/starlink-no-internet-when-reject-leases-from-configured

I will have to reinstall everything, plus I have a second problem of overheating but after moving the box in my "lab" ... grrrrrrrrr

@danioj Bigger takeaway: Netgate Device ID is based on your NICs and their MACs. Add VLANs over interfaces to your VM guests -- you'll be happier long-term.

@johnpoz oh, thats a FANTASTIC idea, i hadnt thought of using haproxy to do this!

You should see old leases in /var/dhcpd/var/db/dhcpd.leases if they still exist anywhere. Though if clients are getting a new lease they may not. You can choose to backup the leases in Diag > Backup > Backup extra data. Steve

@jimp Thanks :-) Ideas looks great :) BTW I do want to user + certificate but in that case when I changed password I was still able to login with just certificate(case 2 above).

Mmm, more info needed! I'd guess it's a subnet conflict though. If the upstream device is a modem it might be handing out a private lease before it syncs. Steve

@stephenw10 Thank you very much. Upgrading to version 2.6 fixed the problem. It's working flawlessly now. Thank you again.

https://redmine.pfsense.org/issues/14356 Redmine is open for this issue. I recently learned that it is a bigger issue over just this small part I found.

Be aware that the igc driver only supports autonegotiation. Setting it to 1G simply omits the other link speeds as choices in the negotiation. If something is not enabled for negotiation it will fallback to a default speed or fail to link entirely.

@johnpoz and @stephenw10 That's what i thought as well so I will "master" them again ;). For me this enough information, this can be closed. Thank you for your help

What's the email server you're trying to connect to? How is it configured? What connection type does it expect? Steve

Update: Looks like this was caused by certain SNORT rules. Disabling the SNORT interface and everything works again. Will update further.