What causes Wan "Error in"?

[LoGman]

@stephenw10

Yeah, I did each test one by one.

I did Check Flow Control is Off on the switch

[stephenw10]

Then you should disable it in pfSense for that interface. Or enable it in the switch.
https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#flow-control

Steve

[LoGman]

@stephenw10

I disabled it to see if the problem is fixed.

[MoonKnight]

@logman
Or maybe try it without your switch. Connect your computer directly to your pfSense LAN ports and run the tests again

[LoGman]

@moonknight
Already Done that.

---

@stephenw10

~5h

Only CRC errors

[WAN] sysctl dev.ix.0 | grep "miss\|errs"
dev.ix.0.mac_stats.checksum_errs: 12
dev.ix.0.mac_stats.rx_missed_packets: 6468
dev.ix.0.mac_stats.rec_len_errs: 0
dev.ix.0.mac_stats.byte_errs: 0
dev.ix.0.mac_stats.ill_errs: 0
dev.ix.0.mac_stats.crc_errs: 0
dev.ix.0.mac_stats.rx_errs: 6480

[stephenw10]

Ah, so good after the flow-control change?

[LoGman]

@stephenw10

Yes, when flow-control is disabled no anymore "missed packets" only "checksum errors".

[WAN] sysctl dev.ix.0 | grep "miss\|errs"
dev.ix.0.mac_stats.checksum_errs: 279
dev.ix.0.mac_stats.rx_missed_packets: 6468
dev.ix.0.mac_stats.rec_len_errs: 0
dev.ix.0.mac_stats.byte_errs: 0
dev.ix.0.mac_stats.ill_errs: 0
dev.ix.0.mac_stats.crc_errs: 0
dev.ix.0.mac_stats.rx_errs: 6747

[stephenw10]

Ok, so to be clear when it was logging missed packets flow control was on in pfSense but off in the switch?

That would make sense if pfSense was sending the 'stop sending' packets and the switch ignored it and kept sending.

Steve

[LoGman]

@stephenw10
Yeah, it was on as default.

[LoGman]

Why is Flow-control not persistent when set in "System Tunables"

Reboot and settings again wrong.

Is this the only way?

So the correct way is "file: /boot/loader.conf.local hw.ix.flow_control="0" ?

And this Is not the correct way? 
/boot/loader.conf.local:
dev.ix.0.fc="0"
dev.ix.1.fc="0"
dev.ix.2.fc="0"
dev.ix.3.fc="0"

[stephenw10]

You can set: hw.ix.flow_control=0 as a loader variable and it will apply the default to all ix NICs.