What causes Wan "Error in"?

[stephenw10]

@logman said in What causes Wan "Error in"?:

dev.ix.0.mac_stats.rx_missed_packets: 6468

So that and 1 crc error. The old driver version would only have shown the crc error in the Rx errors count.

If I'm reading data correctly all of those errors were produced by a single test? And the test competed correctly?

rx missed packets like that is often because the local buffer is full and it cannot recive more data. At Gigabit speeds I wouldn't normally expect to see that on the WAN.
You might have a flow-control mismatch with the switch though.
Currently it shows: dev.ix.0.fc: 3
Which means flow control is active in and out:

[22.01-RELEASE][admin@6100.stevew.lan]/root: sysctl -d dev.ix.3.fc
dev.ix.3.fc: 
Set flow control mode using these values:
	0 - off
	1 - rx pause
	2 - tx pause
	3 - tx and rx pause

Is the switch port set that way?

Steve

[LoGman]

@stephenw10

Yeah, I did each test one by one.

I did Check Flow Control is Off on the switch

[stephenw10]

Then you should disable it in pfSense for that interface. Or enable it in the switch.
https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#flow-control

Steve

[LoGman]

@stephenw10

I disabled it to see if the problem is fixed.

[MoonKnight]

@logman
Or maybe try it without your switch. Connect your computer directly to your pfSense LAN ports and run the tests again

[LoGman]

@moonknight
Already Done that.

---

@stephenw10

~5h

Only CRC errors

[WAN] sysctl dev.ix.0 | grep "miss\|errs"
dev.ix.0.mac_stats.checksum_errs: 12
dev.ix.0.mac_stats.rx_missed_packets: 6468
dev.ix.0.mac_stats.rec_len_errs: 0
dev.ix.0.mac_stats.byte_errs: 0
dev.ix.0.mac_stats.ill_errs: 0
dev.ix.0.mac_stats.crc_errs: 0
dev.ix.0.mac_stats.rx_errs: 6480

[stephenw10]

Ah, so good after the flow-control change?

[LoGman]

@stephenw10

Yes, when flow-control is disabled no anymore "missed packets" only "checksum errors".

[WAN] sysctl dev.ix.0 | grep "miss\|errs"
dev.ix.0.mac_stats.checksum_errs: 279
dev.ix.0.mac_stats.rx_missed_packets: 6468
dev.ix.0.mac_stats.rec_len_errs: 0
dev.ix.0.mac_stats.byte_errs: 0
dev.ix.0.mac_stats.ill_errs: 0
dev.ix.0.mac_stats.crc_errs: 0
dev.ix.0.mac_stats.rx_errs: 6747

[stephenw10]

Ok, so to be clear when it was logging missed packets flow control was on in pfSense but off in the switch?

That would make sense if pfSense was sending the 'stop sending' packets and the switch ignored it and kept sending.

Steve

[LoGman]

@stephenw10
Yeah, it was on as default.

[LoGman]

Why is Flow-control not persistent when set in "System Tunables"

Reboot and settings again wrong.

Is this the only way?

So the correct way is "file: /boot/loader.conf.local hw.ix.flow_control="0" ?

And this Is not the correct way? 
/boot/loader.conf.local:
dev.ix.0.fc="0"
dev.ix.1.fc="0"
dev.ix.2.fc="0"
dev.ix.3.fc="0"

[stephenw10]

You can set: hw.ix.flow_control=0 as a loader variable and it will apply the default to all ix NICs.