WAN choking with bridged cable modem

william.gr

Hello!

I have been facing a very odd behavior with a cable modem on bridge mode connected to pfSense which I have no clue how to debug.

First let me give you some background. I have had ADSL WAN link for quite some time with the modem in bridge mode and it has always worked flawlessly.
I recently acquired an additional Cable WAN link (DOCSIS 3.x) and configured it to work in bridge mode and it just doesn't seem to work right and I cant easily explain how but it goes like this: as soon as DHCP is acquired and there is no network activity I can ping IPs (e.g. 8.8.8.8) and resolve names just fine, but it seems as soon as network activity goes higher (e.g. many internet tabs open or I open up a torrent client) it seems to start to "choke". Suddenly simple ping wont work for some time (first X ICMP requests wont get replies) or TCP connections will take a long time to establish although previously running ping or ongoing connection will keep working.

For example: have torrent client open and then ping 8.8.8.8

root@OPNsense:~ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=7 ttl=43 time=62.375 ms
64 bytes from 8.8.8.8: icmp_seq=8 ttl=43 time=58.086 ms
64 bytes from 8.8.8.8: icmp_seq=9 ttl=43 time=58.837 ms
64 bytes from 8.8.8.8: icmp_seq=10 ttl=43 time=61.004 ms

As you can see it lost first 6 packets and from then on an icmp request is never lost again.

Additional comments

It does seem related to pfSense/FreeBSD -- I have tried using Linux and Windows also in bridge mode and it works just fine, I dont see the described behavior
Configured dual WAN with my ADSL and once I failover to ADSL everything starts to work normally
Also configured with Cable alone (no ADSL), problem persists
Ran tcpdump on it while presenting the behavior and the only odd thing I noticed is that I get a very high TCP SYN retransmission rate
netstat -in does not show any errors or dropped packets
I am using intel driver (igb) and also tried external USB ethernet adapter

I am clueless to what the problem could be. Any clues would be very much appreciated!
Please let me if there is any additional information I can provide.
Thank you!

stephenw10

Sounds very similar to a known issue with some cable modems. See: http://badmodems.com/
Though I would expect that to affect any OS.

How much traffic are you putting across it when that happens? If it's close to the line rate some traffic shaping might help.

Steve

nkaminski

When you ran tcpdump, you mentioned seeing many TCP SYN retransmissions. Did you capture on the WAN interface facing the modem or the LAN interface facing your machine? If you captured on the WAN interface, do you see SYN,ACK packets back?

If no, that would support the hypothesis of a modem related issue.

chpalmer

root@OPNsense:~ # ping 8.8.8.8

And since it appears your using another router product.. you might want to ask them for help. It might be something in their software.

brians

I had similar problem with Shaw and Arris XB6. I would lose all WAN traffic periodically however I could still ping things. I could put exact same pfSense on my ADSL and would work fine. I tried many different configurations of pfSense hardware with no luck.

I got a Technicolor XB6 and Shaw technician came and installed a MoCA filter on my line. Problem is now solved.
Not sure which one fixed the problem though because I got both done at same time. He said MoCA traffic from neighbours can interfere with these modems and make them confused.

Gertjan

Use http://www.dslreports.com/speedtest to test each connection.

william.gr

@stephenw10 Doesnt seem to be, its Arris TG1692A. I have also tried with a EMTA TC7337 before the Arris and had the same problem.
Not much traffic it seems more related to the number of connections than throughput.

william.gr

@nkaminski It was in the WAN interface and didnt see SYN,ACK back. I also thought it was the modem and I have made the ISP replace it but the problem remains. Thing is it works in Linux (ipfire) which is super odd. Perhaps Linux has a workaround implemented for it somehow?

william.gr

@brians I initially thought it could be modem related but what puzzles me is that when connected directly to Linux (laptop or a Linux/ipfire in the same hardware) or Windows (in a laptop, still bridge mode) it works perfectly fine.

Wouldnt that rule out a modem problem?
It seems that FreeBSD has some sort of incompatibility with that modem or perhaps some traffic in the WAN interface confuses FreeBSD PF but not Linux iptables?

brians

@william-gr my problem also was pfsense only and I could use other router and it would not have issue and work fine. This is similar to you using Linux or computer direct. I think still partly pfsense quirk and it is more sensitive to whatever the underlying issue is, but all i know is after moca filter and changing to the technicolor version it works fine now.

I spent a long time trying to solve this before changing modems and moca filter, but with no success.

william.gr

@brians That actually makes quite a bit of sense. I will try to install one and let you all know how it goes. Thank you!