Iftop not capturing vlan traffic, promiscuous mode enabled

GPz1100

Wan traffic with att ISP is a bit bizarre when using bypass (external. ont, gpon, certs/wpa eapol).

Specifically, outbound traffic has no vlan tags, all inbound traffic is vlan0 tagged.

Iftop (along with tcpdump) seems to not handle this. Iftop (even with -p) only shows outbound traffic stats, nothing for inbound. I have the exact same issue with tcpdump when specifying any port or traffic type. I can get both in/out traffic if I use grep to filter for specific traffic instead.

Suggestions?

stephenw10

Hmm, you have an example? tcpdump shows no inbound traffic if you add any filter options?

What NICs are you using?

Steve

GPz1100

@stephenw10 This is a direct byproduct of asymmetrical vlan used by att as indicated above. Things work for typical traffic passing through the firewall to natted interfaces behind it, but not so for tools running on the firewall.

Here's a small snippet of what it looks like;

21:49:36.040861 wan:mac:addr > 18:c3:00:d0:9d:6e, ethertype IPv4 (0x0800), length 237: wan:public:ip:addr.31068 > 149.154.174.200.443: Flags [P.], seq 631951570:631951753, ack 4274239723, win 1025, length 183
21:49:36.079254 18:c3:00:d0:9d:6e > wan:mac:addr, ethertype 802.1Q (0x8100), length 171: vlan 0, p 0, ethertype IPv4 (0x0800), 149.154.174.200.443 > wan:public:ip:addr.31068: Flags [P.], seq 1:114, ack 183, win 1517, length 113
21:49:36.121355 wan:mac:addr > 18:c3:00:d0:9d:6e, ethertype IPv4 (0x0800), length 54: wan:public:ip:addr.31068 > 149.154.174.200.443: Flags [.], ack 114, win 1024, length 0
21:49:36.347988 wan:mac:addr > 18:c3:00:d0:9d:6e, ethertype IPv4 (0x0800), length 71: wan:public:ip:addr.13681 > 172.217.1.106.443: UDP, length 29
21:49:36.353100 18:c3:00:d0:9d:6e > wan:mac:addr, ethertype 802.1Q (0x8100), length 72: vlan 0, p 0, ethertype IPv4 (0x0800), 172.217.1.106.443 > wan:public:ip:addr.13681: UDP, length 26
21:49:36.887820 18:c3:00:d0:9d:6e > wan:mac:addr, ethertype 802.1Q (0x8100), length 60: vlan 0, p 0, ethertype IPv4 (0x0800), 79.124.62.130.49432 > wan:public:ip:addr.51601: Flags [S], seq 955971184, win 1024, length 0

Wan nic is using igb0 - intel i340-t4 card.

The way to get tcpdump to work is by using a specific filter such as this to monitor for both sides of dhcp traffic, or don't include any filters at all. Otherwise it just shows the outbound traffic.

tcpdump -vnes0 -i igb0 '(port 67 or port 68) or (vlan and (port 67 or port 68))'

With respect to iftop, I think it's not picking up the inbound traffic stats.

iftop -i igb0

Note every even line (inbound traffic) is 0.

stephenw10

Ah, OK. Not much to be done there then. Iftop sees it as 2 different interfaces.

GPz1100

Right. I see two workarounds.

Use netgraph ; this would achieve the result of stripping vlan0 from any traffic but also adds overhead. The whole goal over the last few years was to get away from netgraph (Thanks @cmcdonald ).
Use a switch between the ont and wan ports that strips vlan 0 tags. There's several out there that are known to work (https://github.com/owenthewizard/opnatt). This is the route I went.

This solution works well because the ONT is external. This won't work for those on gpon with sfp direct to firewall. Would need a similar arrangement for something in between the sfp and firewall to handle the vlan 0 stripping.