WAN traffic graph not showing outbound traffic



  • My pfSense (2.4.4) was working perfectly on my previous internet connection. I upgraded to Fibre and since then my WAN graph never shows outbound traffic, only inbound. This is impossible clearly. Here is me copying a large file from my LAN to a host on the internet:

    WAN graph not reflecting outbound traffic

    The WAN in traffic must be the ACK packets from the copy. The LAN interface correctly reflects the outbound traffic (as inbound on the LAN interface).

    I cleared my browser's cache. It made no change. I rebooted the firewall - no change. The fibre modem is in bridge mode.

    Even NTOP does not show the outbound traffic:

    NTOP

    Monitoring shows the correct values though but it is not real time:

    Monitoring

    What is the problem?

    Even traffic totals show no outbound WAN traffic:

    Traffic Totals

    UPDATE: tcpdump on the WAN interface only ever shows traffic from external hosts to the WAN IP - never anything that originates from the WAN adapter destined for external IP addresses. So this is much deeper than just a graph not working.

    I am running Suricata IPS - and had to disable some hardware offloading. Not sure if this would affect anything. Traffic flow works fine - i.e. I have full internet access.

    UPDATE2: When I disable Suricate IPS (Inline) mode the graphs all work fine. Is there any way to get Suricata in IPS mode (non legacy) and still get to see outbound WAN traffic?


  • Netgate Administrator

    Hmm, well that's odd!

    I assume you're running Suricata on the WAN? What happens if you run it on LAN instread? That's usually preferred anyway as it gives you internal IP info.

    Steve



  • @stephenw10 Yes I am running Suricata in IPS on WAN, but as IDS on LAN. I always assumed it is better to stop the bad packets when they enter the WAN as opposed to when they have already had a chance to affect the firewall if you check it only on the LAN side.


  • LAYER 8 Global Moderator

    @pwnell said in WAN traffic graph not showing outbound traffic:

    When I disable Suricate IPS (Inline) mode the graphs all work fine

    I'm going to move this to the IPS section, you prob get better traction there with people that have more exp with Suricata


  • Netgate Administrator

    It might be marginally better, especially in in-line mode, but the performance hit by running it twice on all traffic is far more significant IMO. You don't see traffic hitting the firewall itself of course so if you have services there open to the internet you might want it on WAN. Otherwise you are just scanning all the random traffic that is dropped by default anyway.

    Steve



  • @stephenw10 I have some port forwards - would that be protected equally well on the LAN side?


  • Netgate Administrator

    Yes Suricata would see that traffic on the LAN interface and it will see the actual public source IP and the internal destination IP. It would not see anything forwarded to localhost for example.

    Steve



  • This is strange. I changed WAN to IDS and LAN to IPS, now all graphs show inbound and outbound traffic. Why was only WAN affected? It is a different NIC - WAN is em and LAN is ix - does that make a difference?


  • Netgate Administrator

    It could do. The way Suricata uses netmap with different drivers could well tie in differently. That's outside my experience to be honest. @bmeeks may have more insight there.

    Steve



  • Netmap is a NIC driver dependent thing. Suricata itself does not dictate how netmap works (or does not work). Netmap is a kernel device, and all Suricata does when running with Inline IPS Mode is just open the netmap device with an API call and start routing traffic to and from it using the same API calls. The FreeBSD kernel, the netmap device and lastly the NIC driver itself determine the specific behavior. My guess (and it is just a pure guess) is that the em NIC driver is not updating some kernel parameters during netmap operation while the ix NIC driver still updates those parameters when in netmap mode. I'm further guessing these kernel parameters would be those needed by the traffic graph application.

    Inline IPS Mode in Suricata (and Snort on pfSense-2.5) seems to get blamed for a lot of stuff that is not really the fault of Suricata. The issues are with the netmap device itself, and that is a FreeBSD kernel responsibility along with the various vendors of network interface cards (NICs) and their associated NIC drivers.



  • @bmeeks Thanks for the insight.


Log in to reply