Weired spikes in upload on pfSense v1.2.3 - How to trace it?



  • Hi everyone,

    I have a pfSense v1.2.3 installed on an Alix board that has been running very well for few months. All of a sudden now I see spikes in upload when I am checking Status > Traffic Graph.

    The only PC connected to this router is a CentOS and I run iftop on that and it only shows internal LAN traffic and no connections to the internet.

    All other devices are SIP endpoints so they don't generate that much upload either.

    Below is a picture of the traffic graph for LAN and WAN which shows peak uploads of up to 600kbps for WAN. This is disastrous to the VoIP system because now there is no way to make calls as there is only 600kbps available for upload anyhow.

    Seems to me the source of all this is pfSense itself but what is it uploading? I am not sure. How can I confirm this? ***This is happening on another pfSense router of mine as well (started last month). So, something weird is definitely going on.

    Any pointers to finding and tracing the upload source will be greatly appreciated. It will also restore my faith in pfSense since I am thinking of it as a ghost box now  :D

    Thanks,



  • Anyone on this please?

    In the meanwhile I have done a "netstat -an" and I see the following which is worrying:

    [root@pfsense.local]/root(12): netstat -an
    Active Internet connections (including servers)
    Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)
    tcp4       0      0 192.168.1.1.20443      192.168.200.6.62444    ESTABLISHED
    tcp4       0  65484 192.168.1.1.20443      192.168.200.6.62442    ESTABLISHED
    tcp4       0      0 192.168.1.1.20443      192.168.200.6.62440    ESTABLISHED
    tcp4       0      0 192.168.1.1.20443      192.168.200.6.62434    TIME_WAIT
    tcp4       0      0 192.168.1.1.20443      192.168.200.6.62432    TIME_WAIT
    tcp4       0     52 192.168.1.1.20099      192.168.200.6.62002    ESTABLISHED
    tcp6       0      0 *.53                   .                    LISTEN
    tcp4       0      0 *.53                   .                    LISTEN
    tcp4       0      0 *.20443                .                    LISTEN
    tcp4       0      0 127.0.0.1.8021         .                    LISTEN
    tcp4       0      0 *.20099                .                    LISTEN
    tcp6       0      0 *.20099                .                    LISTEN
    udp4       0      0 *.67                   .                    
    udp4       0      0 **69.69.69.69.3853    142.165.36.190.123  **  
    udp4       0      0 *.1194                 .                    
    udp6       0      0 *.53                   .                    
    udp4       0      0 *.53                   .                    
    udp4       0      0 **69.69.69.69.7424    207.61.229.70.123  **    
    udp4       0      0 **69.69.69.69.38950   142.46.203.3.123  **    
    icm4       0      0 .                    .                    
    Active UNIX domain sockets
    Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
    c26ae000 stream      0      0        0        0        0        0 /tmp/php-fastcgi.socket-2
    c26ae0a8 stream      0      0 c28708a0        0        0        0 /tmp/php-fastcgi.socket-2
    c26ae1f8 stream      0      0 c2874000        0        0        0 /tmp/php-fastcgi.socket-1
    c26ae348 stream      0      0 c286f450        0        0        0 /tmp/php-fastcgi.socket-0
    c26aed20 stream      0      0        0 c26aec78        0        0
    c26aec78 stream      0      0        0 c26aed20        0        0
    c26af000 stream      0      0 c26a8e04        0        0        0 /var/run/devd.pipe
    c26af888 dgram       0      0        0 c26ae7e0        0 c26ae5e8
    c26ae5e8 dgram       0      0        0 c26ae7e0        0 c2d8c0a8
    c2d8c0a8 dgram       0      0        0 c26ae7e0        0 c26aeb28
    c26aeb28 dgram       0      0        0 c26ae7e0        0 c26ae498
    c26ae150 dgram       0      0        0 c26ae738        0        0
    c26ae498 dgram       0      0        0 c26ae7e0        0 c26aedc8
    c26aedc8 dgram       0      0        0 c26ae7e0        0 c26ae930
    c26ae930 dgram       0      0        0 c26ae7e0        0 c26ae888
    c26ae888 dgram       0      0        0 c26ae7e0        0        0
    c26ae7e0 dgram       0      0 c280578c        0 c26af888        0 /var/run/logpriv
    c26ae738 dgram       0      0 c28058a0        0 c26ae150        0 /var/run/log

    Can an expert in security please check above high-lighted IPs and let me know what the heck is going on with this pfSense? Am I opening a connection to IPs: 142.46.203.3, 207.61.229.70, and 142.165.36.190 or are they trying to DDoS me? (All those IPs are clearly hacked bots) - I am assuming they might be generating all these spikes in my bandwidth.

    I have no clue what to do with this. I mean there is DDoS module or anything.

    P.S. 69.69.69.69 is the PPPoE interface IP from ISP.

    Thanks



  • @torontob:

    In the meanwhile I have done a "netstat -an" and I see the following which is worrying:

    [root@pfsense.local]/root(12): netstat -an
    Active Internet connections (including servers)
    Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)

    udp4      0      0 69.69.69.69.3853    142.165.36.190.123  
             
    udp4      0      0 69.69.69.69.7424    207.61.229.70.123    
    udp4      0      0 69.69.69.69.38950  142.46.203.3.123

    Can an expert in security please check above high-lighted IPs and let me know what the heck is going on with this pfSense? Am I opening a connection to IPs: 142.46.203.3, 207.61.229.70, and 142.165.36.190 or are they trying to DDoS me?

    I'm no security expert but those entries are connections to NTP servers (port 123 is for NTP, Network Time Protocol).



  • Thanks for the input. But this is what the IP source is and I doubt it's NTP:

    IP Address 142.46.203.3
    Host potato.happydeys.ca
    Location CA, Canada
    City Ottawa, ON -
    Organization Chum Radio
    ISP Ontario Hydro - Telecom

    Can someone please explain if that is an inbound or outbound connection? Also, where can I see all the incoming failed attempts onto the box? option 10 at console?

    Regards



  • That's normal NTP traffic to pool.ntp.org hosts, which are all over the place. Your outbound spikes aren't the NTP though, get a packet capture and use Wireshark's analysis to see what that is.


Locked