FreeBSD: Can I extract band width usage data from any log file?



  • If I want to detect in my shell script if gateways are on, I can do things like:

    ping -S 192.168.1.254 8.8.8.8
    result=$?
    

    But that takes CPU time and has a delay. So I would prefer:

    luis@Balanceador:~$ sudo cat /var/log/gateways.log | grep "Jul 12" | grep "loss"
    Jul 12 00:48:55 Balanceador dpinger: ADSL2_DHCP 80.58.61.254: Alarm latency 55796us stddev 6498us loss 21%
    Jul 12 00:51:20 Balanceador dpinger: ADSL2_DHCP 80.58.61.254: Clear latency 54209us stddev 433us loss 5%
    Jul 12 10:13:00 Balanceador dpinger: send_interval 500ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  dest_addr 80.58.61.250  bind_addr 192.168.1.254  identifier "ADSL1_DHCP "
    Jul 12 10:13:02 Balanceador dpinger: send_interval 500ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  dest_addr 80.58.61.254  bind_addr 192.168.2.254  identifier "ADSL2_DHCP "
    Jul 12 10:13:02 Balanceador dpinger: send_interval 500ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  dest_addr 80.58.61.250  bind_addr 192.168.1.254  identifier "ADSL1_DHCP "
    Jul 12 10:13:03 Balanceador dpinger: send_interval 500ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  dest_addr 80.58.61.250  bind_addr 192.168.1.254  identifier "ADSL1_DHCP "
    Jul 12 10:13:03 Balanceador dpinger: send_interval 500ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  dest_addr 80.58.61.254  bind_addr 192.168.2.254  identifier "ADSL2_DHCP "
    Jul 12 10:13:06 Balanceador dpinger: ADSL2_DHCP 80.58.61.254: Alarm latency 0us stddev 0us loss 100%
    Jul 12 10:14:37 Balanceador dpinger: ADSL2_DHCP 80.58.61.254: Clear latency 54099us stddev 439us loss 5%
    Jul 12 10:27:05 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Alarm latency 58335us stddev 3563us loss 21%
    Jul 12 10:28:48 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Alarm latency 55201029us stddev 0us loss 99%
    Jul 12 10:29:03 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Alarm latency 0us stddev 0us loss 100%
    Jul 12 10:29:17 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Alarm latency 22806236us stddev 0us loss 99%
    Jul 12 10:30:03 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Alarm latency 0us stddev 0us loss 100%
    Jul 12 10:30:06 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Alarm latency 12791487us stddev 0us loss 99%
    Jul 12 10:31:05 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Alarm latency 0us stddev 0us loss 100%
    Jul 12 10:31:31 Balanceador dpinger: send_interval 500ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  dest_addr 80.58.61.250  bind_addr 192.168.1.254  identifier "ADSL1_DHCP "
    Jul 12 10:31:31 Balanceador dpinger: send_interval 500ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  dest_addr 80.58.61.254  bind_addr 192.168.2.254  identifier "ADSL2_DHCP "
    Jul 12 10:31:33 Balanceador dpinger: ADSL2_DHCP 80.58.61.254: Alarm latency 54141us stddev 250us loss 25%
    Jul 12 10:31:34 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Alarm latency 0us stddev 0us loss 100%
    Jul 12 10:31:44 Balanceador dpinger: ADSL2_DHCP 80.58.61.254: Clear latency 54977us stddev 2451us loss 0%
    Jul 12 10:32:57 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Clear latency 58258us stddev 2377us loss 5%
    Jul 12 14:34:10 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Alarm latency 57749us stddev 1533us loss 21%
    Jul 12 14:34:29 Balanceador dpinger: send_interval 500ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  dest_addr 80.58.61.250  bind_addr 192.168.1.254  identifier "ADSL1_DHCP "
    Jul 12 14:34:29 Balanceador dpinger: send_interval 500ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  dest_addr 80.58.61.254  bind_addr 192.168.2.254  identifier "ADSL2_DHCP "
    Jul 12 14:34:32 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Alarm latency 0us stddev 0us loss 100%
    Jul 12 14:35:52 Balanceador dpinger: ADSL1_DHCP 80.58.61.250: Clear latency 58002us stddev 1969us loss 5%
    

    This method is supposed to detect (with low CPU ussage) any gateway failures ("loss").

    Is there any equivalent method for my band widht storing data using BWM-NG?

    Example: This is my CSV mode obtaining data method for WAN interfaces hme2 and hme3 (the CSV output is not very clear, but the info is there :-) ):

    luis@Balanceador:~$ bwm-ng -t 5000 -o csv -c 1 -I hme2,hme3
    1468327933;hme2;578.69;850.40;1429.08;4269;2905;5.78;6.77;12.55;34;29;0.00;0.00;0;0
    1468327933;hme3;276.69;155.78;432.47;782;1389;3.39;2.59;5.98;13;17;0.00;0.00;0;0
    1468327933;total;855.38;1006.18;1861.55;5051;4294;9.16;9.36;18.53;47;46;0.00;0.00;0;0

    But 5 seconds running a program takes some CPU time (e.g: if you wanna run it each minute).
    Isn't there an equivalent just grepping some text/log file?

    Thanks a lot.

    Note: I am running Traffic Graphs widget from DashBoard (i don't know if it bother). I can run/install any other program if needed, but I would prefer not CPU hogging ones.

    Note-2: Of course, I have red the main tutorial on bandwith detection: https://doc.pfsense.org/index.php/How_can_I_monitor_bandwidth_usage , but I have not found the solution there. Maybe I am missing something?

    Note-3: Methods requiring root accepted. Methods not requiring root preferred. Low CPU consumption methods, by the way, are preferred.