Nginx is logging differently to specified syslog-servers than rest of the logs

21hertz · Jun 1, 2016, 11:16 AM

We got 3 different rsyslog-servers, 1 running on OpenBSD and 2 on Linux. pfSense is sending all its logs to these three servers ("Everything").

We are getting more logging since updating to 2.3.1 and nginx is the source of this (about 500 MByte per day extra). Nginx does not log to our syslog-servers in the same way as the rest of the logs (Everything, System/Firewall etc). We are not using any extra packages except NRPE and VMware-tools.

When logging to external rsyslog-servers Nginx creates a new hostname source, in our case adding our domain.tld after hostname (which becomes destination directory/filename in our rsyslog).

You can see what I mean here, a directory listing one of our syslog-servers:


drwxr-xr-x    2 loguser      staff   24064 Jun  1 00:00 my-pfsense                      <--- all logs from pfsense except nginx logs.
drwx------    2 loguser      staff     512 Jun  1 00:00 my-pfsense.mydomain.tld          <--- nginx logs appear in here, nginx logs added "mydomain.ltd".
drwxr-xr-x    2 loguser      staff   31232 Jun  1 00:00 my-pfsense-02                   <--- all logs from pfsense except nginx logs.
drwx------    2 loguser      staff     512 May 29 22:55 my-pfsense-02.mydomain.tld      <--- nginx logs appear in here, nginx logs added "mydomain.ltd".

Here is an example of what the nginx-log file contains:


# tail 2016-06-01_my-pfsense.mydomain.tld.log
2016-06-01T12:37:31+02:00 my-pfsense.mydomain.tld nginx: 10.x.3.77 - [ANONYMIZED@somedomain.tld] [01/Jun/2016:12:37:31 +0200] "POST /Microsoft-Server-ActiveSync?User=[ANONYMIZED@somedomain.tld]&DeviceId=SIVSUP0CTD1D35QNSM4EF9J64C&DeviceType=iPhone&Cmd=Sync HTTP/1.1" 302 5 "-" "Apple-iPhone5C4/1306.69"
2016-06-01T12:37:31+02:00 my-pfsense.mydomain.tld nginx: 10.x.0.220 - - [01/Jun/2016:12:37:31 +0200] "GET /index.php?zone=cpwifise&redirurl=http%3A%2F%2Fofficecdn.microsoft.com%2Fsg%2F39168D7E-077B-48E7-872C-B232C3E72675%2FOffice%2FData%2Fv32.cab HTTP/1.1" 200 91 "-" "OfficeC2R"
2016-06-01T12:37:31+02:00 my-pfsense.mydomain.tld nginx: 10.x.3.77 - windowsdomain\ANONYMIZED [01/Jun/2016:12:37:31 +0200] "POST /index.php?zone=cpzone&redirurl=http%3A%2F%2Fmail.mydomain.tld%2FMicrosoft-Server-ActiveSync%3FUser%3Dtmd HTTP/1.1" 200 1706 "-" "Apple-iPhone5C4/1306.69"
2016-06-01T12:37:31+02:00 my-pfsense.mydomain.tld nginx: 10.x.3.77 - [ANONYMIZED@somedomain.tld] [01/Jun/2016:12:37:31 +0200] "POST /index.php?zone=cpzone&redirurl=http%3A%2F%2Foutlook.office365.com%2FMicrosoft-Server-ActiveSync%3FUser%3DANONYMIZED%40anotherdomain.tld HTTP/1.1" 200 1732 "-" "Apple-iPhone5C4/1306.69"
2016-06-01T12:37:31+02:00 my-pfsense.mydomain.tld nginx: 10.x.0.220 - - [01/Jun/2016:12:37:31 +0200] "GET /sg/39168D7E-077B-48E7-872C-B232C3E72675/Office/Data/v32.cab HTTP/1.1" 302 5 "-" "OfficeC2R"
2016-06-01T12:37:31+02:00 my-pfsense.mydomain.tld nginx: 10.x.0.220 - - [01/Jun/2016:12:37:31 +0200] "GET /index.php?zone=cpwifise&redirurl=http%3A%2F%2Fofficecdn.microsoft.com%2Fsg%2F39168D7E-077B-48E7-872C-B232C3E72675%2FOffice%2FData%2Fv32.cab HTTP/1.1" 200 91 "-" "OfficeC2R"
2016-06-01T12:37:31+02:00 my-pfsense.mydomain.tld nginx: 10.x.3.77 - windowsdomain\ANONYMIZED [01/Jun/2016:12:37:31 +0200] "POST /Microsoft-Server-ActiveSync?User=tmd&DeviceId=SIVSUP0CTD1D35QNSM4EF9J64C&DeviceType=iPhone&Cmd=Sync HTTP/1.1" 302 5 "-" "Apple-iPhone5C4/1306.69"
2016-06-01T12:37:31+02:00 my-pfsense.mydomain.tld nginx: 10.x.3.77 - [ANONYMIZED@somedomain.tld] [01/Jun/2016:12:37:31 +0200] "POST /Microsoft-Server-ActiveSync?User=[ANONYMIZED@somedomain.tld]&DeviceId=SIVSUP0CTD1D35QNSM4EF9J64C&DeviceType=iPhone&Cmd=Sync HTTP/1.1" 302 5 "-" "Apple-iPhone5C4/1306.69"
2016-06-01T12:37:31+02:00 my-pfsense.mydomain.tld nginx: 10.x.3.77 - windowsdomain\ANONYMIZED [01/Jun/2016:12:37:31 +0200] "POST /index.php?zone=cpzone&redirurl=http%3A%2F%2Fmail.mydomain.tld%2FMicrosoft-Server-ActiveSync%3FUser%3Dtmd HTTP/1.1" 200 1706 "-" "Apple-iPhone5C4/1306.69"
2016-06-01T12:37:31+02:00 my-pfsense.mydomain.tld nginx: 10.x.3.77 - [ANONYMIZED@somedomain.tld] [01/Jun/2016:12:37:31 +0200] "POST /index.php?zone=cpzone&redirurl=http%3A%2F%2Foutlook.office365.com%2FMicrosoft-Server-ActiveSync%3FUser%3DANONYMIZED%40anotherdomain.tld HTTP/1.1" 200 1732 "-" "Apple-iPhone5C4/1306.69"

These are my concerns:
1. Our syslog-server gets a lot of nginx logs containing upper layer information (http post etc) (may be normal to nginx, but its a new behaviour of pfSense).
2. nginx seems to log separetly from anything I configure in Settings under Logging in pfSense? (not confirmed every setting)
3. nginx creates another source hostname than the rest of the logs do -> logging destination gets affected (depending on your rsyslog configuration of course). ngninxt sets its logs' hostname source to hostname.domain.tld instead of just hostname for everything else.

It would be nice to be able to configure the nginx logging feature from GUI so that it matches what you need to be logged - and where.

Take care,
J.