Ntopng development

garyd9

I believe you want this repo here:

https://github.com/pfsense/FreeBSD-ports

which corresponds to packages in pfSense 2.3 and later.

Look in net/pfSense-pkg-ntopng.

Okay, thank you. I'll redo my patch tomorrow morning in that other repo.

r_e_d_b_a_r_o_n

@dennypage:

@Cino:

I found bug with the ssl version. You may want to remove "\n" from the new code.

You are quite correct. Not sure how I missed that in testing.

PR created.

Thanks

Hello, I updated ntopng package to 0.8.4_2. From changelog it fixes problem with \n in startup script. BUT after upgrade still same config and startup file is generated. Edited it manually, removed newline but still not starting with error : "[HTTPserver.cpp:503] ERROR: Unable to start HTTP server (IPv4) on ports 3000s: Invalid argument". It only started after removing -W argument (HTTPS).
New bug or my problem?

Modified ntopng.sh

rc_start() {
/usr/local/bin/redis-server –dir /var/db/ntopng/ --dbfilename ntopng.rdb &
/usr/local/bin/ntopng -d /var/db/ntopng -G /var/run/ntopng.pid -s -e -w 0 -W 3000 -i 'vmx1' -i 'vmx2' --dns-mode '0' --local-networks '192.168.0.0/16,172.16.0.0/12,10.0.0.0/8' &

}

dennypage

If you are using HTTPS enabled in the pfSense gui, you will need to visit the ntopng settings page (Diagnostics / ntopng Settings) and press save. This will cause the certificate to be set up in order to use HTTPS with ntopng.

luckman212

@dennypage:

It's the same version of ntopng, with the addition of support for HTTPS in the pfSense package.

Denny, do you know what the main reason is that the 2.4 version of ntopng isn't available in pfSense? I see the FreeBSD port has been ready for some time…
https://github.com/freebsd/freebsd-ports/blob/master/net/ntopng/Makefile

kesawi

I've just upgraded to ntopng 0.8.4_2 on pfSense 2.3.2 and get the following error message in the System Log:

ntopng 		[HTTPserver.cpp:503] ERROR: Unable to start HTTP server (IPv4) on ports 3000s: Invalid argument

The ntopng service is showing as stopped in the service status and the ntopng process isn't running (ie not appearing when I run ps aux or sockstat.

I've tried uninstalling, reinstalling, and unchecking the "Keep Data/Settings" options, but my settings seem to be kept, and the error remains. I've tried changing the settings in the GUI and re-saving as suggested a couple of posts up.

I do have a certificate (ntopng-cert.pem) in /usr/local/share/ntopng/httpdocs/ssl which gets regenerated each time I update the settings through the GUI.

dennypage

Would you mind checking the sum on ntopng.inc please?

The command is this: sum /usr/local/pkg/ntopng.inc

The expected output is this: 51966 12 /usr/local/pkg/ntopng.inc

Thanks

dennypage

@luckman212:

Denny, do you know what the main reason is that the 2.4 version of ntopng isn't available in pfSense?

I don't have direct knowledge, but if I had to guess I would say that it's a fair bit of work and nobody has volunteered or been assigned for it.

luckman212

Hmm ok I didn't realize it was much work after the FreeBSD port. Ok, I wish I could help somehow. I looked at the "Developing Packages" page but tbh I wouldn't know where to start :(

r_e_d_b_a_r_o_n

@dennypage:

Would you mind checking the sum on ntopng.inc please?

The command is this: sum /usr/local/pkg/ntopng.inc

The expected output is this: 51966 12 /usr/local/pkg/ntopng.inc

Thanks

This is output of sum:

51966 12 /usr/local/pkg/ntopng.inc

As kesawi mention I also have certificate at place and it's regenerating.

dennypage

Regeneration of the ssl certificate every time you save settings or reboot is expected.

r_e_d_b_a_r_o_n, can you post the unmodified content of /usr/local/etc/rc.d/ntopng.sh please?

And can you clarify the version of pfSense you are running as well?

Thanks

r_e_d_b_a_r_o_n

@dennypage:

Regeneration of the ssl certificate every time you save settings or reboot is expected.

r_e_d_b_a_r_o_n, can you post the unmodified content of /usr/local/etc/rc.d/ntopng.sh please?

And can you clarify the version of pfSense you are running as well?

Thanks

So I uninstalled package and checked that every file belonging to ntopng is deleted and reinstalled;

Output from sum is: 59748 12 /usr/local/pkg/ntopng.inc (so previosly file was modified by me);

ntopng-cert.pem is avalable in /usr/local/share/ntopng/httpdocs/ssl;

Contents of /usr/local/etc/rc.d/ntopng.sh:
#!/bin/sh

This file was automatically generated

by the pfSense service handler.

rc_start() {
/usr/local/bin/redis-server –dir /var/db/ntopng/ --dbfilename ntopng.rdb &
/usr/local/bin/ntopng -d /var/db/ntopng -G /var/run/ntopng.pid -s -e -w 0 -W 3000
-i 'vmx1' -i 'vmx2' --dns-mode '0' --local-networks '192.168.0.0/16,172.16.0.0/12,10.0.0.0/8' &

}

rc_stop() {
/usr/bin/killall ntopng redis-cli redis-server

}

case $1 in
start)
rc_start
;;
stop)
rc_stop
;;
restart)
rc_stop
rc_start
;;
esac

pfSense vewrsion I am running is:

2.3.2-RELEASE (amd64)
built on Tue Jul 19 12:44:43 CDT 2016
FreeBSD 10.3-RELEASE-p5

kesawi

@dennypage:

Would you mind checking the sum on ntopng.inc please?

The command is this: sum /usr/local/pkg/ntopng.inc

The expected output is this: 51966 12 /usr/local/pkg/ntopng.inc

My output is different:

59748 12 /usr/local/pkg/ntopng.inc

Also my /usr/local/etc/rc.d/ntopng.sh:

#!/bin/sh
# This file was automatically generated
# by the pfSense service handler.

rc_start() {
                /usr/local/bin/redis-server --dir /var/db/ntopng/ --dbfilename ntopng.rdb &
        /usr/local/bin/ntopng -d /var/db/ntopng -G /var/run/ntopng.pid -s -e -w 0 -W 3000
 -H   -i 'lagg0_vlan2' --dns-mode '0'  --local-networks '192.168.0.0/16,172.16.0.0/12,10.0.0.0/8' &

}

rc_stop() {
        /usr/bin/killall ntopng redis-cli redis-server

}

case $1 in
        start)
                rc_start
                ;;
        stop)
                rc_stop
                ;;
esac

I notice I have -H in the rc_start.

When I manually launch ntopng.sh

 /usr/local/etc/rc.d/ntopng.sh start
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.0.7 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 13809
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

13809:M 05 Sep 06:32:26.174 # Server started, Redis version 3.0.7
13809:M 05 Sep 06:32:26.180 * DB loaded from disk: 0.006 seconds
13809:M 05 Sep 06:32:26.180 * The server is now ready to accept connections on port 6379
05/Sep/2016 06:32:26 [Prefs.cpp:915] Logging into /var/db/ntopng/ntopng.log
05/Sep/2016 06:32:26 [Ntop.cpp:1013] Setting local networks to 127.0.0.0/8
05/Sep/2016 06:32:26 [Redis.cpp:92] Successfully connected to redis 127.0.0.1:6379@0
05/Sep/2016 06:32:26 [Ntop.cpp:987] Parent process is exiting (this is normal)
/usr/local/etc/rc.d/ntopng.sh: -H: not found

I modified ntopng.sh to remove the line break before the -H and then ran it manually, and the script started without error, but still received [HTTPserver.cpp:503] ERROR in the log files.

dennypage

@r_e_d_b_a_r_o_n:

Output from sum is: 59748 12 /usr/local/pkg/ntopng.inc (so previosly file was modified by me);

Yep, that's the old version of ntopng.inc.

The problem is in the repository. There are three branches in the repository that are pertinent, devel, 2_3, and 2_3_2. As you would expect, development happens in the devel branch, and changes are then back-ported as appropriate to the other branches. The changed files (Makefile, ntopng.inc) are in devel, and were ported to 2_3 (currently seen as the 2.3.3 test distribution), but the port to 2_3_2 was incomplete. The Makefile change is there, but the ntopng.inc change is not.

There is a new PR to update the version number. I don't believe that the build server will be updated until Monday or possibly Tuesday. In the meantime, you can apply the following patch:

/usr/local/pkg: diff ntopng.inc.old ntopng.inc.new
149c149
< $http_args = "-w 0 -W 3000\n";
–-

$http_args = "-w 0 -W 3000";
152c152
< $http_args = "-w 3000\n";

$http_args = "-w 3000";

kesawi

Made the changes, saved the configuration through the GUI, still getting the error.

dennypage

Okay, let's start over following these steps:

1. Confirm the sum of ntopng.inc. It should be 51966.

2. Save the configuration on the ntopng settings page.

3. Post the contents of /usr/local/etc/rc.d/ntopng.sh as generated. Please use a code section to ensure that there are no changes in posting.

4. Post the entire output from ntopng startup (including the error message).

kesawi

@dennypage:

Okay, let's start over following these steps:

1. Confirm the sum of ntopng.inc. It should be 51966.

sum /usr/local/pkg/ntopng.inc
51966 12 /usr/local/pkg/ntopng.inc

2. Save the configuration on the ntopng settings page.

Done

3. Post the contents of /usr/local/etc/rc.d/ntopng.sh as generated. Please use a code section to ensure that there are no changes in posting.

cat /usr/local/etc/rc.d/ntopng.sh
#!/bin/sh
# This file was automatically generated
# by the pfSense service handler.

rc_start() {
                /usr/local/bin/redis-server --dir /var/db/ntopng/ --dbfilename ntopng.rdb &
        /usr/local/bin/ntopng -d /var/db/ntopng -G /var/run/ntopng.pid -s -e -w 0 -W 3000 -H   -i 'lagg0_vlan2' -i 'lagg0_vlan10' -i 'lagg0_vlan15' --dns-mode '0'  --local-networks '192.168.2.0/24,192.168.10.0/24,192.168.15.0/28' &

}

rc_stop() {
        /usr/bin/killall ntopng redis-cli redis-server

}

case $1 in
        start)
                rc_start
                ;;
        stop)
                rc_stop
                ;;
        restart)
                rc_stop
                rc_start
                ;;
esac

4. Post the entire output from ntopng startup (including the error message).

 /usr/local/etc/rc.d/ntopng.sh start
                 _._                                                                                                                                                                      
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.0.7 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 87223
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

87223:M 05 Sep 21:29:15.310 # Server started, Redis version 3.0.7
87223:M 05 Sep 21:29:15.316 * DB loaded from disk: 0.005 seconds
87223:M 05 Sep 21:29:15.316 * The server is now ready to accept connections on port 6379
05/Sep/2016 21:29:15 [Prefs.cpp:915] Logging into /var/db/ntopng/ntopng.log
05/Sep/2016 21:29:15 [Ntop.cpp:1013] Setting local networks to 192.168.2.0/24,192.168.10.0/24,192.168.15.0/28
05/Sep/2016 21:29:15 [Redis.cpp:92] Successfully connected to redis 127.0.0.1:6379@0
05/Sep/2016 21:29:15 [Ntop.cpp:987] Parent process is exiting (this is normal)

polcape

I have the same problem. It's caused by an error to build the ssl certificate /usr/local/share/ntopng/httpdocs/ssl/ntopng-cert.pem

If you edit this file you have to change line between the certificate and the private key from
–---END CERTIFICATE----------BEGIN PRIVATE KEY-----
to
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----

So you have to add the newline with enter key.

Now works fine.

kesawi

@polcape:

I have the same problem. It's caused by an error to build the ssl certificate /usr/local/share/ntopng/httpdocs/ssl/ntopng-cert.pem

Thanks, that fixed it.

dennypage

Odd that your CERT doesn't have a new line at the end. Easy enough to fix. Where did the certificate originate from? Was it generated or imported?

polcape

My CERT is imported.
Where I find it in PFSense to add newline?

Thanks