Pfsense 2.3 NGINX questions



  • Hope someone has fixed a similar problem and is able to answer my questions…

    I upgraded from 2.2.6 to 2.3.  The upgrade was a success but it was not as smooth as I would have expected.  With a downtime of nearly 2 hours and some minor issues to fix ( I am not here to go over them). The problem I have is with NGINX. I was using nginx on 2.2.6 as a way of hosting multiple websites (virtual hosts that proxy on to internal servers) and I was hoping the native NGINX in 2.3 would be much more convenient  but I am stuck

    1.  Where do the configuration file(s) for the native nginx on 2.3 reside?  Is it /usr/local/etc/nginx ?
    2.  I have made some changes to the nginx.conf at the above location, that seems to be ignored. What am I missing?
    3.  There is a remnant rc.d script from 2.2.6  nginx that I had installed,  could here be two versions of nginx setting on the box ?  How do I check this and if remove one to just retain the 'native' version?
    4.  Is it possible to have virtualhosts configured on the 'native' nginx and what files needs to be changed for the same?

    ps: I use the term native loosely here and refer to the nginx supplied with pfsense 2.3



  • Well I'm not a dev but if I had to guess this is probably a bad idea. I'm sure any changes you make will not survive a reboot let alone a system upgrade.

    I'd look at one of the other proxy packages available or if you have a FreeBSD host set up your proxy in a jail.



  • I'm using 'pound' for the same task on 2.2.6. Works perfect.
    On 2.3, I only had a chance to install the binary, seems to look for configs in the same place.



  • Run a separate instance of nginx for that purpose. That'll work just as well, doesn't require changing base source code, and doesn't risk breaking built-in things. I'd bind it to some high port(s) on 127.0.0.1 and use port forwards to direct traffic to it.

    I'd like to know what issues you had to fix post-upgrade.



  • Well 2.2.6 was perfect for my needs, it is just that I have now upgraded to 2.3 in the belief that nginx is bundled with pfsense and takes me away from having to install  a seperate package.  Surely there must be others that have similar usecase and  nginx that is used to run the webconfigurator could be extended for other uses.  Will wait for some devs to respond before i decide to roll back or look at alternative packages.



  • Thanks cmb, I shall try the separate nginx instance approach. Just to be sure can you confirm if /usr/local/etc/nginx/nginx.conf is used by the webconfigurator?  As it stands the webconfigurator is exposed to the WAN side as the built in nginx seems to be listening on all interfaces and my port 80 is open on the WAN side.

    @cmb:

    I'd like to know what issues you had to fix post-upgrade.

    PFsense was very slow all through the upgrade

    • The web configurator was accessible but still had the message stating  "pfsense was still booting"

    DHCP was not issuing/renewing  leases while it was "still booting" for more than 45 mins.. then it changed to installing packages for another hour and a half (To be fair I did not have that may packed installed in the first place)

    I had to reconfigure DNS servers on clients to (8.8.8.8) to allow internet access while the upgrade completed

    There was a crash (could have been a kernel panic), I have submitted the crash dump but the message keeps popping back every time I login. Not sure why there was a crash other than it seemed to work OK after a restart



  • @jkmuk:

    Will wait for some devs to respond before i decide to roll back or look at alternative packages.

    Um, founder of project here, thousands of source commits to my name. I replied to you just above.

    You don't need to install anything else, just don't try to modify the GUI's nginx instance. Setup your own nginx.conf, add a shell script /usr/local/etc/rc.d/my-nginx.sh containing something like:

    #!/bin/sh
    /usr/local/sbin/nginx -c /path/to/your/nginx.conf
    

    and you're done.



  • That'a a very clean approach! Nice!

    @jkmuk:

    I was using nginx on 2.2.6 as a way of hosting multiple websites (virtual hosts that proxy on to internal servers)

    Can you please share some sample configs on how you did that with nginx?



  • Nginx reverse proxy

    And a sample server block from my proxy config

    server {
    			listen	80;
    			server_name  yourdomain.com;
                    location / {
    					proxy_pass  http://your.internal.IP.address;
    					proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
    					proxy_set_header        Host $host;
    					proxy_set_header        X-Real-IP $remote_addr;
    					proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    					proxy_set_header        X-Forwarded-Proto $scheme;
    					}
    	}
    

    If you're on FreeBSD you'll have to install from ports with the real-IP module enabled since it's not enabled in the default configuration for the pre compiled package.



  • @Jailer:

    If you're on FreeBSD you'll have to install from ports with the real-IP module enabled since it's not enabled in the default configuration for the pre compiled package.

    Does it require installation from ports on pfSense 2.3?



  • @robi:

    @Jailer:

    If you're on FreeBSD you'll have to install from ports with the real-IP module enabled since it's not enabled in the default configuration for the pre compiled package.

    Does it require installation from ports on pfSense 2.3?

    Yes if you want the real-ip module enabled. That's what passes the client IP address to the server. Otherwise all connections on all your servers behind the proxy will show the proxy server's IP.

    Again, I wouldn't do this on the firewall, I'd do it in a jail on a host machine. But that's just my opinion.



  • A follow on NGINX question.  I now have another instance of NGINX running smoothly and doing what I need it to do. Now  it's time to get greedy  ;D Can the nginx version be upgraded to 1.9.x or even 1.10.x ?  Is there a plan as part of pfsense 2.3.1 perhaps to look into this?  I am looking to use the stream_proxy module to  divert openVPN traffic on my WAN port 80 to  OpenVP on the pfsense box.



  • @cmb:

    @jkmuk:

    Will wait for some devs to respond before i decide to roll back or look at alternative packages.

    Um, founder of project here, thousands of source commits to my name. I replied to you just above.

    You don't need to install anything else, just don't try to modify the GUI's nginx instance. Setup your own nginx.conf, add a shell script /usr/local/etc/rc.d/my-nginx.sh containing something like:

    #!/bin/sh
    /usr/local/sbin/nginx -c /path/to/your/nginx.conf
    

    and you're done.

    and how to be sure that my custom configured nginx will start after pressing 11 (restart webConfigurator) on screen?
    it kills all nginx processes and starts only system.



  • Bump !!!

    Following up on the NGINX upgrade question.



  • I have a similar configuration, and I fixed it with a little cron job calling this script each 5 minutes.

    It works for restarts of the pfSense box, or for restarts of the webconfigurator.

    If needed you can run it manually by ssh each time you do a webconfigurator restart.

    #!/bin/sh

    pgrep returns 0 if it founds, 1 if it's missing

    check_process() {
      [ "$1" = "" ]  && echo 0
      [ pgrep -f "$1" ] && return 1 || return 0
    }

    check_process "/usr/local/sbin/nginx -c /var/etc/nginx-vhost.conf"

    if [ $? -ne 1 ]; then
            echo "Starting VHost service"
            /usr/local/sbin/nginx -c /root/nginx-vhost.conf
    fi



  • Hi

    my PC-Engine (APU+wifi adapter) running pfSense 2.3.2 is acting like my FTTH modem (bypass of ISP modem = SFR (France)).

    Fiber OMT ==> re0 : fixed IP (ISP DHCP)
                    ==> re1 : LAN 192.168.0.1/24  =============>TV decoder is 192.168.0.253 (static dhcp lease in PFS)
                    ==> re2 : n/c
                    ==> ath0 : bridged with re1

    I need to add some fake url to PFsense so that the TV decoder can work

    I've followed all suggestions on that forum and started a new NGINX tree under /var/www with a dedicated .CONF file

    PFsense must act like my FTTH modem and respond to

    http://192.168.0.1/api/1.0/?method=system.getInfo
    http://192.168.0.1/api/1.0/?method=ftth.getInfo
    http://192.168.0.1/api/1.0/?method=wan.getInfo
    http://192.168.1.1/api/1.0/?method=lan.getHostsList

    How shall I modify slightly the nginx.conf file in order to be able to make PFS answer those request ?

    Here is the tree structure

    [2.3.2-RELEASE][root@pfSense.home]/var/www: find . -ls
    1203852        8 drwxr-xr-x    3 root                            wheel                                512 Dec 26 18:54 .
    1203853        8 drwxr-xr-x    3 root                            wheel                                512 Dec 26 18:23 ./api
    1203856        8 drwxr-xr-x    2 root                            wheel                                512 Dec 26 18:42 ./api/1.0
    1203867        8 -rw-r–r--    1 root                            wheel                                209 Dec 26 18:14 ./api/1.0/wan.xml
    1203869        8 -rw-r--r--    1 root                            wheel                                712 Dec 26 18:42 ./api/1.0/index.php
    1203857        8 -rw-r--r--    1 root                            wheel                                127 Dec 26 18:14 ./api/1.0/ftth.xml
    1203866        8 -rw-r--r--    1 root                            wheel                                509 Dec 26 18:13 ./api/1.0/system.xml
    1203859        8 -rw-r--r--    1 root                            wheel                                485 Dec 26 18:14 ./api/1.0/lan.xml
    1203870        8 -rw-r--r--    1 root                            wheel                                3957 Dec 26 18:54 ./mime.types
    1203868        8 -rw-r--r--    1 root                            wheel                                949 Dec 26 23:25 ./sfrtvbox.conf

    index.php should rewrite URL and send XML files so that the TV decoder should thing PFS is the FTTH modem.

    header('Content-Type: application/xml; charset=utf-8');
    if (isset($_GET['method'])){
            $method = $_GET['method'];
            switch ($method) {
                    case "system.getInfo":
                            echo file_get_contents('system.xml');
                            break;
                    case 'lan.getHostsList':
                            echo file_get_contents('lan.xml');
                            break;
                    case 'wan.getInfo':
                            echo file_get_contents('wan.xml');
                            break;
                    case 'ftth.getInfo':
                            echo file_get_contents('ftth.xml');
                            break;
            }
            return;
    }
    ?>
    
    

    Here is the nginx.conf special file : IT SHOULD LISTEN TO 80 … not 8088... but PFS is alreadly listening to 80

    
    [2.3.2-RELEASE][root@pfSense.home]/var/www: cat sfrtvbox.conf 
    load_module /usr/local/libexec/nginx/ngx_mail_module.so;
    load_module /usr/local/libexec/nginx/ngx_stream_module.so;
    
    #user  nobody;
    worker_processes  1;
    
    error_log  /var/log/sfrtvbox_error.log;
    pid        /var/run/sfrtvbox_nginx.pid;
    
    events {
        worker_connections  10;
    }
    
    http {
        include       mime.types;
        default_type  application/octet-stream;
        access_log  /var/log/sfrtvbox_access.log;
        sendfile        on;
        keepalive_timeout  65;
        server {
            listen       8088;
            server_name  decodeur;
            location /api/ {
                root   /var/www;
                index  index.php;
            }
            # http://192.168.1.1/api/1.0/?method=system.getInfo
            rewrite ^/api/1.0/(\?method=\S+.getInfo)$ /api/1.0/index.php?$1 last;
            #error_page  404              /404.html;
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
                root   /usr/local/www/nginx-dist;
            }
        }
    }
    
    

    Questions are :

    • how to include mod_rewrite.so in NGINX ( only found /usr/local/lib/lighttpd/mod_rewrite.so )
    • rewrite seems not to work, and throw me a PHP file without interpreting it.

    Thanx for your help

    Herve



  • @cmb:

    Um, founder of project here, thousands of source commits to my name. I replied to you just above.

    You don't need to install anything else, just don't try to modify the GUI's nginx instance. Setup your own nginx.conf, add a shell script /usr/local/etc/rc.d/my-nginx.sh containing something like:

    #!/bin/sh
    /usr/local/sbin/nginx -c /path/to/your/nginx.conf
    

    and you're done.

    Hello,

    I'm using nginx to serve up WPAD files, and I've added a wireless card to my pfsense box, so instead of WAN and LAN interfaces, I have WAN and WIFI1 and WIFI2 interfaces.

    Since each WIFI interface has its own gateway address, I think I need one nginx server for each WIFI interface, since I need to serve up two different sets of WPAD files.

    When I use the command above, the new nginx server seems to "bind" to the LAN interface (which I haven't yet disabled).  Is there a way for me to tell these new instances of nginx to be available on my WIFI1 and WIFI2 interfaces?

    -TM


Log in to reply