Automated scripts for Private Internet Access port forwarding

Bagpuss

**** UPDATE - 30th April 2018 ****

For those who use Deluge on Synology NAS, I've now provided some updated scripts to handle this.
The pia-port script (v1.06) has been updated to ssh into the NAS and update the deluge port on the fly.
There is a new script, deluge-port, which runs via cron on the NAS to poll for port changes. This might seem a little redundant, given that pia-port now updates the port on the fly, but it does handle the case where the port changes and the SSH might have failed.

NOTE: Do not use v1.06 is you want to continue using Transmission. This is for Deluge only.

In order for pia-port to work correctly with deluge, you will need to ensure that your create a valid key to allow login without password.
To achieve this, do the following as root on pfsense:

$ ssh-keygen -t ecdsa -b 521 (do not provide a passphrase when prompted)
$ ssh-copy-id -i ~/.ssh/id_ecdsa.pub root@synology (where synology is the hostname or IP address of your Synology NAS)

To test that you can now login without password, run
$ ssh root@synology

If this works, then the pia-port script will now work to update the deluge port on the fly.

There is another issue with Deluge on Synology, which is related to the umask. By default, all files are created as the deluge user with a umask of 022.
This results in files which users in the same group cannot delete or change (perms are -rw-r–r--).

Changing the umask for deluge is not straightforward on Synology, as the start-stop-daemon command doesn't support the -k or --umask parameters which are present on other OS's.

To fix this, I've provided another script, run-with-umask.sh, which can be used as a wrapper to set the umask to whatever you like.
Copy this script to /usr/local/bin on the Synology, and ensure that it has execute permissions.

Now backup the standard deluge start-stop script, by doing the following:
$ cp /var/packages/deluge/scripts/start-stop-status /var/packages/deluge/scripts/start-stop-status.orig

Replace the original script with the version attached to this post.
You can then set the umask to whatever you want by editing run-with-umask.sh.

**** UPDATE - 24th April 2018 ****

Realised that I'd not visited this thread for quite some time, so wanted to provide some updates to my original post.
PIA are about to radically change the API for acquiring port forwards (see here and here).
As a result of this, my scripts will soon stop working. You might already be seeing issues if you were using the Netherlands endpoint, as port forwarding was recently disabled.
If you're in Europe, the most performant endpoint now appears to be Germany.

I've attached a new version of the main script (v1.05), which has to be used slightly differently.
It's no longer necessary to have a cron job which runs every hour, as the port is only assigned once for the existence of a connection. In addition, you must request the port forward within two minutes of bringing up the connection. Attempts after that will fail.

In order to run the script on interface bring up, you can use devd (as I've seen mentioned by others later in the thread).
To do this, you need to ensure that the directory /usr/local/etc/devd exists. If not, create it.
In this, put the following script. It's name can be anything which ends in .conf. I call mine portforward.conf.

notify 0 {
        match "system"          "IFNET";
        match "subsystem"       "(ovpnc1)";
        match "type"            "LINK_UP";
        action "logger $subsystem is UP";
        action "/usr/local/bin/pia-port";
};

notify 0 {
        match "system"          "IFNET";
        match "subsystem"       "(ovpnc1)";
        match "type"            "LINK_DOWN";
        action "logger $subsystem is DOWN";
};

You need to change ovpnc1 to whatever the interface is that you are using for your port forwarding endpoint.
Once you've created the file, run /etc/rc.d/devd restart.
In the main script, you will also see a –interface ovpnc1 directive to the curl command. You need to change this also.

I've also attached v1.04 of the original script, which contains some fixes that have come up since I originally made this post.

Hope this helps.

Andy.

start-stop-status.txt
run-with-umask.sh.txt
pia-port.106.txt
pia-port.105.txt
pia-port.104.txt
deluge-port.txt

phil.davis

PIA recommend checking the port at least once an hour to see if the mapping has changed.
With that in mind, we now need to add a crontab entry. The easiest way to do this is by running 'crontab -e'.
An editor will open (usually vi), and you simply insert the following into the file

On pfSense, crontab will get rebuilt from config.xml - so manual edit changes will get overwritten.
Install the Cron package, and use that to add the crontab entry.

Bagpuss

@phil.davis:

On pfSense, crontab will get rebuilt from config.xml - so manual edit changes will get overwritten.
Install the Cron package, and use that to add the crontab entry.

Hi Phil,

Thanks for the feedback. I wasn't aware of this.

Can you tell me in what circumstances pfSense will do this? I've had a manually entered crontab entry running for a number of days, and it's still okay.
I'm not totally sure that my manual entry will be overwritten, as it's in a completely different crontab file.

The crontab which is built from the config.xml appears to be /etc/crontab.
The crontab which is changed by crontab -e is /var/cron/tabs/root.

I'm thinking that the two can co-exist without interfering with each other.

Having said that, it does make a lot of sense to use the Cron package, and manage things from the GUI.

Andy.

VTOLfreak

Thank you for this. :) Any chance of packing this up into a handy plugin for pfsense with a GUI page?

I don't really like the idea of adding too much scripts and files to my pfsense setup outside of plugins. Rightnow I'm putting off trying the 2.1.1 snapshots because I dont know if upgrading will wipe out all the changes I made to my 2.1 setup. If this script was in a plugin, pfsense takes care of setting it all up for you.

phil.davis

Can you tell me in what circumstances pfSense will do this?

Definitely at boot time - crontab, passwd databases, all that sort of "system management" setting gets rebuilt from the settings specified in config.xml
For crontab and passwd I don't think there are run-time events that cause them to rebuild from scratch, so manual edits probably survive a long time.
I definitely recommend NOT manually editing any files. Do it all through the GUI and available packages. If the available GUI+packages does not have a button for what you need, then put a feature request in RedMine and (if you have some coding skills) make the GUI enhancements and submit a pull request on GitHub.
It is an Open Source project, and if everyone contributes features that they need then we all get the benefit - or the software bloat ;)

sacredgaming

Do think this would work with plex server for viewing outside my network?

dalesd

Hi Andy,
I'm also a PIA customer and I'm trying your script for port forwarding.

So far, it's not working. Where do I find the log file so I can troubleshoot this? Your post says to check the /var/log/messages file, but there isn't one.

dalesd

Turns out I had my cron entry set up wrong. I fixed that and now the log is showing what's going on.


Mar 1 09:12:19	php: /index.php: New alert found: pfSense is restoring the configuration /conf/backup/config-1393680553.xml
Mar 1 09:12:19	php: /index.php: pfSense is restoring the configuration /conf/backup/config-1393680553.xml
Mar 1 09:12:19	php: /index.php: New alert found: No config.xml found, attempting last known config restore.
Mar 1 09:12:19	php: /index.php: No config.xml found, attempting last known config restore.
Mar 1 09:02:00	root: pia-port: New port number () inserted into config file.
Mar 1 09:02:00	root: pia-port: Created new PIA Client ID.

When I run the script in a terminal, here's what I get:

[2.1-RELEASE][admin@pfsense.localdomain]/usr/local/bin(51): ./pia-port 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   149    0    57  100    92    245    395 --:--:-- --:--:-- --:--:--   519
expr: syntax error

[: -gt: unexpected operator
I/O warning : failed to load external entity "/rule[descr="Torrent"]/local-port"
[2.1-RELEASE][admin@pfsense.localdomain]/usr/local/bin(52):

I think curl isn't working or installed correctly, because this happens:


[2.1-RELEASE][admin@pfsense.localdomain]/var/log(46): pkg_add -r curl           
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/curl.tbz... Done.
pkg_add: package 'curl-7.24.0' or its older version already installed
[2.1-RELEASE][admin@pfsense.localdomain]/var/log(47): curl                      
curl: Command not found.

I don't know how to fix that.

jgorman

Hello,

Thanks a lot for this script.

I'm getting the following:

cron: login_getclass: unknown class 'pia-port # #'

I should not, I created the pia-port using the Diagnostics-Edit File from within pfsense and not by ssh-ing it ont the machine…

Don't know if that makes a difference.

Any help would be greatly appreciated.

THANKS!

ppierre

Nice script.

Juste an edge case that I've encounter. I've got this reply :

{"error":"port forwarding not available for this region"}

Ewrything was fine, but in my setup I haven't setup the VPN as default route (only one IP go through it).

So I had to add this to your script :

curl [b]--interface [/b]"$LOCAL_IP" ...

And everything went smooth. It seem IPA need to get the port request from his own network.
Maybe you should add it to your post. In case someone would try a similar setup.

zounder1

To clarify this. Change the "PORT" line in the the script with this:

PORT=curl –interface $LOCAL_IP -d "user=$USERNAME&pass=$PASSWORD&client_id=$(cat $PIACLIENTID)&local_ip=$LOCAL_IP" https://www.privateinternetaccess.com/vpninfo/port_forward_assignment

This should be put into V1.02 of this script. For setups with multiple gateways you need this to ensure the request goes out via the proper PIA VPN connection. With this change the script will try and use the default gateway which may not be the correct gateway.

Hats off to Bagpuss the creator of this script! This is awesome.

Hats off to ppierre for adding that important last catch with the script.

@ppierre:

Nice script.

Juste an edge case that I've encounter. I've got this reply :
{"error":"port forwarding not available for this region"} 
Ewrything was fine, but in my setup I haven't setup the VPN as default route (only one IP go through it).

So I had to add this to your script :
curl [b]--interface [/b]"$LOCAL_IP" ...
And everything went smooth. It seem IPA need to get the port request from his own network.
Maybe you should add it to your post. In case someone would try a similar setup.

Bagpuss

@zounder1:

To clarify this. Change the "PORT" line in the the script with this:

PORT=curl –interface $LOCAL_IP -d "user=$USERNAME&pass=$PASSWORD&client_id=$(cat $PIACLIENTID)&local_ip=$LOCAL_IP" https://www.privateinternetaccess.com/vpninfo/port_forward_assignment

This should be put into V1.02 of this script. For setups with multiple gateways you need this to ensure the request goes out via the proper PIA VPN connection. With this change the script will try and use the default gateway which may not be the correct gateway.

Hats off to Bagpuss the creator of this script! This is awesome.

Hats off to ppierre for adding that important last catch with the script.

Not checked this thread for a while, and I've just realised that I hadn't posted v1.02 where I did indeed catch this bug. My apologies.
My fix is slightly different, but it achieves the same thing.

Attached to this post is v1.03, where I've caught another issue.
I kept seeing entries in my syslog which said 'pfSense is restoring the configuration /conf/backup/config-1393680553.xml' or similar.
Further debugging revealed that the PIA address wasn't resolving for curl, and hence the script returned a NULL for the port number.
I wasn't catching this error, and so the script was copying an empty /tmp/config.pia over the config.xml.

The new script fixes this, and also provides the return code from curl, so you can try and work out what went wrong.

pia-port.v103.txt

Bagpuss

@dalesd:

Hi Andy,
I'm also a PIA customer and I'm trying your script for port forwarding.

So far, it's not working. Where do I find the log file so I can troubleshoot this? Your post says to check the /var/log/messages file, but there isn't one.

Sorry. That should have been /var/log/system.log

Bagpuss

As an aside, my original instructions contain details for managing cron from the command line.
Whilst this works, there is a cron GUI package available, and this works just as well.

I'm pretty sure that anyone who is okay with installing this stuff in the first place can use cron,
but the pfSense preferred way is to use the GUI package. I'm told that changes to the underlying
pfSense OS going forward might mean that command line cron entries are lost across reboots.
The GUI package stores the cron entries in your config.xml, thereby ensuring their survival.

binaryjay

Amazing post. I was in the middle of reinventing this wheel (moving away from openvpn client on specific hosts) but this saved me a bunch of time. Had to make some minor changes to account for nanoBSD which I have PM'd you about. May be worth checking if the filesystem is ro and remounting before continuing to make your script pfsense distribution agnostic.

Also, instead of force deleting config.cache (doesn't that seem a little messy?), would it not be better to just run:
/etc/rc.reload_all

? Scratch that, running that manually seems to kill everything and requires a reboot.

After I started running this script I've been seeing inetd complaining in the logs about connection aborts and I wonder if it has something to do with the unexpected blowing away of config.cache.

Imaginos

So these two commands are causing an interesting problem:

pkg_add -r curl
pkg_add -r xmlstarlet

They both reach out to ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/… and both generate the following error.

Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/xmlstarlet.tbz: File unavailable (e.g., file not found, no access)
pkg_add: unable to fetch 'ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/xmlstarlet.tbz' by URL

A brief investigation reveals that the error is correct and that file path does not exist. There is a packages-8-stable, 8.4-release, 9-stable and others, but not an 8.3 flavor.

You can see the directory here http://ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/.

I'm at a loss of how to proceed and my kung fu with this is 30 years old and not potent enough to figure it out.

Cino

@Imaginos:

So these two commands are causing an interesting problem:
pkg_add -r curl
pkg_add -r xmlstarlet
They both reach out to ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/… and both generate the following error.
Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/xmlstarlet.tbz: File unavailable (e.g., file not found, no access)
pkg_add: unable to fetch 'ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/xmlstarlet.tbz' by URL
A brief investigation reveals that the error is correct and that file path does not exist. There is a packages-8-stable, 8.4-release, 9-stable and others, but not an 8.3 flavor.

You can see the directory here http://ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/.

I'm at a loss of how to proceed and my kung fu with this is 30 years old and not potent enough to figure it out.

searching with help your kung fu

https://forum.pfsense.org/index.php?topic=78935.0

AMD64
setenv PACKAGESITE http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.3-release/

I386
setenv PACKAGESITE http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.3-release/

zounder1

@Cino:

@Imaginos:
So these two commands are causing an interesting problem:
pkg_add -r curl
pkg_add -r xmlstarlet
They both reach out to ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/… and both generate the following error.
Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/xmlstarlet.tbz: File unavailable (e.g., file not found, no access)
pkg_add: unable to fetch 'ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/xmlstarlet.tbz' by URL
A brief investigation reveals that the error is correct and that file path does not exist. There is a packages-8-stable, 8.4-release, 9-stable and others, but not an 8.3 flavor.

You can see the directory here http://ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/.

I'm at a loss of how to proceed and my kung fu with this is 30 years old and not potent enough to figure it out.
searching with help your kung fu

https://forum.pfsense.org/index.php?topic=78935.0

AMD64
setenv PACKAGESITE http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.3-release/

I386
setenv PACKAGESITE http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.3-release/

I used:
setenv PACKAGESITE http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.3-release/Latest/

Then I could install curl.

But quite the output for xmlstarlet
pkg_add -r xmlstarlet
Fetching http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.3-release/Latest/xmlstarlet.tbz… Done.
Fetching http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.3-release/All/pkg-config-0.25_1.tbz... Done.
Fetching http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.3-release/All/libxml2-2.7.8_2.tbz... Done.
pkg_add: warning: package 'libxml2-2.7.8_2' requires 'libiconv-1.13.1_2', but 'libiconv-1.14_1' is installed
Fetching http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.3-release/All/libgpg-error-1.10.tbz... Done.
pkg_add: warning: package 'libgpg-error-1.10' requires 'libiconv-1.13.1_2', but 'libiconv-1.14_1' is installed
pkg_add: warning: package 'libgpg-error-1.10' requires 'gettext-0.18.1.1', but 'gettext-0.18.3.1' is installed
Fetching http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.3-release/All/libgcrypt-1.5.0.tbz... Done.
pkg_add: warning: package 'libgcrypt-1.5.0' requires 'libiconv-1.13.1_2', but 'libiconv-1.14_1' is installed
pkg_add: warning: package 'libgcrypt-1.5.0' requires 'gettext-0.18.1.1', but 'gettext-0.18.3.1' is installed
Fetching http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.3-release/All/libxslt-1.1.26_3.tbz... Done.
pkg_add: warning: package 'libxslt-1.1.26_3' requires 'libiconv-1.13.1_2', but 'libiconv-1.14_1' is installed
pkg_add: warning: package 'libxslt-1.1.26_3' requires 'gettext-0.18.1.1', but 'gettext-0.18.3.1' is installed
pkg_add: warning: package 'xmlstarlet-1.0.5' requires 'libiconv-1.13.1_2', but 'libiconv-1.14_1' is installed
pkg_add: warning: package 'xmlstarlet-1.0.5' requires 'gettext-0.18.1.1', but 'gettext-0.18.3.1' is installed

It seems to have installed xmlstarlet despite the warnings. But I'm new to this package manager so I'm not sure if this is asking for trouble or if it just installs and hopes that everything runs with different versions of the dependencies.

Soloam

@ppierre:

Nice script.

Juste an edge case that I've encounter. I've got this reply :
{"error":"port forwarding not available for this region"} 
Ewrything was fine, but in my setup I haven't setup the VPN as default route (only one IP go through it).

So I had to add this to your script :
curl [b]--interface [/b]"$LOCAL_IP" ...
And everything went smooth. It seem IPA need to get the port request from his own network.
Maybe you should add it to your post. In case someone would try a similar setup.

Hello, I'm having the same error

{"error":"port forwarding not available for this region"}

All runs well, the pia_port.txt is created but in the log I get that error.

Can any one help me? Using last version of the script.

Best Regards
Soloam

saytar

Not sure I can help as I don't have My VPN back up yet since my box crashed and I had to reflash and set it backup,,,But before the crash I did have PIA up and working on Pfsense 2.1.3.

I didn't have any trouble with it changing IP's when they switched around…my logs showed Pfsense detecting the change and just remaking the tunnel.....and I never noticed it............until I happened to check the logs.....

Their is a how to on line somewhere I went by to set it up. The setup was for ONLY the Client setup. I think maybe everyone is Making the Issue More complex than needed.

Check this link. http://www.bodenzord.com/archives/324
Maybe it will help

Chow