Speedtest CLI. Run speedtest on pfSense box

getrav

I have nanobsd I don't know if the instructions would be different for normal installation
I found the speedtest code https://github.com/sivel/speedtest-cli

Login thru SSH

/etc/rc.conf_mount_rw
setenv PKG_TMPDIR /root/
pkg_add -r http://files.pfsense.org/packages/8/All/python26-2.6.6.tbz
fetch -o /root/speedtest-cli https://raw.github.com/sivel/speedtest-cli/master/speedtest_cli.py
chmod +x /root/speedtest-cli
/etc/rc.conf_mount_ro

To run speedtest after installation:

/root/speedtest-cli

for help type:

/root/speedtest-cli -h

jimp

That looks nice, though it's usually better to test bandwidth through the firewall and not from the firewall itself. pfSense has been optimized to work in a routing role, so sometimes you might see reduced numbers when pfSense itself is acting like the client.

fholzer

Just my 50Cents,

It would be nice to have this Feature via GUI.
I now its better to test the Speed through the firewall and not from the firewall itself.

But in many cases i need to test it from the pfsense itself.
(no - or limited access to the clients)

Generally i would use it to test if the WAN Connection give me the 16Mbit the customer had buyed or if the connection is below 10MBit (or lower)

regards,

Franz

pfBlense

Just a brief update for people using pfSense 2.2+. Instead of using pkg_add you now need to do:

pkg update
pkg install python

Thanks for the post on this!!

nikkon

Got this on 2.2
On a 1000 Mbit/s connection i barely get 350.20 Mbit/s on my setup.
is there any way to tweak network settings and improve this?

Zaphon

@nikkon:

Got this on 2.2
On a 1000 Mbit/s connection i barely get 350.20 Mbit/s on my setup.
is there any way to tweak network settings and improve this?

Well as was suggested, don't run this on the box, run it behind the box. Also, your speed through the box is going to vary greatly based on the type of NIC's you have and the speed of your CPU. I had a computer that had a pair of Realtek nic's on the motherboard (single core box, about 6-7 years old, makes a great little router), and with gigabit I was maxing out in the 300-400Mbit range. I bought a dual port Intel NIC and now I'm pushing in the 995Mbit range (this is tested from behind the box to a server on the other side in a lab). However if the box is much more recent, has a nice fast CPU, your mileage may vary.

nikkon

in any case i tested the speed via iperf from my box to a miktorik switch and i got 1Gbit/s full
I suspect 2 things:

ISP - they came once and tested with another machie…worked well
pppoe performance

luckman212

@getrav:

…I found the speedtest code...

Wanted to thank you for posting this. Today I used this to do a remote speed test at a site where I did not have physical access to nor any remote access to a machine behind the firewall. The vendor needed us to confirm that a speed increase had been implemented by the ISP and this was the perfect tool for this. Got the job done and saved us a few hours of travel time for what amounted to a 5 minute test.

Thank you 8)

floz

Thanks for this, exactly what I was looking for.

Just my little 2 cents how to get this working on pfSense 2.2 and later.


#/etc/rc.conf_mount_rw
setenv PKG_TMPDIR /root/
pkg install python27-2.7.10_1
ln -s /usr/local/bin/python2 /usr/local/bin/python
pkg_add -r http://files.pfsense.org/packages/8/All/python26-2.6.6.tbz
fetch -o /root/speedtest-cli https://raw.github.com/sivel/speedtest-cli/master/speedtest_cli.py
chmod +x /root/speedtest-cli
#/etc/rc.conf_mount_ro

doktornotor

Or just use 2.2.5; python is already there.


$ file /usr/local/bin/python2.7
/usr/local/bin/python2.7: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 10.1, stripped

ariyako

curl -Lo speedtest-cli https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest_cli.py
chmod +x speedtest-cli

nano speedtest-cli

change

#!/usr/bin/env python

to

#!/usr/local/bin/python2.7

;)

mdmogren

Just wanted to post this update here for others who use this tool - it is now a package and can be installed using


pkg install py27-speedtest-cli

Then just run

./usr/local/bin/speedtest-cli

Smoothrunnings

Is there no GUI version of this?

My ubnt USG Pro has a speed tester app on it's firewall GUI. I don't see how it's anymore complicated to get one for Pfsense.

Thanks,

jimp

@Smoothrunnings:

Is there no GUI version of this?

My ubnt USG Pro has a speed tester app on it's firewall GUI. I don't see how it's anymore complicated to get one for Pfsense.

Thanks,

No, because speed testing from the firewall itself is inaccurate and unreliable. We don't want to encourage people to rely on bad data. See my previous response on this thread.

authenticx

Actually this is a good thing to use if you suspect your switch or cabling may be contributing to a loss of bandwidth. This would narrow the points of failure to your firewall, the connection between your firewall and gateway and the gateway itself. If you have a large difference between what this cli test reports and what you see in a speed test from your pc you just gained valuable troubleshooting information.

luckman212

I agree. It can have the standard YMMV/caveat attached, but I have gotten a ton of useful information from having the speedtest_cli pkg installed on my remote units. In my testing, a C2xxxx CPU is more than capable of pulling 400-500Mbps directly with this tool. Very useful!

icewraithuk

I use this to keep my ISP honest, which is a trick in itself :) For those that are interested here's my hack:

Go to www.thingspeak.com and register for a free account, create a channel with three fields (I only care about ping, DL and UL, you may want more), go to the API page and make a note of your write API.

Do the stuff to get speedtest on your firewall, run a speedtest and note which server it decides is your local/fastest - I use the one hosted by my ISP, for example. Then run a speedtest –list | grep <your chosen="" server="" name="">to get the ID (eg. speedtest --list | grep Virgin in my case)

Run a speedtest --server <the id="" you="" found="">--csv to test

Edit the speedtest.py file and add these bits - note, this isn't clean or clever, I've gone with functional!

Under the other import lines at the top:

import urllib2

myAPI="<your api="" key="" goes="" here,="" in="" the="" quotes="">"
baseURL = 'https://api.thingspeak.com/update?api_key=%s' % myAPI
print baseURL</your>

Then go down to around line 700 and look for the CSV output section, add the lines with -> below

def csv(self, delimiter=','):
        """Return data in CSV format"""

        data = self.dict()
        out = StringIO()
        writer = csv.writer(out, delimiter=delimiter, lineterminator='')
        row = [data['server']['id'], data['server']['sponsor'],
               data['server']['name'], data['timestamp'],
               data['server']['d'], data['ping'], data['download'],
               data['upload']]
        writer.writerow([to_utf8(v) for v in row])
->      f = urllib2.urlopen(baseURL + "&field1=%s&field2=%s&field3=%s" % (self.ping, self.download, self.upload))
->      print f.read()
->      f.close ()
        return out.getvalue()

Now run speedtest.py –server 1234 --csv and wait for it to finish, now check on ThingSpeak and check your three fields now have data.

I then put the command in my crontab:

*/30 * * * * /usr/local/bin/python2.7 speedtest.py --server 1234 --csv >/dev/null 2>&1

and now every 30 minutes my firewall does a speed test and updates ThingSpeak, which I can check the lovely graphs and make sure I'm relatively consistently getting decent metrics.</the></your>

Sergestux

@mdmogren:

Just wanted to post this update here for others who use this tool - it is now a package and can be installed using
pkg install py27-speedtest-cli
Then just run
./usr/local/bin/speedtest-cli

Gracias era lo que buscaba

cukal

@icewraithuk What a great hack & thanks for letting me discover ThingSpeak!

I changed the output from bits/sec to Mbit/sec:

f = urllib2.urlopen(baseURL + "&field1=%s&field2=%s&field3=%s" % (self.ping, int(self.download) / 1048576, int(self.upload) / 1048576))

S762

Sorry, I know this is an old topic but how to I uninstall this package? I used the following to install but don't see it listed under the package manager and would like to remove it.

pkg install py27-speedtest-cli

RaidArray

You should be able to do a "pkg remove <package name>" from the cli via ssh

S762

@raidarray said in Speedtest CLI. Run speedtest on pfSense box:

You should be able to do a "pkg remove <package name>" from the cli via ssh

Thank you!

RaidArray

Hi all

Not sure if this is still actively being requested but as a learning exercise for myself i've created a small widget that runs a speedtest from the dashboard happy to share. I'm not really a CLI guy so went down the GUI/Widget route.

Full how to can be found @ https://www.spacejunk-inc.com/pfsense/widget/speedtest/2019/03/10/Speed-Test-Widget.html

please let me know what you think, is it useful? how can i improve it?

Gertjan

Nice !

But keep in mind :

@jimp said in Speedtest CLI. Run speedtest on pfSense box:

No, because speed testing from the firewall itself is inaccurate and unreliable. We don't want to encourage people to rely on bad data. See my previous response on this thread.

@raidarray said in Speedtest CLI. Run speedtest on pfSense box:

how can i improve it?

By adding the option to select the server being used to test ?
Default, speed-test used the closed server on "the list".

provels

@icewraithuk said in Speedtest CLI. Run speedtest on pfSense box:

I use this to keep my ISP honest, which is a trick in itself :) For those that are interested here's my hack:

Very cool. Greetings from the Future!

EDIT - What fun! I was able to combine instructions from both @icewraithuk and @raidarray and throw this chart into the Dashboard using the dark theme. It runs the CLI speedtest against the lowest ping server every 10 minutes, posts to Thingspeak and the Dashboard reads back the Public feed.

Live Chart

kcallis

@icewraithuk Am I missing something or does one have to have a university or company email account? It definitely did not seem to like the gmail account!

provels

@kcallis I just used an Outlook.com account. BTW, the post above from @icewraithuk was posted 2 years ago and it was his only post ever on the forum.

PS - I found a cool chart in the ThingSpeak forums with a chart template that consolidates the CSV data produced by @icewraithuk 's method that I managed to modify to show my upload/download/ping data all in one chart, but you'll need to muddle through the edits manually. There are notes in the .html file that tell you where to put your info. Not saying it's the best data set, but a fun excercise anyway.
https://community.thingspeak.com/forum/announcements/thingspeak-live-chart-multi-channel-second-axis-historical-data-csv-export/

0daymaster

@provels That is rad but it seems like running speedtest every ten minutes would be quite a burden on your WAN connection. Can you post the PHP you used for the widget? Thanks.

provels

@0daymaster I run the speedtest every 30 minutes now, but it's just me, so who cares? You pretty much need everything in this thread to do what I did.
1 - Install speedtest-cli on the FW. Run some tests from the shell to get a feel for it.
2 - Setup a Thinkspeak.com account and do the @icewraithuk edits to the speedtest.py file (make @cukal 's edits for Mbps instead of MB if you wish).
3 - Now you should be able to send the data to Thinkspeak to generate the graphs. Format them in the style/colors as you wish. You'll use the
Read API Key to post the data you'll add to the widget.
4 - Then just use @RaidArray 's instruction to add the URL for your chosen Thingspeak graphs and to create the widget you can load from the FW homepage.
5- You can DL the html file from the Thingspeak forum post and edit it in a text editor to make it useful for your data. I didn't incorporate it into the FW homepage, though. I now also use a Windows version of speedtest-cli (Google) running as a scheduled task on an inside server so the tests actually run through the FW rather than from it
All credit goes to everyone else. I just managed to screw some parts together, in spite of myself.

joeontherun

Just want to have my input here took me a while to figure my way around all the coding but with a bit of persistence i got my 15 min speed checker working & managed to get it working in pfsense will attach screen shot for you to see well chuffed with the outcome just want to say thanks to all that inputted information to make this happen,

It would be nice to drop all 3 data UL DL & Ping in to the 1 graph i tried to do it but it just got to complicated

Thanks

speedtest cli.PNG

provels

@joeontherun Nice work! Just duplicate what you did for your "Speed Checker" graph. Create the graphs on Thingspeak, create the .inc files pointing to the links and the .php widgets! Then smile! Can help as needed, but I think you got it! :)

justheretohelp

The link for how to make a widgit with speed test is dead.

Anyone have a instruction for how to get speedtest in the GUI?

provels

@justheretohelp
Create speedtest.inc in /usr/local/www/widgets/include
This will add the Speedtest entry to the pfSense Dashboard widget choices.

<?php
/* File     : /usr/local/www/widgets/include/speedtest.inc
 * Author   : Zak Ghani
 * Date     : 10-03-2019
 * 
 * This software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
 * OR CONDITIONS OF ANY KIND, either express or implied.
 * 
 */

$Download_title = gettext("Speedtest - Mbps");

?>

Then create speedtest.widget.php in /usr/local/www/widgets/widgets

<?php
/* File     : /usr/local/www/widgets/widgets/speedtest.widget.php
 * Author   : Zak Ghani
 * Date     : 10-03-2019
 * 
 * This software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES 
 * OR CONDITIONS OF ANY KIND, either express or implied.
 * 
 * URL Choices :    https://www.bing.com/widget/t/speedtest
 *                  https://www.measurementlab.net/p/ndt-ws.html

 */

require_once("guiconfig.inc");

?> 

<iframe 
    src="https://www.bing.com/widget/t/speedtest"
width="498"
height="500"
frameborder="0"
>
</iframe>

When you copy the code, be sure to get it all, the windows scrolls. You may need to mess with the pixel height/width of the object.
Think that's it...

RaidArray

Sorry about the dead links. In the middle of moving websites. @provels has it spot on coding wise.
Should be able to get the instructions back on line within the next few days.

luckman212

@RaidArray Are those instructions back up online somewhere by chance?

MarcO42

Hi,
this long discussion was nearly wat I'm locking for.
But in the end I build it on my own...and maybe someone is interested in this. Here is what I did:
I have mounted a USB drive (not really necessary for everybody) and build a script that can called by cron.
This script is parsing the response from speedtest-cli on directly on the WAN interface and writes a log file. From this log file I took the last 7 lines (cron is running every day) and create a /usr/local/www/js/speettest_cli.js as data file for my chart from https://www.chartjs.org.
And here is the result:

The tooltip is a mouse over and you can also hide single elements like this:

The fact that you see only the 03.03. and 04.03. is that I created yesterday and these are just placeholder until the new data arrived.

Here you can finde all files.

speedtest.zip

Feel free to use it. Think everyone have to adapt all the paths in the \usr\local\bin\speedtest.sh
Also in this file you have to set you place for the speedtest...like speedtest-cli --server 16688

Cheers
MarcO

lastb0isct

Does anyone have the speedtest.py still? I can't seem to find it.

luckman212

@lastb0isct Have you tried...

# pkg search speedtest
# pkg install py27-speedtest-cli

if pkg gives you a Shared object "libcryptoauth.so.3" not found, required by "pkg" error, try

# pkg-static install py27-speedtest-cli

provels

@lastb0isct said in Speedtest CLI. Run speedtest on pfSense box:

Does anyone have the speedtest.py still? I can't seem to find it.

It comes with the package when you install it. SSH to your box and run:

pkg search speedtest

[2.4.4-RELEASE][root@fw.workgroup]/root: pkg search speedtest
py37-speedtest-cli-2.1.1       Command line interface for testing internet bandwidth

Then run

pkg install py37-speedtest-cli-2.1.1

Have fun!

lastb0isct

Where does the speedtest.py live after installation?