Net.inet.ip.fastforwarding performance tweak



  • A while ago, I found that enabling net.inet.ip.fastforwarding would greatly aid with openVPN throughput of a pfSense virtual machine.
    As a side effect, the setting also lead to a significant decrease in CPU load. Detailed info over here: http://forum.pfsense.org/index.php/topic,47567.0.html

    I can now confirm that the setting also reduces CPU load on ALIX based pfSenses:

    Didn't bother to mark the point when I enabled the setting, as it's pretty obvious from the above graphs.
    So far, I discovered no ill side effects on the ALIX hardware either. One pfSense user however reported fastforwarding related difficulties with remote printing.
    Some descriptive information about net.inet.ip.fastforwarding can be found in: http://lists.freebsd.org/pipermail/freebsd-net/2004-January/002534.html

    I have no data on openVPN performance so far, because I don't use that on the Alix here.


  • Netgate Administrator

    Wow, I had not read that thread (or maybe I forgot about it  ::)). That's a massive performance increase, makes me want to run some tests immediately.

    Steve



  • Yes, please do and report back your findings :)



  • Will this improve performance on normal routing ?
    Are there any side effects ?
    What is this option doing exactly ? Anything to read about that ?



  • It'll break IPsec for sure, and potentially other things, which is why we don't enable it by default.



  • @gridrun:

    A while ago, I found that enabling net.inet.ip.fastforwarding would greatly aid with openVPN throughput of a pfSense virtual machine.

    I wonder if it's still a problem with the newer FreeBSD 8.3 kernel used by pfsense 2.1-BETA

    I would also imagine that FreeBSD kernel developers might be interested to look into this, as both OpenVPN and VMware are very widely deployed.


  • Netgate Administrator

    Reporting back. I ran some tests on my home box with iperf. The box is a Pentium4-M underclocked to 1.2GHz via a low FSB. Not a fast machine!

    I tested first using the iperf server in the pfSense box.
    No fast fowarding:

    C:\Temp>iperf -v
    iperf version 1.7.0 (13 Mar 2003) win32 threads
    
    C:\Temp>iperf -c 192.168.2.1 -t 30
    ------------------------------------------------------------
    Client connecting to 192.168.2.1, TCP port 5001
    TCP window size: 8.00 KByte (default)
    ------------------------------------------------------------
    [1912] local 192.168.2.22 port 1907 connected with 192.168.2.1 port 5001
    [ ID] Interval       Transfer     Bandwidth
    [1912]  0.0-30.0 sec  1.07 GBytes   306 Mbits/sec
    
    

    With fast forwarding:

    
    C:\Temp>iperf -c 192.168.2.1 -t 30
    ------------------------------------------------------------
    Client connecting to 192.168.2.1, TCP port 5001
    TCP window size: 8.00 KByte (default)
    ------------------------------------------------------------
    [1912] local 192.168.2.22 port 2127 connected with 192.168.2.1 port 5001
    [ ID] Interval       Transfer     Bandwidth
    [1912]  0.0-30.0 sec  1.04 GBytes   297 Mbits/sec
    
    

    No difference to speak of, within normal test variation.
    Then I tested through the box running the iperf server on a different pfSense box.
    No fast fowarding:

    
    C:\Temp>iperf -c 192.168.1.129 -t 30
    ------------------------------------------------------------
    Client connecting to 192.168.1.129, TCP port 5001
    TCP window size: 8.00 KByte (default)
    ------------------------------------------------------------
    [1912] local 192.168.2.22 port 1966 connected with 192.168.1.129 port 5001
    [ ID] Interval       Transfer     Bandwidth
    [1912]  0.0-30.0 sec   672 MBytes   188 Mbits/sec
    

    With fast forwarding:

    
    C:\Temp>iperf -c 192.168.1.129 -t 30
    ------------------------------------------------------------
    Client connecting to 192.168.1.129, TCP port 5001
    TCP window size: 8.00 KByte (default)
    ------------------------------------------------------------
    [1912] local 192.168.2.22 port 2126 connected with 192.168.1.129 port 5001
    [ ID] Interval       Transfer     Bandwidth
    [1912]  0.0-30.0 sec   858 MBytes   240 Mbits/sec
    

    20% improvement.  :)
    Not quite the dramatic increase you saw but I suspect you may have had other issues since 1.5Mbps seems too slow.
    I also note that biggsy's result is tainted by the fact that his initial test was using an 8k window size but the improved result used a 63k window size. The different versions/ports of iperf seem to have different defaults which can catch you out.  ;)

    Since this is my home box I'll see if anything has broken over the next few days.

    Steve



  • @dhatz:

    I wonder if it's still a problem with the newer FreeBSD 8.3 kernel used by pfsense 2.1-BETA

    I have since sold the small WISP business that used the virtualized pfSense system I reported on earlier. However, my new employer (an ISP) asked me to build a vSphere cluster and help them virtualize large parts of their central office. So I will soon be able to do further testing with pfSense in this virtual environment.


Locked