HFSC & VOIP over OpenVPN



  • Hi,

    I've run the traffic shaper wizard and set the ip of the voip gateway, which is a host on the LAN interface.

    I can see packets going to the queue qVOIP (with pfctl -vvs queue) from clients connected via PPTP, but not from clients connected via OpenVPN.

    There rules on the PPTP VPN and on the OpenVPN interfaces are the same (pass all).

    I'm using 2.0-RC1 (i386) built on Thu Mar 3 02:47:15 EST 2011.

    Do you have any suggestion?

    Thank you very much,
    Stenio



  • Can you show your ruleset?
    Please upgrade even to the latest snapshot and test again.



  • Here are the rules. I can't upgrade until friday because the system is in production and every time I do an update it has to be reconfigured  :(

    $ pfctl -sr
    scrub in on et1 all fragment reassemble
    scrub in on rl0 all fragment reassemble
    anchor "relayd/" all
    block drop in log all label "Default deny rule"
    block drop out log all label "Default deny rule"
    block drop in quick inet6 all
    block drop out quick inet6 all
    block drop quick proto tcp from any port = 0 to any
    block drop quick proto tcp from any to any port = 0
    block drop quick proto udp from any port = 0 to any
    block drop quick proto udp from any to any port = 0
    block drop quick from <snort2c>to any label "Block snort2c hosts"
    block drop quick from any to <snort2c>label "Block snort2c hosts"
    block drop quick from <pfsnortsamout>to any label "Block pfSnortSamOut hosts"
    block drop quick from any to <pfsnortsamin>label "Block pfSnortSamIn hosts"
    block drop in log quick proto tcp from <sshlockout>to any port = ssh label "sshlockout"
    block drop in log quick proto tcp from <webconfiguratorlockout>to any port = http label "webConfiguratorlockout"
    block drop in quick from <virusprot>to any label "virusprot overload table"
    block drop in on ! et1 inet from 192.168.0.0/24 to any
    block drop in inet from 192.168.0.254 to any
    block drop in on et1 inet6 from fe80::204:a7ff:fe05:9ac7 to any
    pass in on et1 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
    pass in on et1 inet proto udp from any port = bootpc to 192.168.0.254 port = bootps keep state label "allow access to DHCP server"
    pass out on et1 inet proto udp from 192.168.0.254 port = bootps to any port = bootpc keep state label "allow access to DHCP server"
    block drop in log quick on rl0 from <bogons>to any label "block bogon networks from WAN"
    block drop in on ! rl0 inet from 192.168.1.0/24 to any
    block drop in inet from 192.168.1.2 to any
    block drop in on rl0 inet6 from fe80::204:a7ff:fe08:9380 to any
    pass in on lo0 all flags S/SA keep state label "pass loopback"
    pass out on lo0 all flags S/SA keep state label "pass loopback"
    pass out all flags S/SA keep state allow-opts label "let out anything from firewall host itself"
    pass out route-to (rl0 192.168.1.254) inet from 192.168.1.2 to ! 192.168.1.0/24 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
    pass in quick on et1 proto tcp from any to (et1) port = http flags S/SA keep state label "anti-lockout rule"
    pass in quick on et1 proto tcp from any to (et1) port = ssh flags S/SA keep state label "anti-lockout rule"
    pass in on rl0 inet proto tcp from any to 192.168.1.2 port = pptp flags S/SA modulate state label "allow pptpd 192.168.1.2"
    pass in inet all flags S/SA keep state label "NAT REFLECT: Allow traffic to localhost" tagged PFREFLECT
    match inet proto udp from 192.168.0.40 to any label "USER_RULE: VOIP Adapter" queue qVoIP
    match inet proto udp from 10.8.0.0/24 to 192.168.0.40 label "USER_RULE: VOIP Adapter (stenio)" queue qVoIP
    match proto tcp from any to any port = 3389 label "USER_RULE: m_Other MSRDP outbound" queue(qOthersHigh, qACK)
    match proto tcp from any to any port = 1863 label "USER_RULE: m_Other MSN1 outbound" queue(qOthersLow, qACK)
    match proto tcp from any to any port 6890 >< 6901 label "USER_RULE: m_Other MSN2 outbound" queue(qOthersLow, qACK)
    match proto tcp from any to any port = 6901 label "USER_RULE: m_Other MSN3 outbound" queue(qOthersLow, qACK)
    match proto udp from any to any port = 6901 label "USER_RULE: m_Other MSN4 outbound" queue qOthersLow
    match proto tcp from any to any port = pptp label "USER_RULE: m_Other PPTP outbound" queue(qOthersHigh, qACK)
    match proto gre all label "USER_RULE: m_Other PPTPGRE outbound" queue qOthersHigh
    pass in quick on rl0 reply-to (rl0 192.168.1.254) inet proto udp from any to 192.168.1.2 keep state label "USER_RULE: OpenVPN wizard"
    pass in quick on rl0 reply-to (rl0 192.168.1.254) inet proto tcp from any to 192.168.0.1 port = smtp flags S/SA keep state label "USER_RULE: NAT Forward posta a turing"
    pass in quick on rl0 reply-to (rl0 192.168.1.254) inet proto tcp from any to 192.168.0.1 port = http flags S/SA keep state label "USER_RULE: NAT Forward web a turing"
    pass in quick on rl0 reply-to (rl0 192.168.1.254) inet proto tcp from any to 192.168.0.1 port = ssh flags S/SA keep state label "USER_RULE: NAT Forward ssh nascosta a turing"
    pass in quick on rl0 reply-to (rl0 192.168.1.254) inet proto tcp from any to 192.168.0.1 port = ftp flags S/SA keep state label "USER_RULE: NAT Forward FTP a turing"
    pass in quick on rl0 reply-to (rl0 192.168.1.254) inet proto tcp from any to 192.168.0.1 port 30099 >< 30111 flags S/SA keep state label "USER_RULE: NAT Forward FTP passivo a turing"
    pass in quick on rl0 reply-to (rl0 192.168.1.254) inet proto tcp from any to 192.168.0.1 port = pop3s flags S/SA keep state label "USER_RULE: NAT Forward pop3s a turing"
    pass in quick on et1 inet from 192.168.0.0/24 to any flags S/SA keep state label "USER_RULE: Default LAN -> any"
    pass in quick on openvpn all flags S/SA keep state label "USER_RULE: OpenVPN wizard"
    pass in quick on pptp all flags S/SA keep state label "USER_RULE"
    anchor "tftp-proxy/
    " all
    anchor "miniupnpd" all</bogons></virusprot></webconfiguratorlockout></sshlockout></pfsnortsamin></pfsnortsamout></snort2c></snort2c>



  • You upgraded to latest snapshot?
    Do you have any proxy running on pfSense?

    Please show me the output of  cat /tmp/rules.debug not pfctl -sr.



  • I can't upgrade until friday, because the system is in production. No proxy (AFAIK). Here it is the output of cat /tmp/rules.debug:

    #System aliases

    loopback = "{ lo0 }"
    LAN = "{ et1 }"
    WAN = "{ rl0 }"
    pptp = "{ pptp }"
    OpenVPN = "{ openvpn }"

    #SSH Lockout Table
    table <sshlockout>persist
    table <webconfiguratorlockout>persist
    #pfSnortSam tables
    table <snort2c>table <pfsnortsamout>table <pfsnortsamin>table <virusprot># User Aliases
    table <vpnnet>{   10.8.0.0/24 }
    VPNNet = "<vpnnet>"

    Gateways

    GWalice = " route-to ( rl0 192.168.1.254 ) "

    set loginterface et1
    set loginterface rl0
    set optimization normal
    set limit states 95000
    set limit src-nodes 95000

    set skip on pfsync0

    scrub in on $LAN all    fragment reassemble
    scrub in on $WAN all    fragment reassemble

    altq on  rl0 hfsc bandwidth 960Kb queue {  qACK,  qDefault,  qVoIP,  qOthersHigh,  qOthersLow  }
    queue qACK on rl0 bandwidth 16% hfsc (  ecn  , linkshare 16%  )
    queue qDefault on rl0 bandwidth 8% hfsc (  ecn  , default  )
    queue qVoIP on rl0 bandwidth 64Kb hfsc (  ecn  ,  realtime 256Kb )
    queue qOthersHigh on rl0 bandwidth 8% hfsc (  ecn  , linkshare 8%  )
    queue qOthersLow on rl0 bandwidth 4% hfsc (  ecn  , linkshare 4%  )

    nat-anchor "natearly/"
    nat-anchor "natrules/
    "

    Outbound NAT rules

    nat on $WAN  from 10.8.0.0/24 to any -> 192.168.1.2/32 port 1024:65535
    nat on $WAN  from 192.168.0.0/24 to any -> 192.168.1.2/32 port 1024:65535

    Load balancing anchor

    rdr-anchor "relayd/*"

    TFTP proxy

    rdr-anchor "tftp-proxy/*"
    table <direct_networks>{ 192.168.0.0/24 192.168.1.0/24 10.8.0.240/32 }

    NAT Inbound Redirects

    rdr on rl0 proto tcp from any to 192.168.1.2 port 25 -> 192.168.0.1

    Reflection redirects

    rdr on { et1 pptp openvpn } proto tcp from any to 192.168.1.2 port 25 tag PFREFLECT -> 127.0.0.1 port 19000

    rdr on rl0 proto tcp from any to 192.168.1.2 port 80 -> 192.168.0.1
    rdr on rl0 proto tcp from any to 192.168.1.2 port 122 -> 192.168.0.1 port 22
    rdr on rl0 proto tcp from any to 192.168.1.2 port 21 -> 192.168.0.1
    rdr on rl0 proto tcp from any to 192.168.1.2 port 30100:30110 -> 192.168.0.1
    rdr on rl0 proto tcp from any to 192.168.1.2 port 995 -> 192.168.0.1

    Reflection redirects

    rdr on { et1 pptp openvpn } proto tcp from any to 192.168.1.2 port 995 tag PFREFLECT -> 127.0.0.1 port 19001

    UPnPd rdr anchor

    rdr-anchor "miniupnpd"

    anchor "relayd/*"
    #–-------------------------------------------------------------------------

    default deny rules

    #---------------------------------------------------------------------------
    block in log all label "Default deny rule"
    block out log all label "Default deny rule"

    We use the mighty pf, we cannot be fooled.

    block quick proto { tcp, udp } from any port = 0 to any
    block quick proto { tcp, udp } from any to any port = 0

    Block all IPv6

    block in quick inet6 all
    block out quick inet6 all

    pfSnortSam

    block quick from <snort2c>to any label "Block snort2c hosts"
    block quick from any to <snort2c>label "Block snort2c hosts"
    block quick from <pfsnortsamout>to any label "Block pfSnortSamOut hosts"
    block quick from any to <pfsnortsamin>label "Block pfSnortSamIn hosts"

    SSH lockout

    block in log quick proto tcp from <sshlockout>to any port 22 label "sshlockout"

    webConfigurator lockout

    block in log quick proto tcp from <webconfiguratorlockout>to any port 80 label "webConfiguratorlockout"
    block in quick from <virusprot>to any label "virusprot overload table"
    antispoof for et1

    allow access to DHCP server on LAN

    pass in on $LAN proto udp from any port = 68 to 255.255.255.255 port = 67 label "allow access to DHCP server"
    pass in on $LAN proto udp from any port = 68 to 192.168.0.254 port = 67 label "allow access to DHCP server"
    pass out on $LAN proto udp from 192.168.0.254 port = 67 to any port = 68 label "allow access to DHCP server"
    table <bogons>persist file "/etc/bogons"

    block bogon networks

    http://www.cymru.com/Documents/bogon-bn-nonagg.txt

    block in log quick on $WAN from <bogons>to any label "block bogon networks from WAN"
    antispoof for rl0

    loopback

    pass in on $loopback all label "pass loopback"
    pass out on $loopback all label "pass loopback"

    let out anything from the firewall host itself and decrypted IPsec traffic

    pass out all keep state allow-opts label "let out anything from firewall host itself"
    pass out route-to ( rl0 192.168.1.254 ) from 192.168.1.2 to !192.168.1.0/24 keep state allow-opts label "let out anything from firewall host itself"

    make sure the user cannot lock himself out of the webConfigurator or SSH

    pass in quick on et1 proto tcp from any to (et1) port { 80 22 } keep state label "anti-lockout rule"

    PPTPd rules

    pass in on $WAN proto tcp from any to 192.168.1.2 port = 1723 modulate state label "allow pptpd 192.168.1.2"

    NAT Reflection rules

    pass in inet tagged PFREFLECT keep state label "NAT REFLECT: Allow traffic to localhost"

    User-defined rules follow

    match  proto udp  from   192.168.0.40 to any  queue (qVoIP)  label "USER_RULE: VOIP Adapter"
    match  proto udp  from   10.8.0.0/24 to   192.168.0.40  queue (qVoIP)  label "USER_RULE: VOIP Adapter (stenio)"
    match    proto tcp  from any to any port 3389   queue (qOthersHigh,qACK)  label "USER_RULE: m_Other MSRDP outbound"
    match    proto tcp  from any to any port 1863   queue (qOthersLow,qACK)  label "USER_RULE: m_Other MSN1 outbound"
    match    proto tcp  from any to any port 6890 >< 6901   queue (qOthersLow,qACK)  label "USER_RULE: m_Other MSN2 outbound"
    match    proto tcp  from any to any port 6901   queue (qOthersLow,qACK)  label "USER_RULE: m_Other MSN3 outbound"
    match    proto udp  from any to any port 6901   queue (qOthersLow)  label "USER_RULE: m_Other MSN4 outbound"
    match    proto tcp  from any to any port 1723   queue (qOthersHigh,qACK)  label "USER_RULE: m_Other PPTP outbound"
    match    proto gre  from any to any  queue (qOthersHigh)  label "USER_RULE: m_Other PPTPGRE outbound"
    pass  in  quick  on $WAN reply-to ( rl0 192.168.1.254 )  proto UDP  from any to 192.168.1.2 keep state  label "USER_RULE: OpenVPN  wizard"
    pass   in  quick  on $WAN reply-to ( rl0 192.168.1.254 )  proto tcp  from any to   192.168.0.1 port 25   label "USER_RULE: NAT Forward posta a turing"
    pass   in  quick  on $WAN reply-to ( rl0 192.168.1.254 )  proto tcp  from any to   192.168.0.1 port 80   label "USER_RULE: NAT Forward web a turing"
    pass   in  quick  on $WAN reply-to ( rl0 192.168.1.254 )  proto tcp  from any to   192.168.0.1 port 22   label "USER_RULE: NAT Forward ssh nascosta a turing"
    pass   in  quick  on $WAN reply-to ( rl0 192.168.1.254 )  proto tcp  from any to   192.168.0.1 port 21   label "USER_RULE: NAT Forward FTP a turing"
    pass   in  quick  on $WAN reply-to ( rl0 192.168.1.254 )  proto tcp  from any to   192.168.0.1 port 30099 >< 30111   label "USER_RULE: NAT Forward FTP passivo a turing"
    pass   in  quick  on $WAN reply-to ( rl0 192.168.1.254 )  proto tcp  from any to   192.168.0.1 port 995   label "USER_RULE: NAT Forward pop3s a turing"
    pass  in  quick  on $LAN  from 192.168.0.0/24 to any keep state  label "USER_RULE: Default LAN -> any"
    pass  in  quick  on $OpenVPN  from any to any keep state  label "USER_RULE: OpenVPN  wizard"
    pass  in  quick  on $pptp  from any to any keep state  label "USER_RULE"

    VPN Rules

    anchor "tftp-proxy/*"

    uPnPd

    anchor "miniupnpd"

    Regards,
    Stenio</bogons></bogons></virusprot></webconfiguratorlockout></sshlockout></pfsnortsamin></pfsnortsamout></snort2c></snort2c></direct_networks></vpnnet></vpnnet></virusprot></pfsnortsamin></pfsnortsamout></snort2c></webconfiguratorlockout></sshlockout>



  • @ermal:

    You upgraded to latest snapshot?
    Do you have any proxy running on pfSense?

    Please show me the output of  cat /tmp/rules.debug not pfctl -sr.

    Upgraded to the lastet snapshot. Same result.  :(

    I think I've found the problem: with tcpdump I see data passing on ovpns1 interface with the voip gateway ip, but on the wan interface the traffic comes from wan ip.

    Should I need a rule to use the queue on the openvpn interface?

    Regards,
    Stenio



  • @stenio:

    I think I've found the problem: with tcpdump I see data passing on ovpns1 interface with the voip gateway ip, but on the wan interface the traffic comes from wan ip.

    Should I need a rule to use the queue on the openvpn interface?

    Just tried: same results.  :(



  • Just go to the by Queue view and duplicate the queues of WAN to the Openvpn interface(YES you need to assign the Openvpn interface) and either change the scheduler to PRIQ or make the bandwidth adjustments, if any.



  • @ermal:

    Just go to the by Queue view and duplicate the queues of WAN to the Openvpn interface(YES you need to assign the Openvpn interface) and either change the scheduler to PRIQ or make the bandwidth adjustments, if any.

    Hi, I tried but than the clients cannot "see" the LAN. They can connect to the server, but not to the voip gateway.
    The strange thing is that the shaping works perfectly with PPTP.  :-\



  • Hi Stenio,

    thanks for pointing me to your thread and Ermal's reply. The issue remains for ipsec though, I wish I could get some help with that too since you cannot assign an interface to ipsec tunnels the same as for OpenVPN.
    http://forum.pfsense.org/index.php/topic,34132.0.html

    I just tried it (OpenVPN) and had the same effect as you. You need to configure the openVPN interface as type "none".
    It didn't work for me after that, the tunnel was up but I couldn't reach anything from the OpenVPN Client's side.
    I then re established the vpn from the client side: still no luck.

    To get it to work I simply had to restart the openVPN server on the pfsense box ;)
    Did you try that already?



  • After you create the OpenVPN interfaces(Enabled and set as none) Go to VPN:OpenVPN and resave your tunnel setups. This should work, if not, reboot your box… Also, create allow rules under Firewall:Rules then select the Tab of your new OpenVPN interface name.

    I'm not sure what the default OpenVPN Firewall:Rules tab is used for after you create the interfaces... I disabled my default allow all rule on that tab for now. Waiting to hear back on a topic i started last night http://forum.pfsense.org/index.php/topic,34201.msg177412.html#msg177412



  • @mxx:

    To get it to work I simply had to restart the openVPN server on the pfsense box ;)
    Did you try that already?

    Hi Mmx,

    You are right! Now it works perfectly!  :)

    Thanks,
    Stenio



  • @Cino:

    After you create the OpenVPN interfaces(Enabled and set as none) Go to VPN:OpenVPN and resave your tunnel setups. This should work, if not, reboot your box… Also, create allow rules under Firewall:Rules then select the Tab of your new OpenVPN interface name.

    Thank you Cino, restarting openvpn server worked for me.

    Regards,
    Stenio



  • @ermal:

    Just go to the by Queue view and duplicate the queues of WAN to the Openvpn interface(YES you need to assign the Openvpn interface) and either change the scheduler to PRIQ or make the bandwidth adjustments, if any.

    Hi Ermal,

    I think that the second queue is not working properly. Please, see the image attached. It seems that voip traffic is shaped properly in the new queue OPENVPNQUEUE, but that it goes into the default WAN queue qDefault. Shouldn't it go to queue qVoIP instead?

    Regards,
    Stenio




  • Hi Stenio,

    I'm still trying to get the shaper working for ipsec.
    Regarding the solution for OpenVPN (with a dedicated interface + shaper), I don't really understand how this should work effectively..
    Ok, that way you can shape inside the OpenVPN tunnel, but what about the WAN interface the tunnel is using?
    How much bandwidth should one assign to the openvpn shaper "interface"?Whole wan upstream?Half of it? :D
    What about other queues/services that use the WAN interface fighting for bandwidth?

    I don't know if I understand everything right, but I think that doing it this way one can only assign a rather fixed priority (queue) for the whole openvpn tunnel on the WAN interface and then decide how this bandwidth is distributed inside the tunnel.
    I think that this way it's not possible to get the shaper/queues of the WAN interface to dynamically adjust and reserve more bandwidth depending on a particular service INSIDE the tunnel (something suddenly using a high priority queue of the openvpn interface).
    Maybe that's what you are seeing.
    Openvpn traffic as a whole will always go into the same queue on the WAN interface no matter what, but you can decide on what services inside the tunnel get how much of the cake..

    Please correct me if I'm wrong, that's just my guess..



  • Hi Mxx,

    I have the same uncertainties of yours. I think that the bandwith assigned to the tunnel should be at least the same as the bandwidth assigned to the wan. Moreover, the packets should be handled by the wan queues in the way specified by the tunnel queues. But this is only my opinion. I don't know if it this the correct way to do it.

    I don't know if I understand everything right, but I think that doing it this way one can only assign a rather fixed priority (queue) for the whole openvpn tunnel on the WAN interface and then decide how this bandwidth is distributed inside the tunnel.

    I belive that you are right. From what I'm seeing we can only assign a fixed priority (bandwith) to the tunnel, but I'm wiiling to know if that depends by a wrong configuration or by a limitation of the system.

    Regards,
    Stenio

    P.S.: Sorry for my English.



  • Haha sorry for my English too ;)

    I don't think that this is possible with a dedicated interface for OpenVPN, but I hope I'm wrong.

    I was thinking about trying that by using queues on the Lan interface… that way I could imagine that it's possible to achieve this.

    I understand too little about all this.. for example I don't understand why in examples (IF you actually have Lan queues which aren't generated anymore automatically by the wizards as they are only needed in some scenarios) the Lan root queue gets the value for the wan if's downstream assigned when there are also queues (as children of "Lan") where upstream traffic flows into..
    From my little "logic" I would split up and downstream queues on the lan interface by using parent queues with these respective limits..

    I was unsuccessful in shaping ipsec traffic by just using queues on wan.
    In my thread about ipsec shaping I now posted screenshots to show what I've tried which was unsuccessful and I've asked this question about using Lan queues for that and provided an example..
    http://forum.pfsense.org/index.php/topic,34132.msg178371.html#msg178371



  • Maybe this topic should be moved to the Traffic Shaping forum.


Log in to reply