MLPPP performance? (traffic through router slow) [NOT SOLVED] (Bounty?$$)



  • Using the latest release 2.2.6

    After reading a few posts about trouble when the same gateway was used, I simplified my setup, but the same issue remains. I had single link ppp connections configured on the underlying interfaces to allow me backup ways of connecting to manage the router while I messed with my config. If this was the issue, I've removed all that. I now have a single wan, running MLPPP and it connects and routes traffic. No trouble right? But no… performance is horrible.

    If I test using iperf, TCP or UDP, I get the effective upload and download of both connections I am linking with MLPPP.

    BUT, If I test with say an FTP session behind the router, through it's NAT, I get the effective download of a single connection, and the upload is sporatic - it varies from 10% to 50% of a single connection.

    Is this a known issue? Or is there something dumb I might be missing?

    Thanks!

    Just updated the subject to clarify I am having an issue. Thanks :-)



  • Adding additional background. The ISP uses cisco gear. They support two modes for MLPPP - one they call "Layer2" the other "Layer3". It's a Telus wholesale so I'm not actually sure where the auth and bonding parts are. I DO have some sample config scripts. But what they show and what pfsense does under the covers don't seem similar.

    This config is for a Cisco 871k9 - what is significant to me is…

    1. the way I set up seems to work UNTIL I try to route through the system.
    2. this set up has a different login per connection which is then bundled together.
    3. no idea how I would do that in pfsense?

    N.B. to mask parts of the config, I used:
    III to replace part of an IP address
    PPP# to replace a password
    UUU# to replace a username

    ! at the command prompt, create these vlans
    vlan database
    vlan 2 name ADSL2
    vlan 3 name ADSL3
    vlan 4 name ADSL4
    exit
    ! go into the config prompt to program these lines into the k9
    config t
    service tcp-keepalives-in
    service tcp-keepalives-out
    service timestamps debug datetime localtime
    service timestamps log datetime localtime
    no service password-encryption
    !
    !creating a command prompt or site name by replacing 970-MLPPP-K9 to new site name
    hostname 970-MLPPP-k9
    !
    boot-start-marker
    boot-end-marker
    !
    enable password cisco
    !
    no aaa new-model
    !
    resource policy
    !
    ip cef
    !
    ! creating the 4 ADSL bonding tunnels
    !
    interface Tunnel10012
     ip address III.III.0.46 255.255.255.252
     ip load-sharing per-packet
     ip ospf mtu-ignore
     keepalive 5 3
     tunnel source Dialer1
     tunnel destination III.III.86.108
    !
    interface Tunnel20012
     ip address III.III.0.46 255.255.255.252
     ip load-sharing per-packet
     ip ospf mtu-ignore
     keepalive 5 3
     tunnel source Dialer2
     tunnel destination III.III.86.109
    !
    interface Tunnel30012
     ip address III.III.0.46 255.255.255.252
     ip load-sharing per-packet
     ip ospf mtu-ignore
     keepalive 5 3
     tunnel source Dialer3
     tunnel destination III.III.86.110
    !
    interface Tunnel40012
     ip address III.III.0.46 255.255.255.252
     ip load-sharing per-packet
     ip ospf mtu-ignore
     keepalive 5 3
     tunnel source Dialer4
     tunnel destination III.III.86.106
    !
    !
    ! setting up the k9 Ethernet ports and assign them to the vlans
    interface FastEthernet0
     description ADSL2
     switchport access vlan 2
     no shut
    !
    interface FastEthernet1
     description ADSL3
     switchport access vlan 3
     no shut
    !
    interface FastEthernet2
     description ADSL4
     switchport access vlan 4
     no shut
    !
    interface FastEthernet3
     description LAN
     no shut
    !
    interface FastEthernet4
     description ADSL1
     no ip address
     ip virtual-reassembly
     duplex auto
     speed auto
     pppoe enable group global
     pppoe-client dial-pool-number 1
     no shut
    !
    interface Vlan1
    description LAN
     ip address III.III.7.129 255.255.255.248
     ip tcp adjust-mss 1380
     no shut
    !
    interface Vlan2
     description ADSL2
     no ip address
     ip virtual-reassembly
     pppoe enable group global
     pppoe-client dial-pool-number 2
     no shut
    !
    interface Vlan3
     description ADSL3
     no ip address
     ip virtual-reassembly
     pppoe enable group global
     pppoe-client dial-pool-number 3
     no shut
    !
    interface Vlan4
     description ADSL4
     no ip address
     ip virtual-reassembly
     pppoe enable group global
     pppoe-client dial-pool-number 4
     no shut
    !
    interface Dialer1
     ip address negotiated
     ip mtu 1460
     ip virtual-reassembly
     encapsulation ppp
     dialer pool 1
     dialer-group 1
     ppp chap hostname UUU1@pppoe.net
     ppp chap password 0 PPP1 
     ppp authentication pap callin
     ppp pap sent-username UUU1@pppoe.net password 0 PPP1
    !
    interface Dialer2
     ip address negotiated
     ip mtu 1460
     ip virtual-reassembly
     encapsulation ppp
     dialer pool 2
     dialer-group 2
     ppp chap hostname UUU2@pppoe.net
     ppp chap password 0 PPP2
     ppp authentication pap callin
     ppp pap sent-username UUU2@pppoe.net password 0 PPP2
    !
    interface Dialer3
     ip address negotiated
     ip mtu 1460
     ip virtual-reassembly
     encapsulation ppp
     dialer pool 3
     dialer-group 3
     ppp chap hostname UUU3@pppoe.net
     ppp chap password 0 PPP3
     ppp authentication pap callin
     ppp pap sent-username UUU3@pppoe.net password 0 PPP4
    !
    interface Dialer4
     ip address negotiated
     ip mtu 1460
     ip virtual-reassembly
     encapsulation ppp
     dialer pool 4
     dialer-group 4
     ppp chap hostname UUU4@pppoe.net
     ppp chap password 0 PPP4
     ppp authentication pap callin
     ppp pap sent-username UUU4@pppoe.net password 0 PPP4
    !
    router ospf 100
     log-adjacency-changes
     redistribute connected subnets route-map RDSTB-CON-TO-OSPF
     redistribute static subnets route-map RDSTB-CON-TO-OSPF
     network 10.90.0.0 0.0.255.255 area 0
     network 10.91.0.0 0.0.255.255 area 0
     network 10.92.0.0 0.0.255.255 area 0
     network 10.93.0.0 0.0.255.255 area 0
     network 10.94.0.0 0.0.255.255 area 0
     distribute-list route-map OSPF-IN in
    !
    ip route 0.0.0.0 0.0.0.0 Dialer1 250
    ip route III.III.86.106 255.255.255.255 Dialer4
    ip route III.III.86.108 255.255.255.255 Dialer1
    ip route III.III.86.109 255.255.255.255 Dialer2
    ip route III.III.86.110 255.255.255.255 Dialer3
    !
    !
    no ip http server
    no ip http secure-server
    !
    !
    ip prefix-list OSPF-IN seq 5 permit 0.0.0.0/0
    ip prefix-list OSPF-IN seq 10 deny 0.0.0.0/0 le 32
    !
    ip prefix-list RDSTB-CON seq 4 permit 65.110.7.128/29
    ip prefix-list RDSTB-CON seq 10 deny 0.0.0.0/0 le 32
    !
    !
    snmp-server community cisco-k9 RO
    snmp-server trap-source Vlan1
    !
    route-map OSPF-IN permit 5
     match ip address prefix-list OSPF-IN
    !
    route-map OSPF-IN deny 50
    !
    route-map RDSTB-CON-TO-OSPF permit 10
     match ip address prefix-list RDSTB-CON
    !
    route-map RDSTB-CON-TO-OSPF deny 100
    !
    !
    control-plane
    !
    !
    line con 0
     no modem enable
     transport preferred none
    line aux 0
    ! creating a telnet login with default password cisco
    line vty 0 4
     password k9pass
     login
     transport preferred none
    !
    scheduler max-task-time 5000
    !
    webvpn context Default_context
     ssl authenticate verify all
     !
     no inservice
    !
    ! exit back to the command prompt after programmed the k9
    end
    write mem
    
    

    When I look at the config pfsense generates, it seems to use the same usernames and passwords for all connections.

    I can get online, but for some reason traffic nat'd through the pfsense either drops packets or ? and the performance goes to poop…

    Does this config file provide any insight how I might have to adjust my config? Will keep digging.

    Thanks!



  • The RAW services are matching 6Mb down / 1Mb up DSL - the sync rates are about 7Mb down, and 0.95Mb up. Latency is similar.

    I tried editing the config and restarting the ppp connection manually to see if that had any effect - doesn't seem to.

    Here's an example to clarify the speed difference I am seeing… This FTP was done on the pfsense box to a file located in /tmp (10MB)

    ftp> get test.tmp
    local: test.tmp remote: test.tmp
    229 Entering Extended Passive Mode (|||32549|)
    150 Opening BINARY mode data connection for test.tmp (10485760 bytes)
    100% |***********************************| 10240 KiB  601.40 KiB/s    00:00 ETA
    226 Transfer complete.
    10485760 bytes received in 00:17 (599.71 KiB/s)
    ftp> put test.tmp
    local: test.tmp remote: test.tmp
    229 Entering Extended Passive Mode (|||33341|)
    150 Opening BINARY mode data connection for test.tmp
    100% |***********************************| 10240 KiB  180.28 KiB/s    00:00 ETA
    226 Transfer complete.
    10485760 bytes sent in 00:57 (179.16 KiB/s)
    ftp> 
    

    The upload demonstrates the benefits of the bonding in this case, but the download does not… the download is like the download of an unbonded service...

    Here, I ran an iperf3 test - both directions behave like bonding is working:

    # /usr/local/bin/iperf3 -4 -p 5201 -fm --client 65.110.9.139 --interval 1 --time 120
    Connecting to host 65.110.9.139, port 5201
    [  4] local 66.196.39.68 port 62255 connected to 65.110.9.139 port 5201
    [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
    [  4]   0.00-1.00   sec   831 KBytes  6.81 Mbits/sec    0   75.7 MBytes
    {{SNIP}}
    [  4] 119.00-120.00 sec  1.31 MBytes  11.0 Mbits/sec    0    153 MBytes
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bandwidth       Retr
    [  4]   0.00-120.00 sec   153 MBytes  10.7 Mbits/sec   48             sender
    [  4]   0.00-120.00 sec   153 MBytes  10.7 Mbits/sec                  receiver
    
    iperf Done.
    # /usr/local/bin/iperf3 -4 -p 5201 -fm --client 65.110.9.139 --interval 1 --reverse --time 120
    Connecting to host 65.110.9.139, port 5201
    Reverse mode, remote host 65.110.9.139 is sending
    [  4] local 66.196.39.68 port 33740 connected to 65.110.9.139 port 5201
    [ ID] Interval           Transfer     Bandwidth
    [  4]   0.00-1.00   sec   179 KBytes  1.46 Mbits/sec
    {{SNIP}}
    [  4] 119.00-120.00 sec   179 KBytes  1.46 Mbits/sec
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bandwidth       Retr
    [  4]   0.00-120.00 sec  21.0 MBytes  1.47 Mbits/sec   14             sender
    [  4]   0.00-120.00 sec  21.0 MBytes  1.47 Mbits/sec                  receiver
    
    iperf Done.
    
    

    Where it gets really bad though is if I try to work inside the firewall…

    The downstream (FTP for example) never gets over 600KiB/s and the upstream averages around 45KiB/s

    :(

    I'm sure it's got to do something with how I'm configured. The cisco config seems to require OSPF config - is there something like that I should be doing on pfsense?

    Thanks!



  • There is also some support / recommendations for an RB750 config:

    !!!setup PPPoE username!!!
    /interface pppoe-client add name=MLPPP user=UUU@pppoe.net  password=PPP interface=ether1,ether2,ether3,ether4 add-default-route=yes use-peer-dns=yes allow=chap disabled=no
    !!!setup Lan Port!!!
    /ip address add address=III.110.7.129/29 255.255.255.248 secondary interface=ether5 disabled=no
    /ip address add address=III.110.7.129/29 255.255.255.255 secondary interface=ether5 disabled=no
    /ip address add address=routedIP interface=ether5 disabled=no
    /ip address add address=192.168.1.1/24  interface=ether5 disabled=no
    !!!setup NAT!!!
    /ip firewall nat add chain=srcnat src-address=192.168.1.0/24 out-interface=MLPPP action=masquerade disabled=no
    !!! setup Password !!!
    /password
    k9pass
    k9pass
    !!! setup SNMP !!!
    /snmp community add name=k9pass read-access=yes
    
    

    I don't think the lines above containing "address=III.110.7.129/29" make any sense as the CIDR mask is wrong - but that's what was in the example. Seems to be adding an alias to the LAN for some reason?

    But none of this makes any sense as to why the non-natted WAN traffic seems to work for the most part - and only routed / natted traffic seems to be slow?

    I can't think of anything to try - hopefully someone sees my mistake / error?

    Thanks!

    Mitch



  • Willing to offer a bounty if someone can help resolve. And of course am willing / happy to have said solution / notes posted for all to see.



  • there are very few people running/have knowledge about mlppp.
    also: you run an unsupported version that is no longer used by the vast majority of the community…

    (i know nothing about mlppp)

    1. update to a current release, see if the problem still exists
    2. bounty if it isn't urgent / pfSense commercial support if it is urgent


  • Weird - the unit says it is on the latest release… not sure why it isn't updating.
    Sorry about that - I thought it was running the current stable!
    I just checked again - the auto update url is set, and it checks and says it's:

    Downloading new version information...done
    Obtaining current version information...done
    
    You are on the latest version.
    

    Ok - now I see what happened… this unit has a 1GB CF. I guess when the updates switched to 2GB instead of reporting that it could NOT do an upgrade, it started reporting that it was already current. I'll look at a "forklift" upgrade to a new CF.

    In my defense, my first port included the version number - which I stupidly thought was current:
    Selfquoting "Using the latest release 2.2.6"  :'(

    I'll try a current release.  :-X

    Thanks!
    m