Unable to change PPPOE MTU value.



  • Hi everyone,

    I'm trying to get rid of my ISP Box and to connect the GPON module directly to the Pfsense Box (A1SRi-2758F MB).
    I've done some download speed test and i've better results with the ISP Modem/Router connected to Pfsense, approximatively 240 Mbps.
    While Pfsense connected directly to the GPON directly provide me a bandwith limit of 170 Mbps.

    I've checked the forum and find some interesting hint, especially this one witch point out an MTU problem.
    https://forum.pfsense.org/index.php?topic=60139.0

    I think i'm suffering the same issue but unfortunatelly i was not able to resolve it.
    I'm sure my ISP request a MTU of 1492, and by default PPPOE interface is created with MTU set to 1492.
    I've checked the /var/etc/mpd_opt5.conf file and the MTU is correctly set to 1492.

    But on the status_interfaces page (and ifconfig) show a MTU of 1456.
    I've tried to force on the WebGui the MTU of the PPPOE interface to 1492, but the same MTU is still 1456.

    i've been able to change it with command :
    ifconfig pppoe0 down; ifconfig pppoe0 mtu 1492; ifconfig pppoe0 up
    But i'm not sure it has really change a thing as test perf provide the same bandwith and any interface refresh get it back to 1456.

    Any help/insight is appreciated.

    Thanks,



  • An other insight, when i'm downloading a large file i've a kernel igb2 queue to 100%
    Can that be my problem ?

    last pid: 89466;  load averages:  3.82,  3.35,  2.34                                                                                    up 1+00:38:29  00:41:46
    278 processes: 13 running, 203 sleeping, 62 waiting
    CPU:  3.2% user,  0.0% nice, 23.1% system,  0.2% interrupt, 73.4% idle
    Mem: 55M Active, 241M Inact, 822M Wired, 3204K Cache, 1009M Buf, 6760M Free
    Swap: 16G Total, 16G Free

    PID USERNAME  PRI NICE  SIZE    RES STATE  C  TIME    WCPU COMMAND
        0 root      -92    0    0K  688K CPU3    3  14:56 100.00% [kernel{igb2 que}]
      11 root      155 ki31    0K  128K CPU0    0  24.4H  96.97% [idle{idle: cpu0}]
      11 root      155 ki31    0K  128K CPU4    4  24.3H  90.67% [idle{idle: cpu4}]
      11 root      155 ki31    0K  128K RUN    1  24.3H  81.59% [idle{idle: cpu1}]
      11 root      155 ki31    0K  128K CPU2    2  24.3H  72.85% [idle{idle: cpu2}]
    70868 root        93    0 21100K  4752K CPU6    6  0:48  72.07% fetch: /dev/null [16% of  10 GB] (fetch)
      11 root      155 ki31    0K  128K CPU5    5  24.3H  69.19% [idle{idle: cpu5}]
      11 root      155 ki31    0K  128K RUN    3  24.4H  68.16% [idle{idle: cpu3}]
      11 root      155 ki31    0K  128K CPU6    6  24.3H  61.08% [idle{idle: cpu6}]
      11 root      155 ki31    0K  128K CPU7    7  24.4H  60.89% [idle{idle: cpu7}]
    89233 root        32    0  223M 34632K accept  7  0:00  1.27% php-fpm: pool lighty (php-fpm)



  • If my memory serves me: PPPoE will only use a single queue = cpu core on pfSense (freeBSD thing, maybe fixed in 2.3?) and changing the MTU to support 'baby jumbo frames' has some sort of an issue. Can't remember if fixed in 2.2.5 or 2.3.





  • @fragged:

    If my memory serves me: PPPoE will only use a single queue = cpu core on pfSense (freeBSD thing, maybe fixed in 2.3?) and changing the MTU to support 'baby jumbo frames' has some sort of an issue. Can't remember if fixed in 2.2.5 or 2.3.

    The single queue issue is explained in the 'Traffic Flow' section of https://wiki.freebsd.org/NetworkPerformanceTuning - it's down to the failure of flowid to resolve to other than 0 on Ethernet frames other than IPv4 and IPv6. There is a patch there for igb, which has already been mentioned on the pfSense Redmine bug for this issue (Redmine #4821). If there is no FreeBSD PR for this problem, IMHO there should be.

    Forcing the MTU of pppoe0 is the wrong way to fix the original poster's unexpectedly low MTU. Look in the PPP log (Status -> System Logs, PPP tab) and look at the LCP layer. For example, on my box (which has a version of the RFC 4638 patch running, so I can negotiate an MTU greater than 1492), I see:

    Oct 13 14:07:23 ppp: [wan_link0] LCP: Up event
    Oct 13 14:07:23 ppp: [wan_link0] LCP: state change Starting –> Req-Sent
    Oct 13 14:07:23 ppp: [wan_link0] LCP: SendConfigReq #1
    Oct 13 14:07:23 ppp: [wan_link0] MRU 1500
    Oct 13 14:07:23 ppp: [wan_link0] MAGICNUM c47ae694

    Oct 13 14:07:23 ppp: [wan_link0] LCP: rec'd Configure Request #98 (Req-Sent)
    Oct 13 14:07:23 ppp: [wan_link0] MRU 1500
    Oct 13 14:07:23 ppp: [wan_link0] AUTHPROTO CHAP MD5
    Oct 13 14:07:23 ppp: [wan_link0] MAGICNUM 4b7a6008

    Oct 13 14:07:23 ppp: [wan_link0] LCP: SendConfigAck #98
    Oct 13 14:07:23 ppp: [wan_link0] MRU 1500
    Oct 13 14:07:23 ppp: [wan_link0] AUTHPROTO CHAP MD5
    Oct 13 14:07:23 ppp: [wan_link0] MAGICNUM 4b7a6008
    Oct 13 14:07:23 ppp: [wan_link0] LCP: state change Req-Sent –> Ack-Sent

    Oct 13 14:07:23 ppp: [wan_link0] LCP: rec'd Configure Ack #1 (Ack-Sent)
    Oct 13 14:07:23 ppp: [wan_link0] MRU 1500
    Oct 13 14:07:23 ppp: [wan_link0] MAGICNUM c47ae694
    Oct 13 14:07:23 ppp: [wan_link0] LCP: state change Ack-Sent –> Opened

    I've split this into four chunks for clarity, as it happens in four phases:

    • My end requests the remote end to set its MRU (i.e. my MTU) to 1500

    • The remote end requests my end to set its MRU (i.e. the remote MTU) to 1500 and asks for CHAP or MD5 authentication

    • My end acknowledges (confirms) the remote end's request to my MTU to be 1500 and send CHAP or MD5 authentication

    • The remote end acknowledges (confirms) my end's request for the remote MTU to be 1500

    You may see "Nak", which is a rejection of a request.

    RFC 4638 client support, which permits an MTU > 1492, is hopefully coming to pfSense 2.3 - it being tracked in Redmine #4542. The binary changes have been made in the latest 2.3-ALPHA snapshots and have made pull request pfsense/pfsense#1959 for the pfSense changes.

    If you want to live dangerously, follow the instructions on how to patch 2.2.4-RELEASE for RFC 4638 client support.

    If you use IPv6 on your PPPoE connection and need SLAAC, DHCPv6 and/or DHCP-PD support, two important issues are fixed in recent 2.2.5-DEVELOPMENT snapshots (Redmine #3670 and Redmine #5297). Ordinarily snapshots cannot be proposed for production use and snapshots are subject to random breakage, but 2.2.x is now a relatively stable branch that is feature frozen.



  • Thanks all for your feedback,

    I've open a PR to Freebsd for the one queue issue on igb dirver :
    https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203856

    I will wait for 2.3 to be out for RFC 4638 client support, which permits an MTU > 1492.
    Which i think is not my current limitation.

    Here is my PPPoE log, not clear why 1456 is used instead of 1492.

    15/10/2015 09:38 [opt5] IFACE: Rename interface ng0 to pppoe0
    15/10/2015 09:38 [opt5] IFACE: Up event
    15/10/2015 09:38 [opt5]  xxx.xxx.xxx.xxx -> xxxxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5] IPCP: LayerUp
    15/10/2015 09:38 [opt5] IPCP: state change Ack-Sent –> Opened
    15/10/2015 09:38 [opt5]  SECDNS xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5]  PRIDNS xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5]  IPADDR xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5] IPCP: rec'd Configure Ack #7 (Ack-Sent)
    15/10/2015 09:38 [opt5]  SECDNS xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5]  PRIDNS xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5]  IPADDR xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5] IPCP: SendConfigReq #7
    15/10/2015 09:38 [opt5]  SECDNS xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5]  PRIDNS xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5]    xxx.xxx.xxx.xxx is OK
    15/10/2015 09:38 [opt5]  IPADDR xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5] IPCP: rec'd Configure Nak #6 (Ack-Sent)
    15/10/2015 09:38 [opt5] IPV6CP: LayerFinish
    15/10/2015 09:38 [opt5] IPV6CP: state change Req-Sent –> Stopped
    15/10/2015 09:38 [opt5] IPV6CP: protocol was rejected by peer
    15/10/2015 09:38 [opt5_link0] LCP: protocol IPV6CP was rejected
    15/10/2015 09:38 [opt5_link0] LCP: rec'd Protocol Reject #110 (Opened)
    15/10/2015 09:38 [opt5]  SECDNS 0.0.0.0
    15/10/2015 09:38 [opt5]  PRIDNS 0.0.0.0
    15/10/2015 09:38 [opt5]  IPADDR 0.0.0.0
    15/10/2015 09:38 [opt5] IPCP: SendConfigReq #6
    15/10/2015 09:38 [opt5]  COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
    15/10/2015 09:38 [opt5] IPCP: rec'd Configure Reject #5 (Ack-Sent)
    15/10/2015 09:38 [opt5] IPCP: state change Req-Sent –> Ack-Sent
    15/10/2015 09:38 [opt5]  IPADDR xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5] IPCP: SendConfigAck #176
    15/10/2015 09:38 [opt5]    xxx.xxx.xxx.xxx is OK
    15/10/2015 09:38 [opt5]  IPADDR xxx.xxx.xxx.xxx
    15/10/2015 09:38 [opt5] IPCP: rec'd Configure Request #176 (Req-Sent)
    15/10/2015 09:38 [opt5] IPV6CP: SendConfigReq #2
    15/10/2015 09:38 [opt5] IPV6CP: state change Starting –> Req-Sent
    15/10/2015 09:38 [opt5] IPV6CP: Up event
    15/10/2015 09:38 [opt5]  SECDNS 0.0.0.0
    15/10/2015 09:38 [opt5]  PRIDNS 0.0.0.0
    15/10/2015 09:38 [opt5]  COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
    15/10/2015 09:38 [opt5]  IPADDR 0.0.0.0
    15/10/2015 09:38 [opt5] IPCP: SendConfigReq #5
    15/10/2015 09:38 [opt5] IPCP: state change Starting –> Req-Sent
    15/10/2015 09:38 [opt5] IPCP: Up event
    15/10/2015 09:38 [opt5] IPV6CP: LayerStart
    15/10/2015 09:38 [opt5] IPV6CP: state change Initial –> Starting
    15/10/2015 09:38 [opt5] IPV6CP: Open event
    15/10/2015 09:38 [opt5] IPCP: LayerStart
    15/10/2015 09:38 [opt5] IPCP: state change Initial –> Starting
    15/10/2015 09:38 [opt5] IPCP: Open event
    15/10/2015 09:38 [opt5] Bundle: Status update: up 1 link, total bandwidth 64000 bps
    15/10/2015 09:38 [opt5_link0] Link: Join bundle \opt5""
    15/10/2015 09:38 [opt5_link0] Link: Matched action 'bundle \opt5" ""'"
    15/10/2015 09:38 [opt5_link0] LCP: authorization successful
    15/10/2015 09:38 [opt5_link0]  MESG: CHAP authentication success, unit 44273
    15/10/2015 09:38 [opt5_link0] CHAP: rec'd SUCCESS #1 len: 43
    15/10/2015 09:38 [opt5_link0] CHAP: sending RESPONSE #1 len: 32
    15/10/2015 09:38 [opt5_link0] CHAP: Using authname \xxx.xxx.xxx.xxx""
    15/10/2015 09:38 [opt5_link0]  Name: \xxx.xxx.xxx.xxx""
    15/10/2015 09:38 [opt5_link0] CHAP: rec'd CHALLENGE #1 len: 29
    15/10/2015 09:38 [opt5_link0] LCP: LayerUp
    15/10/2015 09:38 [opt5_link0] LCP: auth: peer wants CHAP, I want nothing
    15/10/2015 09:38 [opt5_link0] LCP: state change Ack-Sent –> Opened
    15/10/2015 09:38 [opt5_link0]  MAGICNUM d7724313
    15/10/2015 09:38 [opt5_link0]  MRU 1492
    15/10/2015 09:38 [opt5_link0] LCP: rec'd Configure Ack #8 (Ack-Sent)
    15/10/2015 09:38 [opt5_link0] LCP: state change Opened –> Ack-Sent
    15/10/2015 09:38 [opt5_link0]  MAGICNUM 105069d0
    15/10/2015 09:38 [opt5_link0]  AUTHPROTO CHAP MD5
    15/10/2015 09:38 [opt5_link0]  MRU 1456
    15/10/2015 09:38 [opt5_link0] LCP: SendConfigAck #109
    15/10/2015 09:38 [opt5_link0]  MAGICNUM d7724313
    15/10/2015 09:38 [opt5_link0]  MRU 1492
    15/10/2015 09:38 [opt5_link0] LCP: SendConfigReq #8
    15/10/2015 09:38 [opt5_link0] LCP: LayerDown
    15/10/2015 09:38 [opt5_link0]  MAGICNUM 105069d0
    15/10/2015 09:38 [opt5_link0]  AUTHPROTO CHAP MD5
    15/10/2015 09:38 [opt5_link0]  MRU 1456
    15/10/2015 09:38 [opt5_link0] LCP: rec'd Configure Request #109 (Opened)



  • Your ISP asked for an MRU (your MTU) of 1456, which your end understandably granted. You asked for an MRU (your ISP's MTU) of 1492, which was granted.

    Only your ISP can explain why they ask for an MRU of 1456.


Log in to reply