RFC 4638 client support (PPPoE MTU > 1492) - testing
-
David, thank you for the work to put this together. I followed the instructions above and got it semi-working. The scripts didn't quite cut-n-paste from your message (it didn't really matter, but the backup of the existing files part didn't work). I managed to get the PPPoE to negotiate a payload of 1500 with my ISP (Snap, NZ) - see log below.
There were a few issues with the transition but a restart of pfSense sorted that out - I also incorrectly put the MTU on the PPPoE configuration page as a first try. I am using a modem connected to it's own NIC (no VLAN, it is a Realtek RTL8111E port on an APU board). Setting a MTU of 1500 on the NIC interface configuration seems a bit counter-intuitive (to me) when the interface MTU will be more than that.
Unfortunately that is as far as I can take it with my current ISP on my current plan. The larger frames didn't make it in or out. I confirmed that my ADSL2 plan doesn't support this with my ISP (and no other technology is available at my address).
If I could get a modem that could do PPPoA with conversion to PPPoE and supported PPPoE max payload I might be ok (an end-to-end PPPoE to my ISP is limited to 1492). I have both a Vigor 120v2 and a Cisco 877 as modems. Can anyone recommend workarounds/alternatives/configurations?
Log
[wan_link0] PPPoE: Set PPP-Max-Payload to '1500' [wan_link0] PPPoE: Connecting to '' PPPoE: rec'd ACNAME "SNAP-08" [wan_link0] PPPoE: rec'd PPP-Max-Payload '1500' [wan_link0] PPPoE: connection successful [wan_link0] Link: UP event [wan_link0] LCP: Up event [wan_link0] LCP: state change Starting --> Req-Sent [wan_link0] LCP: SendConfigReq #1 [wan_link0] PROTOCOMP [wan_link0] MRU 1500 [wan_link0] MAGICNUM 842207ef -
The scripts didn't quite cut-n-paste from your message (it didn't really matter, but the backup of the existing files part didn't work).
They should work, though they were minimally tested. The sh is crucial as the if… fi syntax is Korn shell, not C shell. The default shell on pfSense is tcsh, whereas my brain only works in Korn shell or bash.
I managed to get the PPPoE to negotiate a payload of 1500 with my ISP (Snap, NZ) - see log below.
Your partial log isn't conclusive over negotiating a 1500 byte MTU, as you truncated the log before the 'LCP: LayerUp' line.
The RFC 4638 PPP-Max-Payload tag was sent and received in the PPPoE discovery phase. In your system where the PPPoE goes no further than your PPPoE <-> PPPoA bridge in the DSL modem, this negotiation is between your pfSense box and the PPPoE endpoint in your modem, not pfSense and your ISP. It might be that the modem is naively including the PPP-Max-Payload tag in PADO and PADS when it has no support for RFC 4638, rather than following the stipulation in section 5 of RFC 2516 (the PPPoE RFC) to ignore unknown tags.
In the LCP phase, your end asked for a 1500 byte MRU, but the extract doesn't say whether that was ACKed or NAKed, also it doesn't show what MRU the remote end asked for and what your end did with that request.
There were a few issues with the transition but a restart of pfSense sorted that out
That's possible, I'm afraid. The code to adjust interface MTUs tried to be intelligent, but there might be some corner cases with raising the PPPoE MTU above 1492 for the first time. /etc/inc/interfaces.inc is a bit of a mess, really - I can only endorse the pfSense team's intention to rewrite pfSense 3.0 from scratch using a language other than PHP.
I also incorrectly put the MTU on the PPPoE configuration page as a first try. I am using a modem connected to it's own NIC (no VLAN, it is a Realtek RTL8111E port on an APU board).
Raising the MTU in the Advanced part of the PPP settings page will also work so long as you haven't set an explicit MTU on the PPPoE interface of 1492. The MTU setting logic uses the MTU from the PPPoE port if that is set in the Advanced part of the PPP settings, otherwise it uses the MTU set on the PPPoE interface.
It takes fewer steps to set the MTU of the WAN interface to 1500 (Interface -> WAN, set MTU value) than the MTU of the PPP port to 1500 (Interface -> (assign), choose PPP tab, press E next to PPP interface in question, press Advanced, set MTU value).
Setting a MTU of 1500 on the NIC interface configuration seems a bit counter-intuitive (to me) when the interface MTU will be more than that.
I believe it is intuitive. You are setting the MTU of the PPPoE interface, not the MTU of the physical NIC or VLAN over which that PPPoE interface runs (which you should not configure explicitly - in the absence of explicit configuration an attempt will be made to set the parent interface's MTU to 8 bytes more than the PPPoE interface MTU, allowing for the 8 byte PPPoE overhead).
On my setup, igb0 and igb_vlan10 have an MTU of 1508, whereas pppoe0 has an MTU of 1500.
Unfortunately that is as far as I can take it with my current ISP on my current plan. The larger frames didn't make it in or out. I confirmed that my ADSL2 plan doesn't support this with my ISP (and no other technology is available at my address).
If I could get a modem that could do PPPoA with conversion to PPPoE and supported PPPoE max payload I might be ok (an end-to-end PPPoE to my ISP is limited to 1492). I have both a Vigor 120v2 and a Cisco 877 as modems. Can anyone recommend workarounds/alternatives/configurations?
Assuming you have a known good NIC that supports jumbo frames on another machine, such as an Intel device, it would be worth testing that the RTL8111E works correctly with jumbo frames in pfSense or FreeBSD. Realtek chips aren't always the most reliable, though I believe they have got more reliable in recent years.
If you have a switch that supports mirroring, it would be worth capturing a packet trace of what is actually going over the wire between the NIC and the DSL model.
I suspect that the problem is that the PPPoE <-> PPPoA bridge support in your modem does not have working support for RFC 4638 / MTU > 1492 operation. Whilst RFC 4638 can be used in a part ATM part Ethernet scenario (as in the two scenarios in Figure 3 of the RFC, where the backhaul from DSLAM to BRAS is gigabit Ethernet rather than ATM), it is most commonly available to end users on pure IP networks with no legacy ATM.
The easiest solution would be if your ISP supports Ethernet framing over the DSL (which involves switching the modem from ATM to PTM - if this is supported you might need to configure a VLAN tag), though this support is not very likely on ADSL services. In the UK, PTM is used with VDSL2, but almost all ADSL remains ATM.
If you must use ATM, then I think your chances of finding an PPPoE <-> PPPoA bridge supporting RFC 4638 / MTU > 1492 operation are pretty slim. If you can't find such a bridge and your ISP requires you to use ATM / PPPoA, the only way to get MTU 1500 operation is to terminate the PPPoA session on the device with the DSL modem and link to the pfSense box using IP over Ethernet rather than PPPoE. There are various ways to do this, depending on whether you have one dynamic IP address, one static IP address or a routed IP block.
If you have one IP address, 'PPP half bridge' (Google for it) is one approach, leasing the public IP address to the pfSense box using a short DHCP leases (if you happen to have a static public IP address, you can configure pfSense to use static IP for added robustness unless the 'PPP half bridge' requires you to take up the DHCP lease). 1:1 NAT is another approach, though it can be troublesome.
Routed IP is the neatest approach, but it requires a spare public IP address to dedicate to the DSL modem device.
-
Tested this patch with apu1d (http://www.pcengines.ch/apu1d4.htm) running nanobsd 2.2.4, and seems to working. This time I set MTU 1500 in the WAN interface setting.
Before
[2.2.4-RELEASE][root@pfSense.localdomain]/root: ifconfig re0;ifconfig pppoe0 re0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 options=8219b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,wol_magic,linkstate>ether 00:0d:b9:33:78:70 inet6 fe80::20d:b9ff:fe33:7870%re0 prefixlen 64 scopeid 0x1 nd6 options=23 <performnud,accept_rtadv,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex,master>) status: active pppoe0: flags=88d1 <up,pointopoint,running,noarp,simplex,multicast>metric 0 mtu 1492 inet 208.64.38.204 --> 213.133.104.38 netmask 0xffffffff inet6 fe80::20d:b9ff:fe33:7870%pppoe0 prefixlen 64 scopeid 0xa nd6 options=21<performnud,auto_linklocal></performnud,auto_linklocal></up,pointopoint,running,noarp,simplex,multicast></full-duplex,master></performnud,accept_rtadv,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,wol_magic,linkstate></up,broadcast,running,simplex,multicast>After
[2.2.4-RELEASE][root@pfSense.localdomain]/root: ifconfig re0 ; ifconfig pppoe0 re0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1508 options=82098 <vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,linkstate>ether 00:0d:b9:33:78:70 inet6 fe80::20d:b9ff:fe33:7870%re0 prefixlen 64 scopeid 0x1 nd6 options=23 <performnud,accept_rtadv,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>) status: active pppoe0: flags=88d1 <up,pointopoint,running,noarp,simplex,multicast>metric 0 mtu 1500 inet 208.64.38.116 --> 213.133.104.38 netmask 0xffffffff inet6 fe80::20d:b9ff:fe33:7870%pppoe0 prefixlen 64 scopeid 0xa nd6 options=21<performnud,auto_linklocal></performnud,auto_linklocal></up,pointopoint,running,noarp,simplex,multicast></full-duplex></performnud,accept_rtadv,auto_linklocal></vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,linkstate></up,broadcast,running,simplex,multicast>After ping
[2.2.4-RELEASE][root@pfSense.localdomain]/root: ping -D -s 1472 8.8.4.4 PING 8.8.4.4 (8.8.4.4): 1472 data bytes 1480 bytes from 8.8.4.4: icmp_seq=0 ttl=51 time=134.734 ms 1480 bytes from 8.8.4.4: icmp_seq=1 ttl=51 time=133.631 ms 1480 bytes from 8.8.4.4: icmp_seq=2 ttl=51 time=133.864 ms 1480 bytes from 8.8.4.4: icmp_seq=3 ttl=51 time=133.571 ms 1480 bytes from 8.8.4.4: icmp_seq=4 ttl=51 time=134.276 ms ^C --- 8.8.4.4 ping statistics --- 5 packets transmitted, 5 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 133.571/134.015/134.734/0.436 ms -
Tested this patch with apu1d (http://www.pcengines.ch/apu1d4.htm) running nanobsd 2.2.4, and seems to working.
That's good to know. My confidence in the patch is growing. So far, only gregb has failed to get this to work, and that is almost certainly down to his environment (a PPPoE <-> PPPoA bridge rather than end to end PPPoE with explicit RFC 4638 support from the ISP) rather than the patch.
It's notable how many offload features of the apu1d's re0 interface are unavailable in jumbo mode. You lose hardware checksumming and IPv4 TSO. This contrasts with the vmx interface in your other box and the igb interface in my box, both of which lose no offload features in jumbo mode.
-
I've just submitted pull requests against pfSense 2.3:
mpd5 changes: https://github.com/pfsense/FreeBSD-ports/pull/1
pfSense support: https://github.com/pfsense/pfsense/pull/1959I have posted pointers to those pull requests on Redmine #4542. Hopefully RFC 4638 client support will appear in a future pfSense 2.3 build.
I have also submitted the mpd5 changes as FreeBSD PR 203695.
-
I've posted a 2.2.5-RELEASE version of the patch in a new thread (still amd64 full installs only).
-
Hi David,
I am looking forward to use the PPPoE MTU > 1492. Using a fiber PPPoE connection and can see the mtu=1492 on the WAN connection.
Using 2.3 snapshot for a few days now and wondering if/when your changes for using MTU > 1492 will hit the 2.3 snapshots.
-
All the necessary binary support is in the latest pfSense 2.3 snapshots, but the changes to pfSense itself have not been merged. There is a pull request open at https://github.com/pfsense/pfsense/pull/1959 , which I hope the pfSense team will merge to pfSense 2.3 in due course.
I've not been following pfSense 2.3 that closely in the past few weeks, but if the System Patches package is working, you should be able to install the patch using GitHub's diff feature. In System patches use URL https://patch-diff.githubusercontent.com/raw/pfsense/pfsense/pull/1959.diff with Path Strip Count 2 (not 1 as I earlier said in error) and Base Directory / . This is completely untested, but you're welcome to give it a go.
-
I use 2.3 in production environment (home) and I hasitate to go for your sugested solution.
Maybe I create an VM in my lab to test it and if succesfull, I give it a try.Even better when pfSense team will merge your pull request into the project.
-
Tested on 2.3 built on Wed Dec 23 05:35:02 CST 2015 with System patches package
Fetch = ok, testing patch failed.
Patch can NOT be applied cleanly (detail)
/usr/bin/patch –directory=/ -t -p1 -i /var/patches/567e75dbd3c0c.patch --check --forward --ignore-whitespace
Hmm... Looks like a unified diff to me...
The text leading up to this was:|diff --git a/src/etc/inc/interfaces.inc b/src/etc/inc/interfaces.inc
|index f7e60a6..8adadc0 100644
|--- a/src/etc/inc/interfaces.inc+++ b/src/etc/inc/interfaces.inc No file to patch. Skipping... Hunk #1 ignored at 1851. Hunk #2 ignored at 1932. Hunk #3 ignored at 3061. Hunk #4 ignored at 3081. Hunk #5 ignored at 3262. Hunk #6 ignored at 3332. Hunk #7 ignored at 4644. 7 out of 7 hunks ignored while patching src/etc/inc/interfaces.inc done Patch can NOT be reverted cleanly (detail)
/usr/bin/patch –directory=/ -f -p1 -i /var/patches/567e75dbd3c0c.patch --check --reverse --ignore-whitespace
Hmm... Looks like a unified diff to me...
The text leading up to this was:|diff --git a/src/etc/inc/interfaces.inc b/src/etc/inc/interfaces.inc
|index f7e60a6..8adadc0 100644
|--- a/src/etc/inc/interfaces.inc+++ b/src/etc/inc/interfaces.inc No file to patch. Skipping... Hunk #1 ignored at 1851. Hunk #2 ignored at 1924. Hunk #3 ignored at 3047. Hunk #4 ignored at 3063. Hunk #5 ignored at 3195. Hunk #6 ignored at 3228. Hunk #7 ignored at 4540. 7 out of 7 hunks ignored while patching src/etc/inc/interfaces.inc done -
Tested on 2.3 built on Wed Dec 23 05:35:02 CST 2015 with System patches package
Fetch = ok, testing patch failed.
Patch can NOT be applied cleanly (detail)
/usr/bin/patch –directory=/ -t -p1 -i /var/patches/567e75dbd3c0c.patch --check --forward --ignore-whitespace
My mistake - path strip count needs to be 2 for 2.3. I've edited my earlier post accordingly.
-
No problem, i really appreciate your efforts.
My goal is to patch your RFC 4638 and PPP IPv6 fix.
Still not succesfull.
First patch PPP IPv6 with this test result
/usr/bin/patch –directory=/ -t -p1 -i /var/patches/567e80217400c.patch --check --forward --ignore-whitespace
Hmm... Looks like a unified diff to me...
The text leading up to this was:|diff --git a/etc/inc/interfaces.inc b/etc/inc/interfaces.inc
|index b425434..caefa85 100644
|--- a/etc/inc/interfaces.inc+++ b/etc/inc/interfaces.inc Patching file etc/inc/interfaces.inc using Plan A... Hunk #1 succeeded at 1984 with fuzz 1 (offset 166 lines). Hunk #2 succeeded at 1853 (offset 2 lines). Hunk #3 succeeded at 3265 (offset 226 lines). done The RFC 4638 path test gives this:
/usr/bin/patch –directory=/ -t -p2 -i /var/patches/567e809abd883.patch --check --forward --ignore-whitespace
Hmm... Looks like a unified diff to me...
The text leading up to this was:|diff --git a/src/etc/inc/interfaces.inc b/src/etc/inc/interfaces.inc
|index f7e60a6..8adadc0 100644
|--- a/src/etc/inc/interfaces.inc+++ b/src/etc/inc/interfaces.inc Patching file etc/inc/interfaces.inc using Plan A... Hunk #1 succeeded at 1855 (offset 4 lines). Hunk #2 succeeded at 1936 (offset 4 lines). Hunk #3 succeeded at 3039 (offset -22 lines). Hunk #4 succeeded at 3085 (offset 4 lines). Hunk #5 succeeded at 3240 (offset -22 lines). Hunk #6 succeeded at 3336 (offset 4 lines). Hunk #7 succeeded at 4622 (offset -22 lines). done For the latter I used the Path Stripcount = 2
-
Sorry, i was incomplete.
Apply patch (both) is succesfull, put putting MTU=1500 on WAN PPPoE interface and reboot still gives MTU=1492 in status->interfaces
-
Checking ifconfig:
em2 (=NIC with 2 vlan's), MTU = 1508
pppoe1 on em2 (=WAN vlan), MTU = 1492From log:
mpd_wan.conf:32: Unknown command: 'set pppoe max-payload 1500'. Try "help". -
Checking ifconfig:
em2 (=NIC with 2 vlan's), MTU = 1508
pppoe1 on em2 (=WAN vlan), MTU = 1492From log:
mpd_wan.conf:32: Unknown command: 'set pppoe max-payload 1500'. Try "help".The RFC 4638 patch is behaving as I expect: configuring the MTU of the parent interface of the PPPoE interface to 8 more than the configured PPPoE MTU (to allow for the 8 byte PPPoE overhead) and setting 'set pppoe max-payload' in the mpd configuration.
The problem is that your mpd5 binary doesn't contain RFC 4638 support. The changes were merged back on 12 October and should have started to appear in snapshots shortly afterwards.
I checked that the necessary binary and kernel support exists in the latest builds of 2.3 by installing pfSense-LiveCD-2.3-ALPHA-amd64-20151228-0402.iso.gz (the latest at the time I tried this) and updating via the GUI. RFC 4638 worked correctly on those binaries.
What does pkg info -o net/mpd5 show (use a shell command prompt)? If the version is mpd5-5.7_4 or above, I'd expect the patch to be present. It definitely is not present in mpd5-5.7_3.
If you only have mpd5-5.7_3, when did you last update your binaries? gitsync only updates the script components, not the binaries. If your binaries are old enough not to include mpd5-5.7_4, then you may well find that an update leaves you with an installation that doesn't work correctly. Your best option in this scenario is probably to back up your config.xml, reinstall pfSense 2.3 from a recent snapshot and restore your config.xml.
-
Command: pkg info -o net/mpd5 shows:
mpd5-5.7_4 net/mpd5
So it should be present?
My last update is by GUI (so also updated binary?) and built on Wed Dec 23 05:35:02 CST 2015
I will update (by GUI) to latest snapshot now, will report back in a few minutes.
-
So, strange thing is:
After update without apply the patch, the log gives:
mpd_wan.conf:29: Error in 'set link mtu 1500': max MTU on type "pppoe" links is 1492But after apply of the patch, the log gives:
mpd_wan.conf:32: Unknown command: 'set pppoe max-payload 1500'. Try "help".What am I doing wrong?
One more test: I will skip your patch PPP IPv6, and only test the RFC 4638 patch.
Will report back in a few minutesEdit: No, same error about unknown command
Edit: I am on remote location now, so I cannot reinstall 2.3 on the moment and restore config.xml. Will try it when I am on location again
-
set pppoe max payload is the correct syntax for PPPoE MTU > 1492. The pfSense patch was working correctly before you uninstalled it - the problem was that mpd5 and/or the kernel on your system did not have working RFC 4638 support. You need the pfSense patch to ensure the parent interface MTU is set correctly and to ensure the correct mpd5 configuration syntax is used.
mpd5-5.7_4 is built using the RFC 4638 patch, but the resulting mpd5 binary will only support RFC 4638 if it was built on a machine with the RFC 4638 kernel patch installed. Unknown command: 'set pppoe max-payload 1500'. Try "help". implies that your mpd5-5.7_4 was built on a machine without the kernel patch.
If you don't want to try a total reinstall, try:
pkg upgrade -f net/mpd5and answer y when asked. This should reinstall mpd5 from the pfSense-core repository (at the time of writing, this is identical to the version I tested and confirmed to be working earlier today assuming that you are using amd64).
If that doesn't give you working RFC 4638 support, try an Upgrade from the pfSense GUI, which should install the latest kernel - though back up your config.xml first. As I said earlier, an upgrade on a system that is too old might leave you with an unusable mess that you have to reinstall.
-
pkg upgrade -f net/mpd5 did the trick. :D
Indeed the pkg upgrade ask for reinstall and then it worked!
Thank you for your support!
Will report back if this is stable.
-
It's good to know that a forced reinstall of mpd5 was sufficient to resolve your issue, as that might help others. If anyone else has a similar problem, I suggest pkg upgrade -fy net/mpd5 (the additional y means there's no confirmation required, so the command will work from the 'Execute shell command' part of Diagnostics -> Command Prompt).
The PPP log should indicate whether you successfully negotiated an MTU of 1500. To test whether MTU 1500 is working, try ping -D -s 1472 -c 10 8.8.4.4
If things are working correctly, you'll successfully send 10 1500 byte packets (1472 byte payload, 8 byte ICMP overhead and 20 byte IPv4 overhead) to Google DNS, and Google DNS will send 10 back. You can experiment with another destination instead of 8.8.4.4.
If things are not working correctly, you'll get packet loss and/or complaints about 'frag needed and DF set'.
The IPv6 test, if you have native IPv6 support, is ping6 -D -s 1452 -c 10 www.google.com
In this case, you send 1500 byte packets (1452 byte payload, 8 bytes ICMPv6 overhead and 40 byte IPv6 overhead) and the remote end sends 72 bytes back - that's the way that ICMPv6 echo request and reply works.
