MTU issue with PPPoE Server

[Ruddimaster]

Hello

I have an issue with PPPoE Server.

We have a bunch of external IP-Addresses. So we want to give some host fixed IPs via PPPoE.

DMZ 1–---------------------------------------------- pfsense -----------------------------------------[www]  8.8.8.8
1.2.3.1/30 [external Address]                                    |                                                           
10.16.10.1/24 [internal Address]                                |
                                                                                    –-- DMZ 2 1.2.3.6/30

PPPoE works... Ping works also
But big packets (MTU >1452) wont go thought.
The same to the second DMZ

ifconfig

poes10: flags=88d1 <up,pointopoint,running,noarp,simplex,multicast>metric 0 mtu 1480
        inet6 fe80::20c:29ff:fec9:bdcf%poes10 prefixlen 64 scopeid 0x19
        inet 10.16.10.254 --> 1.2.3.1 netmask 0xffffffff
        nd6 options=21 <performnud,auto_linklocal>poes11: flags=8890 <pointopoint,noarp,simplex,multicast>metric 0 mtu 1500</pointopoint,noarp,simplex,multicast></performnud,auto_linklocal></up,pointopoint,running,noarp,simplex,multicast> 

I have tried to set different MTUs on this host (down till 1200)…. and also different hosts.
Without PPPoE big Packets went to DMZ.

Packet Capture :
Test1 via PPPoE on Interface DMZ 2

20:26:31.294850 IP 1.2.3.1 > 1.2.3.6: ICMP echo request, id 1, seq 255, length 40
20:26:31.295371 IP 1.2.3.6 > 1.2.3.1: ICMP echo reply, id 1, seq 255, length 40

Test2 via PPPoE on Interface DMZ 2

20:29:18.308484 IP 1.2.3.1 > 1.2.3.6: ICMP echo request, id 1, seq 263, length 2008
20:29:21.336250 IP 1.2.3.1 > 1.2.3.6: ICMP echo request, id 1, seq 266, length 2008

Test2 without PPPoE  on Interface DMZ 2

20:37:58.065317 IP 10.16.10.1 > 1.2.3.6: ICMP echo request, id 1, seq 287, length 1480
20:37:58.065322 IP 10.16.10.1 > 1.2.3.6: ip-proto-1
20:37:58.066207 IP 1.2.3.6 > 10.16.10.1: ICMP echo reply, id 1, seq 287, length 1480
20:37:58.066243 IP 1.2.3.6 > 10.16.10.1: ip-proto-1

With these tests I had a MTU on 1.2.3.1 of 1480…
The same captures are on the external interface to 8.8.8.8
Unfortunately I can't capture the PPPoE-Server interface because it is not selectable.

I have read https://forum.pfsense.org/index.php?topic=82939.0 but it seems not be the same issue…

Dirk

[Ruddimaster]

Does any one need more information? I am the only one, who have this issue?

Dirk

[Ruddimaster]

###Bump###

[David_W]

If you're testing MTU using ping, you need to set the 'don't fragment' bit and make sure that pfSense is not set to override invalid DF bits (System -> Advanced, Firewall / NAT tab, 'Clear invalid DF bits instead of dropping the packets' should be unchecked).

If you're using the pfSense shell prompt, the correct syntax is ping -D -s <payload size=""><destination>. You specify the payload size - there is 28 bytes of overhead on top. For example, if you find the maximum payload size that works over a link is 1472, this means the MTU is 1500.

If your big packets had a payload size of 1452, this would correspond to the MTU of 1480 configured on the PPPoE interface. Meanwhile, the mention of ip-proto-1 in the packet capture of the supposedly functional big packet scenario without PPPoE may well indicate fragmentation was in use.

I have to ask - why use PPPoE for DMZ IPs? It seems to be the most complex and highest overhead option. I would either use routed IP or 1:1 NAT.</destination></payload>

[Ruddimaster]

Hi David,

many thanks for your reply….

to answer your question:
we have several costumer connected to us via microwave. Our DC is for this costumer the internet breakout.
I am the owner of the external ip-addresses. I am responsible for the communication, to and from the internet (in German called "Störerhaftung"). To guarantee that a specific costumer use a specific IP in this range , I need PPPoE, or I must use for each costumer his own VLAN with an overhead  of unused addresses (Broadcast, Net-IP).

I will try your suggestion...

Dirk