1501 length packages - problem with MTU on virtual pfSense (Proxmox)

taenzerme

Hello all,

I'm running into some strange problems with too large packets on our WAN interface.

Setup:

pfSense 2.2 64Bit on Proxmox 3.4 host, 2 cores, 4GB RAM, CPU max 5%
HW NIC eth1 => WAN, MTU 1500
HW NIC eth4 = > LAN, MTU 9000
HW NIC eth2 => LAN, connected to same switch, but not active
vmbr0, OVS Bridge => eth4 => LAN
vmbr1, OVS Bridge => eth1 => WAN
Jumbo Frames on switches enabled
pfSense MTU WAN If.: 1500
Clear invalid DF bits instead of dropping the packets: Enabled
Disable hardware checksum offload: Enabled
Disable hardware TCP segmentation offload: Enabled
Disable hardware large receive offload: Enabled
All other local if's on 9000 MTU
Storage cluster (Synology): 9000 MTU
VMs on all proxmox hosts: Default MTU 1500

Log on Proxmox hosts tells me:


...
Mar 24 18:40:46 vmhost1 kernel: __ratelimit: 6 callbacks suppressed
Mar 24 18:40:46 vmhost1 kernel: openvswitch: tap108i7: dropped over-mtu packet: 1501 > 1500
Mar 24 18:40:46 vmhost1 kernel: openvswitch: tap108i7: dropped over-mtu packet: 1501 > 1500
Mar 24 18:40:46 vmhost1 kernel: openvswitch: tap108i7: dropped over-mtu packet: 1501 > 1500
Mar 24 18:40:46 vmhost1 kernel: openvswitch: tap108i7: dropped over-mtu packet: 1501 > 1500
Mar 24 18:40:46 vmhost1 kernel: openvswitch: tap108i7: dropped over-mtu packet: 1501 > 1500
Mar 24 18:40:46 vmhost1 kernel: openvswitch: tap108i7: dropped over-mtu packet: 1501 > 1500
...

tap108i7 is the OVS bridge on the Proxmox host for WAN If. (vtnet7).

I did some package capturing showing that large packets on the WAN interface come from an virtual IP, i.e. inside the network:


Id = 12
Source = 217.76.xxx.xx
Destination = 7x.x.x.xxx
Captured Length = 1506
Packet Length = 1506
Protocol = TCP
Date Received = 2015-03-24 17:28:54 +0000
Time Delta = 0.00888514518737793
Information = HTTP -> 58826 ([ACK], Seq=4188548632, Ack=3381854676, Win=243)

The source IP is a public IP from our public pool currently NATing to a VM on another proxmox host on the same network.
Destination is some random public IP (not ours).

Any ideas why these large packages are beeing generated? Where do they come from? How do I stop them?

The VMs "behind" the pfSense are on multiple vlans, each having their own DHCP server. The VLANs are created on the switches and assigned to the pfSense's virtual NICs. Should I set the VMs MTU to 9000, too, as they are on the local networks (the public IP's are NATed on the pfSense and not directly connected to the VM)?

Thanks
Sebastian