Fast routing but slow NAT performance

[frooop]

Hello,
I have a pretty basic pfsense setup with 3 local subnets (all an dedicated NICs) and one WAN interface.

The routed traffic between the subnets is performing fine, but the traffic that goes from any of these local networks to WAN (and thus is affected by NAT) is really slow (throughput around 200kbit).

I run my pfsense in virtualized (vmware vcloud director) infrastructure and I'm using the E1000 NICs, but I don't think that this can be the problem here since everything else is performing fine and only nat seems to be affected.

My problem is that I pinned to problem down to "something with nat", but now I'm stuck. What can be the problem here? I already tried resetting the state table with no effect.

Can someone help me out here? What more information would you need?

Thanks.

[firewalluser]

Have you rebooted pfsense after making all your rule changes and is still slow then? There are some circumstances I've seen where even a reset all states doesnt make everything right so a reboot is the best option to be 100% imo.

Anything show up in the logs, are you logging the rules and checking the Status:System Logs:Firewall, Firewall tab, Normal view tab to make sure the traffic is passing through properly. What rules do you have in place and what order do you have them in (screenshot might be useful).

[frooop]

I had this problem once before (couldn't find the time to really fix it) and then a full reboot helped. Right now, I can't reboot the system, since it is in full production) and I don't want to, because I need to find the root cause of this.
My point is: I'm quite sure that a reboot would in fact fix the problem (for now).

I attached a screenshot for the relevant network firewall rules tab. I don't see any blocking happening in the logs. Traffic seems to pass through fine.


[firewalluser]

I think the problem lies with freeBSD PF rather than pfsense but need to investigate that more when I get the time.

When I changed the allocation of my nics around the other day swapping from a usb nic on wan to EM nic on wan, resetting the states before DDOs testing it went slow which only a reboot sorted.

I cant imagine pfsense having other issues with the nics changing around as much of it is based around bsd's PF anyway, so I suspect freeBSD & PF instead are the culprits but I could be wrong.

I guess if its in production you dont have a fall back device to take over in case of serious hw failure then, not even a smaller less capable device?  :(

[frooop]

Well, since it is a VM the term "fallback device" is a little bit misplaced I guess ;) In case of a hardware failure I would just move it to another ESX.

Is there anything that I can do to help you analyse the problem further?

[firewalluser]

I havent got time atm, I'm running some other tests at the moment to check the state handling in pfsense 2.1 and 2.2, they are bit time consuming as the schedule only allows 15min increments so I cant set say a 5min time span with state timeouts set to aggressive (if thats below 5mins).