PFSense 1.2.2 in VMWare ESXi dropping connections if filter is turned on

banstyle

Okay everyone, this one has got me bad. I've been using pfSense for years (best FW out there..) and this is my first setup on ESX 3.5.

Here's how it's configured:

NICs are setup as e1000 in VMWare.

LAN: vlan0 on em2, VLAN ID 1
WAN: vlan1 on em2, VLAN ID 3

LAN: 192.168.200./24
WAN: 192.168.7./24

I'm trying to test the packet filter performance. I'm using iperf to connect through the firewall, and I've ensured that the route on the iperf hosts is setup correctly. I've created a rule on the WAN interface to allow traffic to route from all hosts to a host called 192.168.200.X. I am not using NAT at all.

pFSense seems to be dropping connections after about 80KB of traffic. I can connect to that host just fine with RDP, but then after some activity, RDP will simply lose connection. Same with iperf- I will start the test, it connects, but then it stalls, and the iperf test stops at 80KB.

The funny thing? If I turn off packet filtering in the Advanced menu, then everything routes FINE and stays connected.
The moment I turn on packet filtering (it'd be nice… ;D), then it starts dropping connections.

I've narrowed this down as much as possible by changing the interfaces to directly connect to the VLANs through ESX, IE: em0 to LAN, em1 to WAN, and no change in behaviour.
I've set the packet filter mode to conservative.
I've setup default allow all rules on the WAN to no change.

My head hurts now. Does anyone have any ideas why it would be dropping a perfectly legit connection? The state table shows CLOSED:CLOSING or SYN_SENT:CLOSED for these connections, even right after they've just connected.

As an aside, I have so far gotten performance up to 600Mbits/sec when the packet filter is off, barely using CPU cycles in the VM. Damn good performance!

Thanks for any help in advance!

trendchiller

do you have the open-vm-tools installed from the packages menu ?

banstyle

@trendchiller:

do you have the open-vm-tools installed from the packages menu ?

I used the VMWare 1.2.2 image from the pfsense.org website, and assumed tools were working when I looked at the console. I installed the package just in case, but no difference- RDP session was kicked off within seconds. iperf terminated with connection reset by peer.

banstyle

Ugh, my stupidity. Sometimes you cant see the forest for the trees…

So it turns out that on my iperf test node, the return route went through another gateway instead of the pfSense firewall. Once I set a static route back through the firewall (I thought I had..), the states are now maintained and connections are good.

Thanks for the help!