Default rule blocking Outlook RPC and HTTP RPC packets
-
This happens with both the stable 2.0.3 release and every beta and RC snapshot I've tried, including HD installs and USB drive nano versions going back for several weeks.
The Pfsense system has 2 network cards, a WAN (realtek bge) and LAN (intel em). There are a couple of VLANs on the LAN side to separate servers and workstations/wireless. Workstations in one VLAN can access the server VLAN and the Internet via the WAN just fine, and all connections to the company MS Exchange server (internally or externally accessed) work fine if they are using POP3S and SMTP-TLS (Ports 995, 25, and 587). Basically everything works great and fast….
BUT:
On workstations with Outlook 2007 and 2010 using MS Windows RPC or Outlook Anywhere to access Exchange, the connection is unstable and sending mail seems to not work at all. The Outlook icon in the task tray reports connecting and disconnecting from Exchange over and over again, and clicking on "Send and Receive" returns this error message for the sending task:
Task 'Microsoft Exchange - Sending' reported error (0x80040115) : 'The connection to Microsoft Exchange is unavailable. Outlook must be online or connected to complete this action.
The log shows many connections from the workstation on ports in the 50902-50904 range to the Exchange server on ports 135, 443, and 34756 blocked by the default rule. I'm not sure what about these packets is setting off the very early internal rules before it even gets to anything I've set up, which at this point is "allow everything" until I figure this out. Since I'm actually having blocked connections I'm reasonably sure it's not this problem either.
If I disconnect the Pfsense box and drop the old Cisco in, everything works fine again. I'm at a loss as to what to do now to get this to work. Forcing everyone to use POP3 isn't an option. Is there some tunable or advanced setting I'm missing somewhere that could be causing this? I've tried different network cards already and they did the same thing.
-
Presumably your VLANs have corresponding VLAN interfaces in pfSense. If so, at least one of them is NOT the pfSense LAN interface and consequently has default firewall rule that blocks all accesses outside that VLAN. So you would need to add firewall rules to that interface to allow the accesses. Then after adding the rules you need to reset firewall states: see Diagnostics -> States then click on the Reset States tab and take appropriate action.
-
If that was the case wouldn't all traffic between these two VLANs be stopped and not just a couple ports on one server (especially when traffic to other ports on that same server passes with no issue)? All other traffic between these two VLANs works as it should.
-
If that was the case wouldn't all traffic between these two VLANs be stopped and not just a couple ports on one server (especially when traffic to other ports on that same server passes with no issue)? All other traffic between these two VLANs works as it should.
It appears your accesses are hitting a default block rule. Then either:
1. Your rules to allow the access are incorrectly specified (for example, you specified a particular source port rather than 'any'); or
2. Your rules to allow access are wrongly positioned (interface rules are processed top down, first match terminates rule processing); or
3. You didn't reset states after adding rules to allow access.It has been my experience that if I add rules to allow a particular access I need to reset states for those rules to take effect.
I expect you will need to provide more detail if you want further help. Details such as some log entries and screenshots of appropriate firewall rules would be a good place to start.
-
In order to simplify this and rule out problems with the VLAN setup or complicated rule sets I installed an extra NIC into the box and setup the WAN, LAN, and DMZ with no VLANs.
The interfaces have an "allow any from this network to any" rule followed by a logging default deny.
WAN Rules:
LAN Rules:
DMZ Rules:
As soon as I hit "send/receive" in outlook I checked the logs. The first one shows only the blocks. The second one I added logging to the allow rule to see what was getting through.
Log showing deny only:
Log showing deny and allow:
Only segments with TCP:SEW flags are going through and everything else is blocked unless pfsense is passing and blocking things that are incapable of being logged. They might appear to be out of state to pfsense, but they shouldn't be, and the legitimate connection is failing regardless and as soon as I unplug this box and drop the cisco back in place this behavior stops. I can't find any useful information regarding pfsense interaction with Exchange and/or VMWare (the exchange server is a guest system on ESXi 3.51), so I'm stuck at this impasse.
-
pfSense already has an implicit default deny at the end, there is no reason to add those yourself.
Note that your explicit deny is what is showing up in the logs… if you dump that, does it work? (I'm not sure why the deny is triggering, because it should be first-match and hitting on the Allow All you have first, but in your shoes I'd be in the "try anything" phase...)
-
I put explicit denials at the end just to make sure the rules applied to the interfaces weren't being ignored or skipped entirely. When I didn't have them in place, the behavior was the same. So the ultimate question here is why do the allow all rules before them seem to be skipped over or ignored? I can see no reason for this at all.
-
The common way of discussing firewall rules can easily give the impression the rules are processed on every packet through the firewall. This would impose considerable overhead so an optimisation is commonly applied: the firewall forwards packets if the packet matches a "state" (interface on which the packet arrives, source and destination IP address and ports all match the values stored in the state) otherwise the firewall sees what the firewall rules say about the packet and creates a state if the rules allow.
States have an associated timer; If the state is "idle" (no data flow) for "too long" the state is deleted. TCP connections can only have a state created when the TCP connection is setup.
Your observation: @mknit74:
Only segments with TCP:SEW flags are going through and everything else is blocked unless pfsense is passing and blocking things that are incapable of being logged.
gives some clues to what is going on.
The segments with S (=SYN) flag are TCP connection setup. The firewall won't (shouldn't) have a state for them so firewall rules are checked and those connection attempts are allowed by firewall rule. The segments without the S flag are not TCP connection setup so there should be an existing state. A new state can't be setup because these are not connection setup segments.
There might not be a state because the firewall restarted after connection setup or the connection was idle for "too long" or firewall states were reset or _the WAN link went down which apparently causes a "filter reload" and I suspect that clears states.Perhaps your firewall optimisation option (System -> Advanced, click on Firewall/NAT tab) is too aggressive (deleting states too quickly), perhaps the clients are buggy (attempting to use "old" TCP connections after the firewall thinks the connections have been closed), perhaps the server is not sending "keep alives" or its "keep alive" interval is too long.
Since you apparently don't have a problem with a Cisco firewall (which presumably is firewalling between the clients and servers under discussion here) a good place to start might be examination of firewall optimisation option and other settings on the System->Advanced page Firewall/NAT tab._
-
Switching the firewall optimization to conservative did not solve the problem, I wasn't expecting it to though, unless pfsense deletes states after only a few seconds of idle time on "normal".
I put in TCP specific rules (TCP from the interface's network any port to anywhere/any port) on all the interfaces and tried messing with states from there. Sloppy state had no effect. Synproxy state causes most of the systems in my network to be unable to communicate with each other at all. I can get outside to internet sites but nothing that is TCP works Inter-VLAN. (This is almost the same behavior this setup exhibited when I tried m0n0wall instead of pfsense, which leads me to believe that m0n0wall uses something like synproxy state keeping by default… but I digress).
I have a gut feeling this issue has to do with tunables (either on pfsense/nanobsd or the VMWare box I'm trying to get to), specifically with TCP window size scaling, but I haven't a clue what to try in that arena.
-
Just incase anyone else is having a similar issue, I think I might have solved this and it wasn't, as far as I can tell, an issue with TCP window scaling.
I changed the Firewall optimization options in advanced settings back to conservative again as wallabybob suggested but this time I also changed the Windows TCP Keepalive settings (to 1 minute) on the workstation running Outlook. It seems (so far) to be working as expected. I may try slightly longer times, or setting it on the Exchange server itself if I don't notice any problems over the next 24 hours.
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters] "KeepAliveTime"=dword:0000ea60KeepAliveTime is in milliseconds.
-
That actually did not solve the problem. It seemed to work for about 15 minutes or so before sending mail with file attachments began failing and then shortly after all sent mail was failing. Back to square one.
-
Hello, (I'm learning english)
I excactly have the same problem since I installed Exchange 2010 3 months ago.
All run on ESXI (PfSense + Exchange). Every services are working great inside LAN or with a VPN.
But, if I'm connecting Outlook 2010 from outside, Outlook just hang/block 10 sec (some times when I send an email or change directory).
I Temporaly solved this by disabling "hardware checksum offload" (System: Advanced: Networking), but as I don't know really all this option do. I reactivated it.10.0.6.1 is my Exchange Server, and even if 443 port or 6008 sometime packet just block
I also tried to add easy rules on blocked ones, but that don't workhttp://www.hostingpics.net/viewer.php?id=594622sshot16.png
I think it's a packet segmentation with checksum error, but… i'm really not an exert :) I was just looking for help.
Hope that can help someone. I can do some test but as that a prod environment, I can't just play with firewall rules.
Kind regards,
-> 2.0.2-RELEASE (amd64) built on Fri Dec 7 22:39:43 EST 2012 FreeBSD 8.1-RELEASE-p13
-
I had this same very issue!
My workaround was to remove the outlook profile then re-add it, all fixed, it took me hours and hours of tinkering to get to this point!