IPSEC traffic denied by default IPv4 Rule
-
Hoping someone can help me resolve this puzzling issue. I have a remote site connected via IPSEC I have recently begun having an issue with on one of our VPN gateways. The device is a new Canon MFP, when attempting to access port 80 for the management site of the printer my traffic is being blocked over my IPSEC interface with the error "blocked by IPv4 defualt Rule" TCP & TCP:A packets. This only occurs coming from the remote site (if I filter down on the IP as the source in the log), going to the remote device it is passing the packets. Traffic on port 25 from this device is also being blocked in the same manner, here's the kicker, from the device I can communicate to ntp with no issues, I can also hit port 9100 from my end as well. However, all other site traffic for the subnet is passing without issue, I telneted from one of the servers at the remote to our Exchange box via port 25, no issue. I have 16 sites in total on this VPN gateway, none of the other sites are experiencing these issues with their MFP devices or any device for that matter. Just this one device, tried different IP addresses as well. This is what I have:
pfsense 2.3.5-RELEASE-p2 / BSD 10.3-RELEASE-p29 on a Dell server - I upgraded when I started having the issue hoping it would clear.
IPSEC rule is to allow ipv4/ipv6 any any -set sloppy state as a test to try and clear
I put a LAN rule in place to allow ipv4/ipv6 any any
No default gateway on the LAN in the pfsense
Disabled route to LAN in static routes
Bogon is not check on WAN or LAN
Bypass firewall for traffic on the same interface is checkedOther side is a Cisco ISR4321 (standard for our remote sites) -all sites share the same configuration, no DMVPN, just direct IPSEC from remote to datacenters
Like I stated, there are no other sites experiencing these issues and all other traffic from this particular remote is flowing.
-
Sounds like packets to the printer are not flowing along the same path as from the printer.
If you're thinking sloppy states will fix it, it sounds like you are already suspecting asymmetry in your routing somewhere.
Best thing to do is design the network so packets flow in an out "properly."
-
I was on that track, but everything else from the network is working properly. It is only the MFP devices, I also do not have this issue with other MFPs at other remotes terminating to this pfsense box. I can send NTP fine from the device, but when I try to browse http from the datacenter network to the device the packets are blocked inbound.
From other devices:
It is also experienced from a Canon and a Ricoh printer.
-
Are their default gateways set the same as the rest?
-
Yes, default gateway set the same on all of the devices.
-
Is that default gateway the pfSense device or some other router on your network?
You are almost certainly seeing asymmetry. You just need to identify it and fix it.
-
Default gateway on that network goes direct to the router which is the VPN device terminating to the pfsense on the other side. On the pfsense there is a route pointing to distribution switch in the datacenter.
It just seems that if I had a routing issue it would affect all devices on the network there.
-
Sorry. Going to need some sort of diagram. Right. And if it was all correct it would be working for all devices, not just some. When it's asymmetric some devices deal with things like ICMP redirects differently.
-
-
OK there's nothing special there.
In the firewall log it looks like there is a connection from 10.77.84.24 to 10.77.87.3 that is failing for some reason.
That looks like the initial connection is blocked inbound but it is not a SYN. Then the next packet is from the same source/dest but is an ACK. That doesn't make much sense. And it looks like they are both arriving over the tunnel that way. The firewall is (apparently correctly) rejecting the connection.
I would be looking at the differences between the TCP connections that work in that direction and those that don't. And from what I can tell the problem is on the other side.
You might need to pcap and examine that traffic to see what's actually going on.
What are the rules on your IPsec tab?
Is there a GRE or anything in the mix here or just a straight IPsec transport VPN?
-
Straight IPSEC, rule on IPSEC right now to allow all protocols ipv4/ipv6 any any. I can open a TAC case with Cisco and see what we come up with, will post the results in case anyone else runs across this issue.
-
I have been informed that another possibility is those logs are the result of fragmented traffic that, for some reason, cannot be reassembled.
A packet capture might reveal something there as might looking at the raw firewall logs. You can see them by checking the Show raw filter logs checkbox in System > Logs, Settings or by doing something like
clog /var/log/filter.log | grep '10\.77\.84'. -
Yes, Cisco just asked for that. We are going to do a packet capture on both ends.
