pfSense and IPSEC lan to lan: a big doubt about the correct implementation

stephenw10 · Nov 30, 2022, 4:53 PM

Ok, then check both ends, do the traffic counters still show as 0 after trying to ping in both directions?

How exactly are you running that ping?

Do you see anything blocked in the firewall logs?

mauro.tridici · Nov 30, 2022, 5:01 PM

@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:

Ok, then check both ends, do the traffic counters still show as 0 after trying to ping in both directions?

yes, 0 stats on both the ends

How exactly are you running that ping?

I'm executing ping from the VPCS on the left of the schema (192.168.118.10) to the VPCS on the right (192.168.120.10) and viceversa

Do you see anything blocked in the firewall logs?

During the last attempts, it seems that nothing have been blocked but I will check again soon

stephenw10 · Nov 30, 2022, 5:09 PM

The other thing to check is the state table in Diag > States. You should see states on the LAN and IPSec interfaces for the pings on each side.

At site B there is really nothing to stop IPSec at least sending pings from 192.168.120.10 to 192.168.118.10 over the tunnel.
At site A there are other things that could be preventing it.

mauro.tridici · Nov 30, 2022, 5:17 PM

@stephenw10

Ok, these are the states for the pfsenseA

and these are the states for the pfsenseB

Question: on the Router A I created a route 192.168.120.0/24 > 10.99.0.2.
Do I need to create a route rule also on pfsense A ? something like that: 192.168.118.0/24 > 10.99.0.1?

stephenw10 · Nov 30, 2022, 5:21 PM

I see no icmp states at all between the two subnets in either of those tables. Were the pings running?

You should not need to a route back on pfSense A because Router A is the default route there. (Or at least it should be!)

mauro.tridici · Nov 30, 2022, 5:27 PM

@stephenw10 yes, the pings are running and I'm making some tests on both the directions.

I checked again the firewall logs. no rule is blocking the traffic.
I'm trying to understand where is the error

stephenw10 · Nov 30, 2022, 5:30 PM

If there are still no icmp states then either there are firewall rules blocking that traffic or the pings never arrive at pfSense. A pcap on the pfSense B em2 interface should show that.

mauro.tridici · Nov 30, 2022, 5:33 PM

VPCS on the left can ping every hosts on the schema (except the 192.168.120.1)
VPCs on the left is not able to ping 192.168.120.0/24

VPCS on the right can ping every hosts on the schema (except the 10.99.0.0/24)
VPCs on the left is not able to ping 192.168.118.0/24

If I'm not wrong, it seems that the ping traffic is not passing through the tunnel.
Sorry for this stupid information...

mauro.tridici · Nov 30, 2022, 5:34 PM

@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:

If there are still no icmp states then either there are firewall rules blocking that traffic or the pings never arrive at pfSense. A pcap on the pfSense B em2 interface should show that.

I'm going to check this. I will let you know

mauro.tridici · Nov 30, 2022, 5:37 PM

@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:

If there are still no icmp states then either there are firewall rules blocking that traffic or the pings never arrive at pfSense. A pcap on the pfSense B em2 interface should show that.

pcap on pfsense B em2 interface is completely blank/empty

stephenw10 · Nov 30, 2022, 5:40 PM

Then the client PC there is not attached to em2 as it shows in the disagram or it is sending pings some other way, it has the wrong default route.

mauro.tridici · Nov 30, 2022, 5:54 PM

@stephenw10

VPCS on the left:

VPCS> show

NAME IP/MASK GATEWAY MAC LPORT RHOST:PORT
VPCS1 192.168.118.10/24 192.168.118.1 00:50:79:66:68:01 20096 127.0.0.1:20097
fe80::250:79ff:fe66:6801/64

VPCs on the right:

VPCS> show

NAME IP/MASK GATEWAY MAC LPORT RHOST:PORT
VPCS1 192.168.120.10/24 192.168.120.1 00:50:79:66:68:00 20098 127.0.0.1:20099
fe80::250:79ff:fe66:6800/64

Both the client PC an reach internet and other machines in the schema

For example, the client PC on the left:

VPCS> ping www.google.it
www.google.it resolved to 142.250.184.35

84 bytes from 142.250.184.35 icmp_seq=1 ttl=126 time=22.655 ms
^C
VPCS> ping 10.99.0.2

84 bytes from 10.99.0.2 icmp_seq=1 ttl=63 time=3.166 ms
^C
VPCS> ping 10.99.0.1

84 bytes from 10.99.0.1 icmp_seq=1 ttl=64 time=1.025 ms
^C
VPCS> ping 192.168.122.180

84 bytes from 192.168.122.180 icmp_seq=1 ttl=63 time=2.851 ms

But if I try to make a ping from one client PC to the other one the pcap is empty.

stephenw10 · Nov 30, 2022, 7:48 PM

Do you see those successful pings in pcaps?

Those VPCS client machines only have one NIC?

Either they are not sending the pings via pfSense or the pcap is filtering them out.

mauro.tridici · Nov 30, 2022, 9:47 PM

@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:

Do you see those successful pings in pcaps?

Yes, I can see them in caps

Those VPCS client machines only have one NIC?

Yes, each VPCS has only one NIC

Either they are not sending the pings via pfSense or the pcap is filtering them out.

I don't think that pcap is filtering the pings.
Maybe something related to IPSEC tunnel because I just tried to ping pfsense B LAN IP (192.168.120.1) directly from pfsense A LAN IP (192.168.118.1) with no success.

During the next minutes, I would like to verify that static route on routerA is working as expected.

stephenw10 · Nov 30, 2022, 10:25 PM

If you try to ping 192.168.118.1 from the client behind pfSense B you must see those ping packets entering the interface at pfSense B.

What have you named that interface? (em2) Because it isn't the LAN. What firewall rules have you added on it?

mauro.tridici · Nov 30, 2022, 10:36 PM

@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:

If you try to ping 192.168.118.1 from the client behind pfSense B you must see those ping packets entering the interface at pfSense B.

Yes, I can confirm that. But the ping packets have not a response

What have you named that interface? (em2) Because it isn't the LAN. What firewall rules have you added on it?

em2 interface on pfsenseB is named "ACCESS"
I added a very open rule "from any to any"

I don't know if it can help, but please note that, in the IPSEC configuration page, I didn't set the name of the subnet "ACCESS" but the networks:

This is for the pfsenseB. I did the same for the pfsenseA (changing the order).

stephenw10 · Dec 1, 2022, 1:56 AM

Ok, that looks good.

Earlier though you said those pings did not appear in a pcap in pfSense on the Access interface. Is that still the case?
It looks like that pcap was done on the client directly.

With that rule if they are arriving it should definitely create at least a state on the Access interface. And since the P2 matches it it should go out over the tunnel creating a state on IPSec too.

Steve

mauro.tridici · Dec 1, 2022, 10:36 AM

@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:

Ok, that looks good.
Earlier though you said those pings did not appear in a pcap in pfSense on the Access interface. Is that still the case?

it is no longer happening.

With that rule if they are arriving it should definitely create at least a state on the Access interface. And since the P2 matches it it should go out over the tunnel creating a state on IPSec too.

I just repeated the test pinging 192.168.118.1 from PC with IP 192.168.120.10.
Ping is not working and the states for ACCESS interface are 0.

Attaching the virtual Wireshark to the em0 port of RouterA I see these lines :

168 2022-12-01 11:35:34.467546 192.168.120.10 192.168.118.1 ICMP 146 Echo (ping) request id=0x004b, seq=956/48131, ttl=63 (no response found!)
169 2022-12-01 11:35:34.468024 192.168.122.230 192.168.122.180 ICMP 174 Destination unreachable (Protocol unreachable)

stephenw10 · Dec 1, 2022, 2:50 PM

Ok, the states exist though and on both ACESS and IPSec. You can see both show 193 packets outbound and no replies.

So check the packet counters in the IPSec status again.

I would not expect to be able to see that traffic on the WAN dircetly, it should be encrypted.

However that 193 packet AH state on WAN looks suspicious. Do you have the Phase 2 protocol set to AH instead of ESP at one end?
It should always be ESP there, AH is unencrypted.

mauro.tridici · Dec 1, 2022, 3:27 PM

@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:

Ok, the states exist though and on both ACESS and IPSec. You can see both show 193 packets outbound and no replies.
So check the packet counters in the IPSec status again.
I would not expect to be able to see that traffic on the WAN dircetly, it should be encrypted.

Yes, you are right.

However that 193 packet AH state on WAN looks suspicious. Do you have the Phase 2 protocol set to AH instead of ESP at one end?
It should always be ESP there, AH is unencrypted.

P2 protocol was the same on both endpoints (it was AH, you are right).

Anyway, everything works as expected now. It was a Nat problem on the RouterA.
Many thanks for your help. You are a genius :)

As soon as possible I will share what was happening. right now I'm travelling