pfSense and IPSEC lan to lan: a big doubt about the correct implementation
-
I see no icmp states at all between the two subnets in either of those tables. Were the pings running?
You should not need to a route back on pfSense A because Router A is the default route there. (Or at least it should be!)
-
@stephenw10 yes, the pings are running and I'm making some tests on both the directions.
I checked again the firewall logs. no rule is blocking the traffic.
I'm trying to understand where is the error -
If there are still no icmp states then either there are firewall rules blocking that traffic or the pings never arrive at pfSense. A pcap on the pfSense B em2 interface should show that.
-
VPCS on the left can ping every hosts on the schema (except the 192.168.120.1)
VPCs on the left is not able to ping 192.168.120.0/24VPCS on the right can ping every hosts on the schema (except the 10.99.0.0/24)
VPCs on the left is not able to ping 192.168.118.0/24If I'm not wrong, it seems that the ping traffic is not passing through the tunnel.
Sorry for this stupid information... -
@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:
If there are still no icmp states then either there are firewall rules blocking that traffic or the pings never arrive at pfSense. A pcap on the pfSense B em2 interface should show that.
I'm going to check this. I will let you know
-
@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:
If there are still no icmp states then either there are firewall rules blocking that traffic or the pings never arrive at pfSense. A pcap on the pfSense B em2 interface should show that.
pcap on pfsense B em2 interface is completely blank/empty
-
Then the client PC there is not attached to em2 as it shows in the disagram or it is sending pings some other way, it has the wrong default route.
-
VPCS on the left:
VPCS> show
NAME IP/MASK GATEWAY MAC LPORT RHOST:PORT
VPCS1 192.168.118.10/24 192.168.118.1 00:50:79:66:68:01 20096 127.0.0.1:20097
fe80::250:79ff:fe66:6801/64VPCs on the right:
VPCS> show
NAME IP/MASK GATEWAY MAC LPORT RHOST:PORT
VPCS1 192.168.120.10/24 192.168.120.1 00:50:79:66:68:00 20098 127.0.0.1:20099
fe80::250:79ff:fe66:6800/64Both the client PC an reach internet and other machines in the schema
For example, the client PC on the left:
VPCS> ping www.google.it
www.google.it resolved to 142.250.184.3584 bytes from 142.250.184.35 icmp_seq=1 ttl=126 time=22.655 ms
^C
VPCS> ping 10.99.0.284 bytes from 10.99.0.2 icmp_seq=1 ttl=63 time=3.166 ms
^C
VPCS> ping 10.99.0.184 bytes from 10.99.0.1 icmp_seq=1 ttl=64 time=1.025 ms
^C
VPCS> ping 192.168.122.18084 bytes from 192.168.122.180 icmp_seq=1 ttl=63 time=2.851 ms
But if I try to make a ping from one client PC to the other one the pcap is empty.
-
Do you see those successful pings in pcaps?
Those VPCS client machines only have one NIC?
Either they are not sending the pings via pfSense or the pcap is filtering them out.
-
@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:
Do you see those successful pings in pcaps?
Yes, I can see them in caps
Those VPCS client machines only have one NIC?
Yes, each VPCS has only one NIC
Either they are not sending the pings via pfSense or the pcap is filtering them out.
I don't think that pcap is filtering the pings.
Maybe something related to IPSEC tunnel because I just tried to ping pfsense B LAN IP (192.168.120.1) directly from pfsense A LAN IP (192.168.118.1) with no success.During the next minutes, I would like to verify that static route on routerA is working as expected.
-
If you try to ping 192.168.118.1 from the client behind pfSense B you must see those ping packets entering the interface at pfSense B.
What have you named that interface? (em2) Because it isn't the LAN. What firewall rules have you added on it?
-
@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:
If you try to ping 192.168.118.1 from the client behind pfSense B you must see those ping packets entering the interface at pfSense B.
Yes, I can confirm that. But the ping packets have not a response
What have you named that interface? (em2) Because it isn't the LAN. What firewall rules have you added on it?
- em2 interface on pfsenseB is named "ACCESS"
- I added a very open rule "from any to any"
I don't know if it can help, but please note that, in the IPSEC configuration page, I didn't set the name of the subnet "ACCESS" but the networks:
This is for the pfsenseB. I did the same for the pfsenseA (changing the order).
-
Ok, that looks good.
Earlier though you said those pings did not appear in a pcap in pfSense on the Access interface. Is that still the case?
It looks like that pcap was done on the client directly.With that rule if they are arriving it should definitely create at least a state on the Access interface. And since the P2 matches it it should go out over the tunnel creating a state on IPSec too.
Steve
-
@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:
Ok, that looks good.
Earlier though you said those pings did not appear in a pcap in pfSense on the Access interface. Is that still the case?it is no longer happening.
With that rule if they are arriving it should definitely create at least a state on the Access interface. And since the P2 matches it it should go out over the tunnel creating a state on IPSec too.
I just repeated the test pinging 192.168.118.1 from PC with IP 192.168.120.10.
Ping is not working and the states for ACCESS interface are 0.
Attaching the virtual Wireshark to the em0 port of RouterA I see these lines :
168 2022-12-01 11:35:34.467546 192.168.120.10 192.168.118.1 ICMP 146 Echo (ping) request id=0x004b, seq=956/48131, ttl=63 (no response found!)
169 2022-12-01 11:35:34.468024 192.168.122.230 192.168.122.180 ICMP 174 Destination unreachable (Protocol unreachable) -
Ok, the states exist though and on both ACESS and IPSec. You can see both show 193 packets outbound and no replies.
So check the packet counters in the IPSec status again.
I would not expect to be able to see that traffic on the WAN dircetly, it should be encrypted.
However that 193 packet AH state on WAN looks suspicious. Do you have the Phase 2 protocol set to AH instead of ESP at one end?
It should always be ESP there, AH is unencrypted. -
@stephenw10 said in pfSense and IPSEC lan to lan: a big doubt about the correct implementation:
Ok, the states exist though and on both ACESS and IPSec. You can see both show 193 packets outbound and no replies.
So check the packet counters in the IPSec status again.
I would not expect to be able to see that traffic on the WAN dircetly, it should be encrypted.Yes, you are right.
However that 193 packet AH state on WAN looks suspicious. Do you have the Phase 2 protocol set to AH instead of ESP at one end?
It should always be ESP there, AH is unencrypted.P2 protocol was the same on both endpoints (it was AH, you are right).
Anyway, everything works as expected now. It was a Nat problem on the RouterA.
Many thanks for your help. You are a genius :)As soon as possible I will share what was happening. right now I'm travelling
-
Hello Steve, @stephenw10
a last question for you about this case.
I was able to deploy the IPSEC Lan to Lan successfully (thanks to you).
Now it works using the transport network as you suggested. It is almost done.Anyway, the throughput is a little bit slow (280Mbits/s but 1Gb/s link is available). I noticed that, during the previous tests between two PFsense 2.6CE instances, the throughput was about 900 Mbits/s. The test instances had each one a public IP.
Now, as in the previous test, AES-NI is enabled on both the endpoints, but the pfsense CE versions are different (2.6 and 2.5.2).
Do you think that this could be the reason of the poor performance?In one of my previous post, a user said:
" If AES_NI is in the game pfSense since 2.6 CE or Plus version will benefit from that"
Thanks
-
It's probably just a factor of the latency between the sites. I imagine it's significantly higher than in your test setup?
How are you testing the throughput?
Steve
-
Good Morning Steve,
here some additional info about the test and real scenario.
TEST environment details:
- n. 2 virtual machines with pfSense 2.6 CE with public IPs on WAN interfaces (let's say, 1.1.1.1/25 and 2.2.2.2/25);
- both the virtual machines have been deployed on two VMware ESXi 6.7 U3 hypervisors (one per site);
- the test has been made in order to test IPSEC feature and connectivity
- both the hypervisors have AES-NI enabled on bios
- both the virtual machines have the NetGate suggested configuration (AES-GCM, and so on).
- the throughput has been checked using iperf2 (iperf2 client running in the LAN at site A, iperf2 server running in the LAN at site B)
- the throughput is very near the 1Gb/s link capacity.
REAL/PRODUCTION environment:
- n. 2 virtual machines, one with pfSense 2.6 CE and the other with 2.5.2 CE;
- only one pfsense (at site B) has a public IPs on WAN interface (let's say, 2.2.2.3/25). The other one is behind the router (please, refer to the schema above).
- both the virtual machines have been deployed on two VMware ESXi 6.7 U3 hypervisors (one per site) - the same hypervisors mentioned above;
- both the hypervisors have AES-NI enabled on bios
- both the virtual machines have the NetGate suggested configuration (AES-GCM, and so on).
- the throughput has been checked using iperf2 (iperf2 client running in the LAN at site A, iperf2 server running in the LAN at site B)
- the throughput is about 280Mbits/s.
- please, note that the iperf test between the router WAN interface (1.1.1.2/25) at site A and the pfsense Wan interface (2.2.2.3/25) at site B returns 890 Mbits/s.
Do you think that the router at site A is doing something bad? Or it is really due to the different versions of pfSense?
Thank you,
Mauro -
Are those two environments actually between the same sites?
What is the latency across the tunnel in each case? Or just between the WANs outside the tunnel?
What were the iperf test conditions? Single TCP stream?
If you can pass close to 900Mbps outside the tunnel I would expect to see better inside it. That test does not include pfSense at all though at site A? The router at site A could be restricting something.
Is there any reason one of those VMs is still running 2.5.2?
Steve
