PfSense 2.4.4 LAN interface stops routing traffic - stops working after some minutes, sometimes hours

[e066377]

I setup IPsec vpn with 9 Phase 2 tunnels that requires to NAT LAN subnet to a specific public IP.
Although the connection is stable, Phase 2 tunnels stop working sometimes after few minutes, sometimes after 7-8 hours.

If i restart a computer in local network or restart IPsec service multiple times and try to ping remote IP of one of the tunnel then tunnels start to get packets but after a while traffic stops again.

People had the same problem before : https://forum.netgate.com/topic/98893/pfsense-2-3-lan-interface-stops-routing-traffic-stops-working-after-2-or-3-day
And it was fixed with 2.3.1, it think it was a solution by disabling all but 1 CPU.

[e066377]

If I enter the Remote IP in Automatically ping host field of P2 Advanced Configuration the bytes and packets out increases periodically with the default frequency even if it stops accepting pings from LAN subnet.

After it stops accepting traffic from LAN subnet I try stopping and starting or restarting IPsec service, and even rebooting pfSense but cannot make it accept traffic from LAN subnet. But only after I restart windows server from which I send traffic in tunnels it starts to work. As if its not an IPsec problem but network problem, firewall forgets local network and it does not apply IPsec routing.

To solve this problem any extra direction would be greatly appreciated!

Thanks.

[e066377]

My network is VMware virtual machines including pfSense, can it be the cause of the problem? Should i install pfSense on a separate hardware?

[Rico]

Did you basically follow the Virtualization Guide? https://www.netgate.com/docs/pfsense/virtualization/virtualizing-pfsense-with-vmware-vsphere-esxi.html
VMware Tools installed?
Tried to switch between E1000 and VMXNET3 or vice versa?

-Rico

[e066377]

Hi Rico,
Thanks for reply.

Unfortunately i don't have access to the host machine to install pfSense myself. Some people in another country has access to the host machine to install or manage virtual machines. I just asked a pfSense machine with public static IP to be used as PEER IP.

dmesg | grep cpu command shows 6 cpu
cpu0: <ACPI CPU> numa-domain 0 on acpi0
cpu0: <ACPI CPU> numa-domain 0 on acpi0
cpu0: <ACPI CPU> numa-domain 0 on acpi0
cpu0: <ACPI CPU> numa-domain 0 on acpi0
cpu0: <ACPI CPU> numa-domain 0 on acpi0
cpu0: <ACPI CPU> numa-domain 0 on acpi0

Should only 1 cpu be assigned?

MY Gateways and Interface and DHCP configurations are as follows

DHCP is enabled on LAN (but not on WAN)

If anything is wrong with these configurations please warn me to correct it, i am new with pfSense

[e066377]

After disabling all IPv6 support in gateway, and interfaces and enabling Split connections in Phase 1 all tunnels are up for several hours. If tunnels can stay up more than 24 hours then i will accept it acceptably stable.

[e066377]

Traffic stops after Phase 1 Lifetime (8 hrs), tunnels do not accept traffic from LAN subnet.

And I could start traffic by deleting DHCP Leases and restating DHCP service. Probably only restarting DHCP would suffice, i will try it next time.

[e066377]

Without deleting DHCP releases restart of DHCP server did not solve the problem, but when I released (ipconfig/release) IP address of Windows Server by which users make use of remote site services connected by IPsec s2s vpn. This Windows Server should stay connected always, but after a while its vpn service requests do not go through tunnels until it is forced to renew its IP.

I can write a program to force Windows Server to renew its IP when it tries and cannot connect vpn services. But it would be a better solution if it can be handled by pfSense.

[e066377]

Here is my solution;

The vpn connection is used to call webservices (soap) of the remote site.
To check every connection I created a derived class of SoapHttpClientProtocol from which webservice references are derived and edited all service references and drived them from new class NWSoapHttpClientProtocol.

C# code

    public class NWSoapHttpClientProtocol : SoapHttpClientProtocol
    {
        protected new object[] Invoke(string methodName, object[] parameters)
        {
            try
            {
                return base.Invoke(methodName, parameters);
            }
            catch
            {
                RenewLocalIP();
                return base.Invoke(methodName, parameters);
            }
        }

        private void RenewLocalIP()
        {
            try
            {
                ProcessStartInfo processStartInfo = new ProcessStartInfo();
                processStartInfo.FileName = "ipconfig";
                processStartInfo.Arguments = "/release ";
                processStartInfo.WindowStyle = ProcessWindowStyle.Hidden;
                Process process = Process.Start(processStartInfo);
                process.WaitForExit();

                processStartInfo = new ProcessStartInfo();
                processStartInfo.FileName = "ipconfig";
                processStartInfo.Arguments = "/renew";
                processStartInfo.WindowStyle = ProcessWindowStyle.Hidden;
                process = Process.Start(processStartInfo);
                process.WaitForExit();
                
                EventLogger.Log(LogType.Information, MethodBase.GetCurrentMethod(), "Renewed Local IP");
            }
            catch (Exception ex)
            {
                while (ex.InnerException != null)
                {
                    ex = ex.InnerException;
                }

                EventLogger.Log(LogType.Error, MethodBase.GetCurrentMethod(), ex.Message);
            }
        }
    }

InvokeAsync methods of SoapHttpClientProtocol can also be implemented.