2.5 with many tunnels - Apply Changes fails
-
Anything in the system or IPsec logs at all?
There isn't much to go on here.
-
It is really busy. Constantly get:
logfile turned over due to size>500K
Modifying the log setting will constantly get the same timeout. Perhaps that's the issue. How do I silence it?
-
Is there a mapping of log entries in the UI so that I can turn down specific log entries after persistent retries?
app: applications other than daemons asn: Low-level encoding/decoding (ASN.1, X.509 etc.) cfg: Configuration management and plugins chd: CHILD_SA/IPsec SA dmn: Main daemon setup/cleanup/signal handling enc: Packet encoding/decoding encryption/decryption operations esp: libipsec library messages ike: IKE_SA/ISAKMP SA imc: Integrity Measurement Collector imv: Integrity Measurement Verifier job: Jobs queuing/processing and thread pool management knl: IPsec/Networking kernel interface lib: libstrongswan library messages mgr: IKE_SA manager, handling synchronization for IKE_SA access net: IKE network communication pts: Platform Trust Service tls: libtls library messages tnc: Trusted Network Connect
This help link seems to go nowhere:
https://docs.netgate.com/pfsense/en/latest/book/ipsec/ipsec-advanced-settings.html -
That link should go to https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/advanced.html
There isn't a detailed list of what is logged where, though you can probably turn most things down to silent if you want to focus on just one part of the log.
See also https://wiki.strongswan.org/projects/strongswan/wiki/LoggerConfiguration
-
So I guess on an related issue the ? go the wrong URL.
I was able to finally turn down the logging and I cannot find any errors in IPSec log.
Here are some entries in System log:
2021/02/18 13:38:18 [error] 51390#100109: *64336 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 5.6.7.8, server: , request: "POST /vpn_ipsec_settings.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "fwname.example.com:1234", referrer: "https://fwname.example.com:1234/vpn_ipsec_settings.php"
-
Can you watch the console as the firewall boots up and see how long it takes to configure IPsec at boot time?
-
That is hard to do as it is an AWS instance, but I will try.
-
It was stuck on this screen for about 4 minutes.
Please let me know if you need anything else.
-
That
g_vfs_done
line implies it's having problems communicating with its storage.You may have some other problem in your hypervisor/guest config which is causing it to perform slower than it should, leading to your timerout.
-
Only post 2.5 upgrade and only for IpSec tunnels?
-
Yes, it's common for hypervisors to need adjustments when moving from one FreeBSD version to another, depending on your settings.
It may not be related to IPsec at all, just that it's something that is time consuming and provokes the general slowness.
-
Thank you.
What is your recommendation to correct this?
-
That depends on your hypervisor and guest settings, I don't have any general recommendations there other than to check what your hypervisor recommends for use with FreeBSD 12.2 (or at least 12.x).
-
This is an AWS instance with your approved image and size. This is a c5n.large.
-
Then I suggest you redeploy it instead of upgrading in-place to see if the problem happens that way.
-
When I load the XML file into a new instance, your software produces the same timeout issue and the XML never loads.
Any other ideas?
-
Also, after a few attempts it loaded. It continues to hang in the same spot during boot up, and actually does not boot up at all. I don't think that its hardware or disk related.
Please let me know how you would like to proceed.
-
Is there any way you can try at least loading the IPsec portion of that configuration in a non-AWS system?
It's difficult to tell if it's related to IPsec or if there is a general problem with AWS.
The only other potentially-related report I've seen is a report of a kernel panic on AWS with someone that has even more IPsec tunnels than you, but as far as I'm aware they were not experiencing any slowness or boot delays.