IPSec Down after Upgrade to 2.3

choudharyprabhat

Hi All ,

I am a newbie in pfsense , i have recently updated on pfsense 2.3 (AMD 64) ,all the things were working like a charm including IPSEC tunnels before installing OpenBgpd, as per our requirement i have installed OpenBGPD both IPsec and OpenBgpd worked for some hours ….....then all the IPSEC tunnels gone down , In Gui configuration tunnels still showing in established state but no traffic passing , for fixing the issue i have to restart the firewall , then it again starts working for some hours .

I have tried these below steps also but all goes in vain :'(

killall -9 charon
killall -9 starter
ipsec stop
ipsec start

ipsec start states as :-

Starting strongSwan 5.4.0 IPsec [starter]…
charon is already running (/var/run/charon.pid exists) -- skipping daemon start
no netkey IPsec stack detected
no KLIPS IPsec stack detected
no known IPsec stack detected, ignoring!
starter is already running (/var/run/starter.charon.pid exists) -- no fork done

Please suggest anybody i am using pfsense in production environment :'(

I was wondering if a downgrade to previous version can fix this issue , is any version is workable with both IPSEC and OPENBGPD.
Please if anyone have any idea on this .

fattylewis

@choudharyprabhat:

Hi All ,

I am a newbie in pfsense , i have recently updated on pfsense 2.3 (AMD 64) ,all the things were working like a charm including IPSEC tunnels before installing OpenBgpd, as per our requirement i have installed OpenBGPD both IPsec and OpenBgpd worked for some hours ….....then all the IPSEC tunnels gone down , In Gui configuration tunnels still showing in established state but no traffic passing , for fixing the issue i have to restart the firewall , then it again starts working for some hours .

I have tried these below steps also but all goes in vain :'(

killall -9 charon
killall -9 starter
ipsec stop
ipsec start

ipsec start states as :-

Starting strongSwan 5.4.0 IPsec [starter]…
charon is already running (/var/run/charon.pid exists) -- skipping daemon start
no netkey IPsec stack detected
no KLIPS IPsec stack detected
no known IPsec stack detected, ignoring!
starter is already running (/var/run/starter.charon.pid exists) -- no fork done

Please suggest anybody i am using pfsense in production environment :'(

I was wondering if a downgrade to previous version can fix this issue , is any version is workable with both IPSEC and OPENBGPD.
Please if anyone have any idea on this .

Dude, you shouldnt have updated a prod system to 2.3 without testing!

Anyway, yes i can confirm 2.2.6 works perfectly with IPSEC and openbgpd. Im using it myself on a prod network.

There is an open bug report for this issue: https://redmine.pfsense.org/issues/6223

choudharyprabhat

Thanks fattylewis , i have downgraded my pfsense box to 2.2.6 , now everything is working fine. :) ;D

there is one more thing to notice :- when i had edited /boot/loder.config.local >>>> net.inet.raw.maxdgram="131072"
net.inet.raw.recvspace="131072" .

Pfsense worked like charm with ipsec and bgp even on pfsense 2.3. :)

for me that trick worked.

Thanks CMB and fattylewis for your replies….you guys rocksssss.. ;)

fattylewis

@choudharyprabhat:

Thanks fattylewis , i have downgraded my pfsense box to 2.2.6 , now everything is working fine. :) ;D

there is one more thing to notice :- when i had edited /boot/loder.config.local >>>> net.inet.raw.maxdgram="131072"
net.inet.raw.recvspace="131072" .

Pfsense worked like charm with ipsec and bgp even on pfsense 2.3. :)

for me that trick worked.

Thanks CMB and fattylewis for your replies….you guys rocksssss.. ;)

Oh, nice find. Ill see about knocking up another network on 2.3 and adding your change and seeing what happens.

timw

We've also had this issue on 2.3, and as we required BGP for our network, we've downgraded back to 2.2.6

Looking forward to a confirmed fix (need to wait until after hours again to try the upgrade again)

studioelement

I'm having the same problem with OpenBGP and IPSec.

Restarted the following services:
-OpenBGP
-IPSec

No luck. Only rebooting worked.

Then tried restarting:
-OpenBGP
-IPSec
-OpenVPN

Tunnel came back up.

Not sure if that helps some of the developers with troubleshooting.

I have stopped the OpenVPN service for now and will see if the issue returns.

UPDATE: Still having the issue even after disabling OpenVPN

obrienmd

Same issue, pair of SG-8860s with CARP failover, dual IPSec tunnels to a Verizon Private Network with OpenBGPd required for their routing. Exact same errors, even changing both tunables:

net.inet.raw.maxdgram="131072"
net.inet.raw.recvspace="131072"

May extend the time, but definitely doesn't solve. Really don't want to go back to 2.2.6 :)

cmb

@obrienmd:

Same issue, pair of SG-8860s with CARP failover, dual IPSec tunnels to a Verizon Private Network with OpenBGPd required for their routing. Exact same errors, even changing both tunables:

net.inet.raw.maxdgram="131072"
net.inet.raw.recvspace="131072"

It's not just those two. Add:

net.raw.recvspace=65535
net.raw.sendspace=65535

jnorell

FWIW, still seeing this problem here. Yesterday I updated to 2.3.1 and also set these:

@cmb:

net.inet.raw.maxdgram="131072"
net.inet.raw.recvspace="131072"
net.raw.recvspace=65535
net.raw.sendspace=65535

I just bumped those up higher hoping it will help, but at least for us neither the 2.3.1 update nor those specific values fixed it. Does it matter if they're set at System > Advanced > System Tunables rather than in loader.config.local?

jnorell

We've now been up for over a week with these settings (set in System > Advanced > System Tunables):

net.inet.raw.maxdgram 131072
net.inet.raw.recvspace 1048576
net.raw.recvspace 1048576
net.raw.sendspace 1048576

Edit: up over 2 weeks now, still no problem

augustinermonch

Hi I'm new here and have a problem with my PFSense and the IPsec connection .

The environment :
Location A pfsense 2.3.1_1
Location B pfsense 2.3.1_1

Connected via IPSec " SitetoSite "

I tried all the tips from this thread. Unfortunately without success.

Like
changeing net.inet.raw.maxdgram 131072
net.inet.raw.recvspace 1048576
net.raw.recvspace 1048576
net.raw.sendspace 1048576

The problem is when I try to access Site B about RMTC works without problems .
However, if I want to print a print job from B to site A drops the connection and restarts.

Does somebody has any idea ?

I'm a bit desperate .

Thank you very much

I Forget to say that it works perfect before i updatet my pfsense …

augustinermonch

Hi it´s me again, i tryed to use OPENVPN instead of IPSEC
I have the same Problem and my PFSENSE reboot new after 2 min.

Does anyone know this situation ?

Kamyk

Hello everybody!

I have read that thread but unfortunately I have the same issue. We use PfSense 2.3.1 with OpenBGPD+IPsec to Amazon AWS.

We have set that:


net.inet.raw.maxdgram="131072"
net.inet.raw.recvspace="131072"
net.raw.recvspace=65535
net.raw.sendspace=65535

Our IPsec disconnect every couple hours. When I check IPsec status - looks ok, but I can not transfer any packets. I don't have to reboot Firewalls but only stop OpenBGPD and IPsec. Start again and all is working again ok for next couple of hours.

Do you have any idea what I can check more? I didn't check that fix from GitHub. But do you think it could be it?

Thank you for any help or answer.

Best,
Kamyk

olobley

I run a couple of pfsense boxes to link my house to a few neighbors (so hardly mission critical).
Since the upgrade to 2.3, then 2.3.1, then 2.3.1p1, my IPsec tunnels haven't worked.
I don't run OpenBGP (at least I don't think I do) and I tried applying the System Tuneables that jnorell suggested.
I also tried purging all my VPN configurations, and recreating them. Still no love :(
What's odd (at least to me), is that all the tunnels come up in the web interface, but they don't pass traffic.
It's not the end of the world, as I moved to OpenVPN in the interim, however I'd prefer to get back to IPsec.
Thanks in advance

JorgeOliveira

@Kamyk:

I have read that thread but unfortunately I have the same issue. We use PfSense 2.3.1 with OpenBGPD+IPsec to Amazon AWS.

Known issue: https://redmine.pfsense.org/issues/6223

jnorell

@olobley:

Since the upgrade to 2.3, then 2.3.1, then 2.3.1p1, my IPsec tunnels haven't worked.
…
What's odd (at least to me), is that all the tunnels come up in the web interface, but they don't pass traffic.

It sounds like you have a different problem (try enabling cisco extentions in ipsec advanced settings), this one is indicated by 'No buffer space available' errors in the logs.

jnorell

Today, after almost 29 days uptime, we're getting 'error sending to PF_KEY socket: No buffer space available' again .. I'm bumping settings up some more:


net.inet.raw.maxdgram = 131072
net.inet.raw.recvspace = 1048576
net.raw.recvspace = 1048576
net.raw.sendspace = 2097152

mraymond

Hello, new to this forum. Just throwing my hat in the ring for this issue as well. Plagued by "error sending to PF_KEY socket: No buffer space available".
I'm using three IPsec tunnels. One to AWS (with BGP), one to Azure, one to a mikrotik router at a remote office.

Is there a way to effectively restart IPsec and flush that buffer without rebooting?
Restarting the service via the GUI, or manually killing charon and starter and restarting ipsec via terminal does not do it.

EDIT: Of course I should mention this problem started happening after upgrading from 2.2.(6?) to 2.3.1_1
I have increased
net.inet.raw.maxdgram
net.inet.raw.recvspace
net.raw.recvspace
net.raw.sendspace

to recommended values, but have not rebooted since. I will reboot late tonight.

obrienmd

Same issue with upgrade to 2.3.1_5, any idea if this will be resolved in 2.3.2 or 2.4.x (FreeBSD 11, right?)

cmb

Not something we're going to have time for in 2.3.2 (release next week), hopefully it's either resolved already in FreeBSD 11, so 2.4 will be fine, or someone can track down the root cause and get it fixed (my last day here is in two weeks).

2.4 snapshots should be out soon. Help testing then would be appreciated.