IPSec not working between SG1100s
-
Hi,
A few days ago, I purchased SG1100 to replace the HP PC that served me well if we exclude issues with RTL cards.
My current setup uses site2site and remote access model using IPSec. It used to work well on the HP, but that's not the case with the netgate appliance. Considering there were no issues between HP and site2's SG1100, I have to assume the problem lies on my side.
My IPSec configuration is based on documentation provided by Netgate. Essentially AES-GCM256 with 128bit length and DH14 on both sides. At first, I thought it could be an issue with the crypto accelerator, but it seems like that's not the case. Changing it to i.e. blowfish doesn't make any change.
What is definitely not an issue?
- P1 & P2 - the PSKs and crypto settings are ok, no errors/warnings/notifications
- Not firewall rules - I have opened both sides fully to verify if 500/4500 UDP are open - and they are.
- With no firewall rules applied, two hosts can communicate between each other (so IPs are not blocked in any way). Port forwarding works well.
- Crypto hardware
What could be potentially the issue?
- VLANs - no clear reason as to why
- pfSense+ update.
- Something with IKE_SA
Screenshot below represents IPSec logs. Nothing meaningful, just about peer not responding. This was done on my side and it's pretty much the same on other one too. I have tried IKEv1, no luck.
I should mention both appliances run the 21.05.01 version.
Where would be good place to start looking? Thanks, redacid
-
@redacid Well since it worked with you other box, it must be some kind of mis-configuration in your setup.
Why not try and disable “auto created VPN rules”, and create the firewall rule for opening UDP 500/4500 listening on WAN yourself. Enable logging on the rule, and then start from the ground up.
1: Can you see (logged) recieved UDP packets in both ends if you manually transmit UDP 500/4500 packets from the opposite site?
2: If yes, can you see any UDP packets arriving in an attempt to establish the IPsec tunnel? (Try both ends) If not, it must be a mal configured IPSec tunnel setup on the opposite site.
-
@redacid said in IPSec not working between SG1100s:
it's pretty much the same on other one
Is it actually the same?
Check the state table for incoming packets from the other side. pfSense will allow IPSec traffic from all configured remote endpoints by default. But that can be overridden by a user created block rule if you have one.
Did you import the config into the 1100 or re-create it?
Steve
-
Sorry for the delayed answer.
The issue still persists and as a temporary fix I have managed to change the connection ports from 500 to 600 and 4500 to 4600 respectively. This does the trick for site to site connection, however not for remote access.
The partial config was imported into 1100, I was thinking about factory resetting it, but it doesn't seem like a perfect solution for now.
Under states, I could see the :500 used by mobile devices to use wifi-calling. I have tried to disable that and then kill the states, but it doesn't seem to change anything at all.
It almost would feel like UDP 500/4500 is glitched for whatever reason and is not really easy to trace the direct issue.
P.S. I have tried to manually open those ports but there is 0/0kb traffic.
-
@redacid said in IPSec not working between SG1100s:
The partial config was imported into 1100,
That's probably the issue. If it was a config fro an older pfSense version and you did not import the complete config it will not have been updated to the correct config version.
By setting the ports (which was not an option in older versions) it has probably created the correct config. You will probably find it still works if you set them back to the defaults.
Really though you should import the complete config. If you need to make changes to the config first to allow it to import to the SG-1100 we can help you with that.
Steve
-
@stephenw10 I decided to perform a factory reset on sg1100 and try to do the IPSec from scratch - I can confirm that this does not solve the issue at all. I was hoping WiFi calling would be working, but nada. Out of desperation, I have plugged an old PC that had pfsense installed and IPSec/wifi calling are working flawlessly.
Not sure what could be the issue, perhaps default sg1100's VLANs for whatever reason?
-
So the IPSec tunnels still don't connect? Or just that you are routing wifi calling over that and it fails?
-
@stephenw10 Neither. IPSec still don't connect unless I change ports to 600 on sg1000, phones are trying to connect to career's vpn and they do use port 500 with status SINGLE:NO_TRAFFIC. Disabling wifi-calling on the phones and then freeing port 500 from states does not change anything - it still doesn't work.
-
Hmm, the only thing pfSense does that could be affecting anything is the static source ports for outbound port 500. That is the default setting though so would apply equally to an 1100 and a CE install. Also it would only be an issue if somehow the wifi calling was holding open a port 500 state to the remote tunnel IP which I wouldn't expect it to ever connect to.
Reviewing; did you see port 500 or 4500 traffic arriving from the remote side of the tunnel?
Are those 1100s connected with public IPs directly or behind some other router?
Steve
-
Currently, I've disabled any IPSec (both remote and site to site) and will focus on WiFi-Calling as it seems to be the most independent service - technically if WiFi-Calling would work, the rest will follow.
As you've mentioned, states are created but no actual connection is made (I guess). There are currently two Apple devices that attempt to connect to the career's port 500, both of them fail.
And no, when I had site2site and/or remote access enabled, despite making the connection, I have never seen incoming connections being allocated in the states table. Oddly enough, if IPSec was working site2site, RemoteAccess would never work and just throw configuration error - I suppose anything other than UDP 500 will break stuff in most cases.
I did notice, that during the RemoteAccess connection, instead of instantly throwing a configuration error / refusing connection it simply timed out, again, never appeared in the states table.
Both SG1100s are connected directly to an intermediate device (modem in this case) in a bridge mode. There are no double-nat occurring or subnet collisions with ISPs - we have even tried 192.168.x subnet. I did narrow the issue to my unit though.
P.S. I was just about to send this reply when I noticed 4500 incoming in States table, but with no real traffic - it disappeared fairly quickly anyway. Random botnet I guess. The apple devices are still retrying to connect to the career's IPsec service with no luck.
-
Hmm, OK. You can see the wifi calling devices creating outbound states there but there is no reply traffic at all. At least none that matched those states.
When you have multiple devices connecting the same external IP and outbound NAT is using static source ports you can see a conflict as the WAN side state is identical. That's normally not an issue since the connection switches to UDP/4500 once it's established.
I would try disabling the static source port NAT rule for port 500 and see if that makes any difference. Most recent VPN devices can cope with random source ports anyway.I would run a packet capture at both ends of the site to site tunnel to be sure you are seeing any port 500 traffic arrive at all.
Normally only traffic from configured end points is allowed but if you have a remote access tunnel setup that obviously allows traffic from any remote IP.
Steve
-
@stephenw10 I can confirm 4500 port would work, since I have tried using my work VPN today, which does use IPSec as well. Turns out 4500 was never the issue at all.
Static Port Mapping has been disabled on the outbound NAT and now outgoing connections are used randomized ports. This however still doesn't fix the issue, so there is still NO_TRAFFIC:SINGLE or SINGLE:NO_TRAFFIC on 500 port equivalents.
-
If you Upgrade from a older Version to 21.05.1 it is possible to run into a Crypto Bug.
Shutdown your Netgate, unplug the Power for 1 minut and start up again. Try the VPN Tunnel now.
-
I have tried doing so and then even factory resetting again - nothing has changed. I've plugged the HP-pfsense back and works like a charm. Does that mean it's time for RMA?
-
That crypto hardware issue would not effect IPSec, that device is only used to access authenticate the repos.
Even if you were seeing an issue with the safexcel hardware crypto, which IPSec can use, it would not prevent the tunnel connecting at all.When you are swapping back the other pfSense device are you doing that at both ends? Or is one 1100 working as expected?
Steve
-
@stephenw10 the remote sg1100 is working as expected. It’s just mine unit that seems to not be working.
-
The only thing that could present a difference here is the hardware crypto in the safexcel driver. But you said you tried using a cipher that does not effect (blowfish) so it can't be that directly.
So I'm left trying to think of something you might have had set in the old device that's somehow incompatible with the SG-1100. I can't see what that could be though.
The fact setting the tunnel to use ports 600/4600 allowed it to come up implies something in the path blocking the standard ports. The crypto hardware doesn't care what ports are in use for example.
It really 'feels' like the upstream device trying to do something clever with IPSec traffic.
Are we able to review the config you are importing to the 1100? If you open a ticket with us and reference this thread the guys will make sure I see it.
It's hard to see how this could be a hardware issue. If we swapped it out I would expect another device to do exactly the same thing given the same config.
Steve
