Converstion of backup file from 2220 to 4100

LarryM04

Ran a 2220 for a couple years, paid for the support team to set up a pair of OpenVPN connections with WatchDog so traffic would always go thru one or the other VPNs, never exposing my "real" IP. Worked fine. Sadly, the 2220 died and at the suggestion of the Netgate sales team, upgraded to a 4100. They told me that support would convert my backup file so it could be read into the 4100 and I'd be right back in business. So that's what I did. But now that I have the 4100 (with the updated backup read in) there are little "flickers" when my real IP address is exposed. I'm not great a shell scripts, but I'm running this:

LASTIP=`wget http://ipinfo.io/ip -qO -`
while true
do
	IP=`wget http://ipinfo.io/ip -qO -`
	if [ ${IP} != ${LASTIP} ]
	then
		echo `date` ${IP} ${LASTIP}
	fi
	LASTIP=${IP}
	sleep 1
done

The output periodically shows the IP changing from one VPN address to the other, and sometimes my real IP. When I had the 2220 I almost never saw a change in my IP address. I think something wasn't converted properly in the backup file. Support initially showed interest, but now says what I'm asking isn't covered by Lite support.

Is there any hope?

Edit: I've checked the OpenVPN status and both clients are running continuously, no indication that one or the other when down necessitating a switch

SteveITS

@larrym04 The config file is XML, can you run a comparison program to see the differences? I wouldn't run it through a web site as it will contain passwords.

LarryM04

@steveits The file is 1244 lines, diff reports 1243 different lines.

I can edit the passwords, if that would help.

stephenw10

Hmm, if neither went down then there shouldn't be a switch at all I would think.

Are you running that script on the 41000 directly or a client behind it?

Were you running that script on the SG-2220?

What pfSense version were you running on the SG-2220?

Steve

LarryM04

@stephenw10
Ya, I don't understand the "flicker" either since the interfaces didn't go down.

I'm running that shell script on one of the PCs behind the 4100. I normally see the IP addr in a conky display that runs that same line. I split it off into a shell script so I could add the timestamp and get the history. On the 2220 I almost never saw the IP change except the few times the server (other side) did indeed go down.

Okay, ran a different kind of diff and find lots of lines that a similar to this on the 2220: "<description><![CDATA[All Users]]></description>" changed to this on the 4100: "<description>All Users</description>" and the WAN/LAN names. So why does the 4100 act different?

I don't see any deformities in either file, but when I initially loaded the 4100 the openvp2 had some extra characters at the start of Advanced Configuration Custom Options, I know because I had to edit them out (via the web page, not the .xml file) before the interface would come up. But I don't see any extra chars in the xml file, so where did they come from?

As far as version on the 2220, sorry don't know. The box itself was about 2-3 years old and I think I was good about keeping it updated

LarryM04

@stephenw10 Could there be something wrong with the hardware?

stephenw10

It's very unlikely to be a hardware issue. Anything in hardware wouldn't present like this.

It's much more likely to be a different pfSense version where the config behaves slightly differently. For your porposes the SG-2220 and the 4100 are very similar. I would expect a config conversion to be very simple there. In fact I wouldn't expect a conversion to be necessary really. You should be able to import the old config and it will ask you to reassign WAN and LAN and then boot into it.

Do you have the ticket number this was done on so I can review the config?

Steve

LarryM04

@stephenw10 I don't see a ticket number mentioned in any of the emails that have gone back and forth. I can tell you that Georgiy was the one that converted and sent the file. Another name in the emails from Netgear TAC was Lev Prokofev who did seem to be working with me, but then another got involved and the conversation turned... one of the last comments was that this wasn't supported in Lite and I'd have to buy a better support package. That would be fine if I thought I'd done something to mess it up, but its clearly not working the way it should.

The 4100 was bought with Order SO22-44374 if that helps

thanks, Larry

stephenw10

Ok, let me look for it....

Edit: Found it. For reference your ticket ID was: 1036190288

Checking it now...

LarryM04

@stephenw10 There were two tickets open when they initially were trying to help me get the 2220 working 1029506379 and 1036190288. I think the 2nd one may have been for hte conversion

LarryM04

@stephenw10 FYI - Just sent a new email to support with an explanation of what's happened from the beginning. It got ticket 1078056888

stephenw10

Yes, OK the issue here is that old config from the 2220 was from a much older version. 2.4.3p1 from 2018.

There's no real problem with importing a config that old but since then the WAN failover handling has changed significantly.

Looking through your config there is one key setting you don't appear to have.

In Sys > Adv > Misc set 'Skip rules when gateway is down'.

Without that is both OpenVPN gateways show as down then your policy routing rule on LAN will be created without a gateway set and traffic will just leave via the default route, the WAN.

Steve

LarryM04

@stephenw10 So I want this checked on?

Skip rules when gateway is down
Do not create rules when gateway is down By default, when a rule has a gateway specified and this gateway is down, the rule is created omitting the gateway. This option overrides that behavior by omitting the entire rule instead.

stephenw10

Yes. Otherwise it creates the rule but without any gateway set which passes it out the WAN.

LarryM04

@stephenw10 Got an email from Kris saying that I deserve Professional level support, sorry I wasn't getting it, and instructions to let them come in and review my 4100. I think your fix will keep my raw ip from leaking out, but I would like to know why I was seeing that flicker which hasn't happened in a while now

Thank you very much for your assistance

stephenw10

No worries let me know how it goes.

The only way you could have been seeing that is if the firewall was opening connections to the WAN directly. And the only way that could happen, with the rules you have, is if the failover gateway group ended up with no on-line gateways.
You could have hit that in in 2220 but just never did. That could be for a number of reasons but probably the gateway monitoring changes that have gone in since then have made it more likely.

That gateway setting change should prevent it happening again but if for some reason it does not there are a few other things we an add to stop it.

Steve

LarryM04

@stephenw10 In my reply I, of course, gave you credit for the find.

I don't understand the dithering between addrs... If the connection thru both VPNs were up continuously, why would it flip between them? Is there some condition where the interface is up, but the 4100 thinks there is something wrong?

LarryM04

This post is deleted!

stephenw10

If the gateway monitoring stops seeing reply pings it will show as down even if the tunnel is up.

You can tune how it detects that and what it monitors in the gateway settings:
https://docs.netgate.com/pfsense/en/latest/routing/gateway-configure.html#advanced-gateway-settings

The WAN which is considered the 'primary' is set by assigning the tiers in the gateway group. The lowest tier gateway is preferred:
https://docs.netgate.com/pfsense/en/latest/routing/gateway-groups.html#tier-priority-example

Steve

LarryM04

@stephenw10 Ah! So the pings on the primary exceeded the threshold, it flipped over to the secondary, then when the pings worked again on the prim it switches back. Right? The "evil" scenario was the pings stopped on prim, then also stopped on secondary, so went out raw and that was when my real IP showed. Am I catching on?