Internet connexion drop all 10min because my wan IP is on different subnet than default Gateway

Elrick75

Hi to all,

I use the last version of pfSense (v2.4.4, release P3)
My internet connection cuts off every 10 minutes, then goes up 10 minutes after.

Root cause seems to come from my internet provider, it gives me a particular IP configuration.
My wan ip address is different from my gateway, my IP and gateway are not on the same subnet mask.

IP = 185.73.xxx.xxx
Gateway = 185.4.79.254/32 (notice /32 !!)

At the beginning of the configuration, I did not have any internet connection.
I ticked "Use non-local gateway" option in menu "System -> Routing -> Gateways -> Edit".

But it's not enough to keep my connexion alive.
My internet connection cuts off every 10 minutes, then goes up 10 minutes after.

I noticed that some people were using ARPING and CRON to send ARP requests regulary to maintain their connection, but I find this workaround heavy, it is more like do-it-yourself solution.

Does someone know a real stable solution to fix this issue properly? Many thank in advance.
I will be interested to understand the problem, is there a useful log to consult to know more?

Best Regards.

johnpoz

Why do you think this problem is related to the gateway being outside your IP mask? That has zero to do with 10 min cycle..

Does the gateway not answer ping every 10 minutes? This will cause issue your connectivity..

What does your ISP say about this 10 min connectivity?

Elrick75

You are right...my ISP is only suggest me to tick "Use non-local gateway" in pfSense.
It doesn't answer me about this 10 min connectivity.

You may have any idea to help me to troubleshoot connexion drop with pfsense ?

Best Regards.

Gertjan

Tell us more about your WAN interface.
It has static settings ? DHCP ?
The interface goes down - or connection is just lost ?
What happens when you invert the LAN and WAN assignment ?

What is is drivers name of the interface ?
Bare bone hardware or VM ?
What shows up in the logs ( except the pinger complaining about ping loss) ?

Elrick75

Wan interface is in DHCP mode

Connection up or not, status is always offline

Gateway logfile, i start my PC at 21:31, but no more trace after 2 hours start... but it cut off every 10 minutes, then goes up 10 minutes after my connexion

Many thanks in advance for your help.

chpalmer

@Elrick75 said in Internet connexion drop all 10min because my wan IP is on different subnet than default Gateway:

Gateway = 185.4.79.254/32 (notice /32 !!)

What gives you the idea that that gateway only has a /32?

Elrick75

It's written under my pfSense page when i start my server.
When it display me all current interface, i see my "wan ip address/32"

It has static settings ? DHCP ? > DHCP
The interface goes down - or connection is just lost ? Cannot say exactly... ping and dns resolve no more working
What happens when you invert the LAN and WAN assignment ? Not tested

What is is drivers name of the interface ? How to find it ?
Bare bone hardware or VM ? Dedicated PC (Dell R230), no VM
What shows up in the logs ( except the pinger complaining about ping loss) ? See in my previous post below Gateway log

Elrick75

Any idea ?

chpalmer

@Elrick75 said in Internet connexion drop all 10min because my wan IP is on different subnet than default Gateway:

When it display me all current interface, i see my "wan ip address/32"

Nothing anywhere says your gateway address is a /32 and my bet is it is not. Your WAN address definitely is.

What kind of internet connection is this and who is your ISP? Im to lazy to look.

johnpoz

The gateway is an IP, pfsense would not have a mask associated with it ;)

Pfsense can have a mask associated with an IP that is on one if its interfaces, but not a dest IP. A route would have a mask, a firewall rule would have a mask... but a gateway would not have a mask..

Nor do I get why you think your IP having a /32 would have anything to do with a 10 min cycle of connectivity.. The default arp cache time in pfsense is 20 minutes.. So its not related to loss of mac, and even if it was it wouldn't take 10 mintues to come back.

If arp was your problem - you could always just create a static arp ;)

What is the lease time your getting wan IP? look in your /var/db dir you should see a dhclient.leases.interface like mine is .igb1

renew 5 2019/7/19 01:55:12;
rebind 5 2019/7/19 10:55:12;
expire 5 2019/7/19 13:55:12;

So you can see when its schedule to renew, when it will expire, etc. So what does yours show for your lease on your IP on your wan you got via dhcp?

Elrick75

@chpalmer You have "my ip/32 in" just on screenshot copy below with all interface.

@johnpoz That's my ip address who have /32, not the gateway, but the gateway ip is outside the subnet of my wan ip address.
My isp provider is K-Net.

I know that some person use ARPING and CRON to send ARP requests regulary to maintain their connection (avoid connexion drop).
They suggest to use this command every 5 minutes :
*/5 * * * * root /usr/local/sbin/arping -v -c 3 -i em1 185.4.79.254

See below DHCP log after start... after 10 minutes, when my connexion is not working, i have no trace in it, same things when my connexion is come back., no more log

I edit the file suggested on my WAN interface from command line, here are the result, as you see, lease time is 43200 and didn't match drop connexion timing related.

Elrick75

My isp switch my IP configuration from /32 to /24 but it doesn't fix drop connexion every 10 minutes.
My isp give me another tips to fix this issue, more clean than ARPING every 5 minutes to keep alive my connexion.

I share below the method:

Go to System > Advanced

Select System Tunables

Click on New button

Enter net.link.ether.inet.max_age as Tunable
Enter 300 as Value and enter a description
Click on Save button

And voilà, no more connexion drop after 10min.

johnpoz

You do understand you just lowered the arp cache time from default 20 minutes to 5 minutes right. Is the mac of their IP (your gateway) changing?

[2.4.4-RELEASE][admin@sg4860.local.lan]/root: sysctl net.link.ether.inet.max_age
net.link.ether.inet.max_age: 1200
[2.4.4-RELEASE][admin@sg4860.local.lan]/root:

Elrick75

Yes, they change my IP from /24 to /32 but WAN IP setup was not the root cause because the drop connexion were still present.
For unknow reason, adding this line solve my drop connexion issue that i have.

johnpoz

Not talking about your IP or the mask... I am talking if the mac address of your "gateway" is changing??

So any tcp device that wants to talk to an IP that is on its own network, or told that is via that hey gateway is on another network check box will ARP.. Hey who has 192.168.1.1 for example - and you will get back the mac address 00:11:22:33:44:55..

Pfsense out of the box will "cache" this for 20 minutes.. So it will not have to arp again for this mac address until that is expired.. So any time it needs to send traffic to 192.168.1.1 it knows what the mac address is.

If your gateway which is going to be on the same L2 as you!!! mac address is change then yeah your going to have all kinds of problems... But the mac should NEVER change!! Your setting told pfsense hey only cache that mac for 5 minutes not 20, so its going to have to arp more often.. But the mac should not be changing every X minutes... It should be the same forever, until they actually changed the hardware of their gateway, etc..

So what I am asking you is the mac of their gateway IP... What is your gateway IP? Look in your arp table - what is its mac? Check that every so often for a while - is the mac changing??

Maybe their end is dropping your mac, unless it sees an arp from you? But there is really no reason you should have to lower the arp cache time for your gateway.. If anything you should be able make it longer ;)

example her is mac of my gateway

d53-64<snipped>.nap.wideopenwest.com (64.53.xxx.xxx) at 00:01:5c:82:36:46 on igb1 expires in 914 seconds [ethernet]

Elrick75

Mac address of my gateway seems the same than the previous gateway than i had before switching from /32 to 24.

Old WAN gateway was 185.4.79.254, mac = 74:8e:f8:63:e7:c1
New WAN gateway 185.109.201.254, mac = 74:8e:f8:63:e7:c1

For now, i monitor ARP table, when it expire, it remain the same mac address again.

What do you think about it ?

johnpoz

Well then you changing the cache default time makes no sense how it could fix anything..

Have your isp explain what that setting "fixes" If the mac doesn't change then your cache could be for 10 years ;)

Seems like your isp wants to see arps more often than every 20 minutes for whatever reason?