Pfsense can`t keep connection alive to provider
-
Hello pfsensers,
I love to use my pfsense. I think it is a great peace of software but now I have some problems with it since a cupple months. The current problem is that the pfsense can`t keep the connection to my provider.
My configuration is:
I am running my pfsense (2.4.4 release p2) on a Super Micro C2758. I got my 100MBit cable connection from Vodaphone. The router of my provider is in bridge mode and the WAN interface of my pfsense is configured as DHCP.The history to this cause is:
This configuration was doing very well until a couple months. Out of the blue the internet connection has gotten unstable. So I restarted the WAN interafce and the show could go on. After some days I configured the speed of the connection of the WAN interface to a permanent value, 1000 base... . After that I got no more trouble.
Since a some weeks the internet connection became unstable again. Unfortunately restarting only of the WAN interface brings no benefit. Now I have to restart the whole system.
My first move as to set the permanent value to the default (autoselect), nothing changed. My second move was that I activated some watchdogs (dpinger, unbound, dhcpd) and that went semi good. When the cause arrives the watchdogs where permanently restarted and so I got a connection with a short break after each 5 min. So I decided to remove the watchdogs. Now, if the cause appears, my connection is down and stays down.
My guess was that this is a problem of my provider. After some investigation I discovered that, if I reboot my router the issue is the same. The pfsense can`t establish the the connection. So I could reproduce this problem and the conclusion is, that it is not my provider who is causing this.To buy me some time and get a permanent happy wife I wrote a script. This script pings google and checks if the connection is alive. If not it restarts the system and writes an entry in a log file. Now the it restarts two or three times a day.
Unfortunately I did not found any error entries in the pfsense logs since I use this script. Except shortly before the restart I recognized some package loss.
Mar 3 00:46:57 dpinger send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 77.21.62.254 bind_addr 77.21.62.93 identifier "WAN_DHCP " Mar 3 00:46:53 dpinger send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 77.21.62.254 bind_addr 77.21.62.93 identifier "WAN_DHCP "
Before I activated the script I got this.
Feb 24 19:05:29 php-fpm 63855 /rc.linkup: DEVD Ethernet detached event for wan Feb 24 19:05:29 check_reload_status Reloading filter Feb 24 19:05:29 check_reload_status updating dyndns wan Feb 24 19:05:27 xinetd 55463 Reconfigured: new=0 old=1 dropped=0 (services) Feb 24 19:05:27 xinetd 55463 readjusting service 6969-udp Feb 24 19:05:27 xinetd 55463 Swapping defaults Feb 24 19:05:27 xinetd 55463 Starting reconfiguration Feb 24 19:05:25 php-fpm 339 /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1551031525] unbound[90440:0] error: can't bind socket: Address already in use for 127.0.0.1 port 953 [1551031525] unbound[90440:0] error: cannot open control interface 127.0.0.1 953 [1551031525] unbound[90440:0] fatal error: could not open ports' Feb 24 19:05:25 php-fpm 85913 /rc.newwanip: rc.newwanip: on (IP address: 77.21.62.93) (interface: WAN[wan]) (real interface: igb1). Feb 24 19:05:25 php-fpm 85913 /rc.newwanip: rc.newwanip: Info: starting on igb1. Feb 24 19:05:23 check_reload_status Reloading filter Feb 24 19:05:22 php-fpm 338 /rc.start_packages: [squid] Starting a proxy monitor script Feb 24 19:05:22 xinetd 55463 Reconfigured: new=0 old=1 dropped=0 (services) Feb 24 19:05:22 xinetd 55463 readjusting service 6969-udp Feb 24 19:05:22 xinetd 55463 Swapping defaults Feb 24 19:05:22 xinetd 55463 Starting reconfiguration Feb 24 19:05:22 check_reload_status Restarting ipsec tunnels Feb 24 19:05:22 php-fpm 339 /rc.linkup: Gateway, none 'available' for inet6, use the first one configured. '' Feb 24 19:05:21 php-fpm 339 /rc.linkup: Gateway, none 'available' for inet, use the first one configured. 'WAN_DHCP' Feb 24 19:05:21 php-fpm 338 /rc.start_packages: [squid] Reloading for configuration sync... Feb 24 19:05:21 check_reload_status rc.newwanip starting igb1 Feb 24 19:05:21 php-fpm 34701 /rc.newwanip: Gateway, none 'available' for inet6, use the first one configured. '' Feb 24 19:05:21 php-fpm 34701 /rc.newwanip: Gateway, none 'available' for inet, use the first one configured. 'WAN_DHCP' Feb 24 19:05:20 kernel igb1: link state changed to UP Feb 24 19:05:20 check_reload_status Linkup starting igb1 Feb 24 19:05:20 php-fpm 338 /rc.start_packages: [squid] Stopping any running proxy monitors Feb 24 19:05:20 php-fpm 338 /rc.start_packages: [squid] Reloading C-ICAP... Feb 24 19:05:20 php-fpm 338 /rc.start_packages: [squid] Reloading ClamAV... Feb 24 19:05:20 php-fpm 338 /rc.start_packages: [squid] Creating 'clamd.sh' rc script. Feb 24 19:05:19 check_reload_status Syncing firewall Feb 24 19:05:19 php-fpm 338 /rc.start_packages: [squid] Adding freshclam cronjob. Feb 24 19:05:19 php-fpm 338 /rc.start_packages: [squid] Adding cronjobs ... Feb 24 19:05:17 kernel arpresolve: can't allocate llinfo for 77.21.62.254 on igb1 Feb 24 19:05:17 kernel igb1: link state changed to DOWN Feb 24 19:05:17 check_reload_status Linkup starting igb1 Feb 24 19:05:17 xinetd 55463 Reconfigured: new=0 old=1 dropped=0 (services) Feb 24 19:05:17 xinetd 55463 readjusting service 6969-udp Feb 24 19:05:17 xinetd 55463 Swapping defaults Feb 24 19:05:17 xinetd 55463 Starting reconfiguration Feb 24 19:05:16 php-fpm 339 /rc.linkup: HOTPLUG: Configuring interface wan Feb 24 19:05:16 php-fpm 339 /rc.linkup: DEVD Ethernet attached event for wan Feb 24 19:05:16 php-fpm 338 /rc.start_packages: [squid] - squid_resync function call pr:1 bp: rpc:no Feb 24 19:05:16 php-fpm 77417 /rc.start_packages: Skipping STARTing packages process because previous/another instance is already running Feb 24 19:05:16 php-fpm 338 /rc.start_packages: Restarting/Starting all packages. Feb 24 19:05:15 php-fpm 63855 /rc.linkup: DEVD Ethernet detached event for wan Feb 24 19:05:15 check_reload_status updating dyndns wan Feb 24 19:05:15 check_reload_status Starting packages Feb 24 19:05:15 php-fpm 77417 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 77.21.62.93 -> 77.21.62.93 - Restarting packages. Feb 24 19:05:15 check_reload_status Starting packages Feb 24 19:05:15 php-fpm 338 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 192.168.10.1 - Restarting packages. Feb 24 19:05:15 check_reload_status Reloading filter Feb 24 19:05:15 php-fpm 338 /rc.newwanip: rc.newwanip called with empty interface. Feb 24 19:05:15 php-fpm 338 /rc.newwanip: rc.newwanip: on (IP address: 192.168.10.1) (interface: []) (real interface: ovpns1). Feb 24 19:05:15 php-fpm 338 /rc.newwanip: rc.newwanip: Info: starting on ovpns1. Feb 24 19:05:13 check_reload_status rc.newwanip starting ovpns1 Feb 24 19:05:13 php-fpm 77417 /rc.newwanip: Creating rrd update script Feb 24 19:05:13 php-fpm 77417 OpenVPN PID written: 35475 Feb 24 19:05:13 kernel ovpns1: link state changed to UP Feb 24 19:05:12 check_reload_status Reloading filter Feb 24 19:05:12 kernel ovpns1: link state changed to DOWN Feb 24 19:05:12 php-fpm 77417 OpenVPN terminate old pid: 84858 Feb 24 19:05:12 xinetd 55463 Reconfigured: new=0 old=1 dropped=0 (services) Feb 24 19:05:12 xinetd 55463 readjusting service 6969-udp Feb 24 19:05:12 xinetd 55463 Swapping defaults Feb 24 19:05:12 xinetd 55463 Starting reconfiguration Feb 24 19:05:11 php-fpm 85913 /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP. Feb 24 19:05:11 php-fpm 85913 /rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. '' Feb 24 19:05:11 php-fpm 85913 /rc.openvpn: Gateway, none 'available' for inet, use the first one configured. 'WAN_DHCP' Feb 24 19:05:11 php-fpm 77417 /rc.newwanip: Resyncing OpenVPN instances for interface WAN. Feb 24 19:05:11 php-fpm 34701 /rc.newwanip: rc.newwanip: on (IP address: 77.21.62.93) (interface: WAN[wan]) (real interface: igb1). Feb 24 19:05:11 php-fpm 34701 /rc.newwanip: rc.newwanip: Info: starting on igb1. Feb 24 19:05:10 php-fpm 77417 /rc.newwanip: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Feb 24 19:05:10 check_reload_status Restarting ipsec tunnels Feb 24 19:05:10 php-fpm 58291 /rc.linkup: Gateway, none 'available' for inet6, use the first one configured. '' Feb 24 19:05:10 php-fpm 58291 /rc.linkup: Gateway, none 'available' for inet, use the first one configured. 'WAN_DHCP' Feb 24 19:05:10 check_reload_status Reloading filter Feb 24 19:05:10 check_reload_status Restarting OpenVPN tunnels/interfaces Feb 24 19:05:10 check_reload_status Restarting ipsec tunnels Feb 24 19:05:10 check_reload_status updating dyndns WAN_DHCP Feb 24 19:05:10 rc.gateway_alarm 55991 >>> Gateway alarm: WAN_DHCP (Addr:77.21.62.254 Alarm:1 RTT:22.693ms RTTsd:17.570ms Loss:25%) Feb 24 19:05:10 check_reload_status rc.newwanip starting igb1 Feb 24 19:05:09 kernel igb1: link state changed to UP Feb 24 19:05:09 check_reload_status Linkup starting igb1 Feb 24 19:05:07 php-fpm 76182 /rc.dyndns.update: Dynamic DNS () There was an error trying to determine the public IP for interface - wan (igb1 ). Feb 24 19:05:05 check_reload_status Linkup starting igb1 Feb 24 19:05:05 kernel igb1: link state changed to DOWN Feb 24 19:05:05 php-fpm 58291 /rc.linkup: HOTPLUG: Configuring interface wan Feb 24 19:05:05 php-fpm 58291 /rc.linkup: DEVD Ethernet attached event for wan Feb 24 19:05:05 check_reload_status Reloading filter Feb 24 19:05:04 php-fpm 85913 /rc.start_packages: [squid] Starting a proxy monitor script Feb 24 19:05:04 php-fpm 77417 /rc.newwanip: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1551031504] unbound[71145:0] error: can't bind socket: Address already in use for 127.0.0.1 port 953 [1551031504] unbound[71145:0] error: cannot open control interface 127.0.0.1 953 [1551031504] unbound[71145:0] fatal error: could not open ports' Feb 24 19:05:04 xinetd 55463 Reconfigured: new=0 old=1 dropped=0 (services) Feb 24 19:05:04 xinetd 55463 readjusting service 6969-udp Feb 24 19:05:04 xinetd 55463 Swapping defaults Feb 24 19:05:04 xinetd 55463 Starting reconfiguration Feb 24 19:05:04 php-fpm 63855 /rc.linkup: DEVD Ethernet detached event for wan Feb 24 19:05:04 check_reload_status updating dyndns wan
I hope some people can help me with this problem.
Have a nice day.Kalle
-
@kalle13 said in Pfsense can`t keep connection alive to provider:
kernel igb1: link state changed to DOWN
Looks like the WAN interface actually goes down, loses link.
Try switching the cable. Swapping the NIC assigned as WAN. Putting in an unmanaged switch between the WAN and the modem.Make sure you have WAN_DHCP set as the default gateway in System > Routing > Gateway not automatic. Remove the v6 gateway if you're not using it.
Steve
-
Thanks for your fast reply Steve.
I will realize all of your proposals but one afte another. So I can see which one (maybe) caused the problem.
I will start with the default gateway option.
Both, IPv4 and 6 were on automatic. I set the IPv4 to WAN_DHCP and IPv6 to nothing.I will let you guys know if it worked or not.
Kalle
-
Also setting 1000 to anything than auto is always going to be a BAD idea!! Gig is meant to be auto! Auto-negotiation is required by 802.3ab at 1GE...
I would be curious if what is meant by gui when set to 1000BaseT or 1000BaseT full-duplex... Since gig can not run non full duplex.. So that shouldn't even be listed. And possible when set vs auto that gig is the only thing advertised in the autoneg?
Interfaces for gig really need to auto to determine who is master and slave for timing, etc..
You shouldn't be messing with that setting unless you want your gig interface to run at something lower than gig. If its suppose to be gig and doesn't neg to that - then you have something wrong that needs to be addressed!!
Have to look into the docs to see if that is called out directly - but they shouldn't really allow you to set 1000 anything other than autoneg.
-
Yes, I've always found that a bit odd. The gui just reports what ifconfig sees. Quite why the driver has those modes is beyond me:
[2.4.4-RELEASE][root@5100.stevew.lan]/root: ifconfig -m igb0 igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000 options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6> capabilities=753fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:90:0b:76:8e:51 hwaddr 00:90:0b:76:8e:51 inet6 fe80::290:bff:fe76:8e51%igb0 prefixlen 64 scopeid 0x1 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet autoselect status: no carrier supported media: media autoselect media 1000baseT media 1000baseT mediaopt full-duplex media 100baseTX mediaopt full-duplex media 100baseTX media 10baseT/UTP mediaopt full-duplex media 10baseT/UTP
Steve
-
Anyway to remove them from the gui - its just going to lead to stuff like this user did because they don't know any better..
But curious if setting that mode just actually removes the 100 and 10 from the neg process?
-
Probably going to need to ask Intel that. Or at least someone much more familiar with the code than me.
Steve
-
I had switched it back to default (autoselect) since a few weeks.
-
I have done 4 from the things of the other posts. Now I've got no connection and so no internet. I am completly lost!
This I've tried:
- switched the cable
- setted tha WAN as dafult gateway
- swapped the NICs
After I swapped the NICs I am not able to get an IP address now. My WAN adress shows 0.0.0.0. After some searching I recognized that I am not the only one with this issue and read some hints. This is what I did:
- reset the router
- switched from bridge mode to normal mode and back
-> in normal mode I got good and fast internet - spoofed the MAC adress from the router
- restarted the pfsense several times after the router was finally up
I the status from the NIC it says :"Status : up" and "DHCP: down"
The DHCP protokoll says this 7 times in a row with diffenerent intervalls:
dhclient 76735 DHCPDISCOVER on igb0 to 255.255.255.255.255 port 67 interval 9The system-generic protokoll has 3 entries:
-
php-fpm - /status_interfaces.php: the command '/sbin/dhclient -c /var/etc/dhclient_wan.conf igb0 > /tmp/igb0_output 2 > /tmp/igb0_error_output' gave exitcode '15' and the result was "
-
php-fpm - /status_interfaces.php: the command '/sbin/dhclient -c /var/etc/dhclient_wan.conf igb0 > /tmp/igb0_output 2 > /tmp/igb0_error_output' gave exitcode '1' and the result was "
-
php-fpm - /status_interfaces.php: the comand '/usr/local/sbin/dhclient {$ipv} -d -r -lf '/var/db/dhclient.igb0' -cf '/var/rtc/dhclient_wan.conf' -sf '/usr/local/sbin/pfsense-dhclient-script" gave exit code '1' back, the result was "Internet Consortium DH CP Client .... Listening on BPF/igb0/54:67:44:45:gg:34 Sending on BPF/igb0/... Canˋt attache interface {} to bpf device /dev/bpf0:Device not configured ....
-
Yes, if you changed the WAN MAC you might need to restart your modem device or even cal your ISP and ask them to reset it.
There should be no reason a different NIC would not get a DHCP address unless the server is refusing it.Steve
-
Did my MAC change when I spoof the MAC of my router in my pfsense?
My router reseted several times due to the switching between the normal and bridge mode.My new attempt is: f**k this router, I buy a used 6360 FritzBox cable and see if this works.
But if this is not going to be working I am calling my ISP to reset their modem.
-
Nothing of the following worked:
- bought another cable router (Fritzbox) -> no success
- called my ISP -> they resetted their hardware -> no success
I am now finished reinstalling pfsense and nothing changed. I canˋt get a IPv4 adress via DHCP.
I am desperate now. Does anyone has some tipps for me? -
Run a packet capture on the WAN see what's happening. Is it actually sending dhcp requests? Is it seeing replies?
It still gets an IP if you put the modem back in router mode I assume?
Steve
-
@stephenw10 I did that yesterday (with wireshark). The pfsense sends the request but after that there is no offer from the ISP.
That´s right. In normal mode it get`s an IP.
Kalle
-
What gets this working again?
Rebooting the cable router (Fritzbox), or rebooting pfSense?
-
The mystery is is solved!
Heureka!
After days of working and searching.It was a problem on the ISP side. I think, but don`t know it for sure, that they had a problem with the DHCP server. Maybe.
Today they worked on it. After some time it worked.Thanks for you all for your help!
This case is closed. -
Hmm, well why does the ISP not respond I wonder...
Can you try a pcap from a switch mirror port in the connection? That would prove it's actually being send.
Is it spoofing the MAC correctly?
Steve
-
Sadly I do not know. I did not talked to the IT people. I only had a ticket for my failure. So the only that I have is, that they where working on it and now it's working. That`s all.
I don't understand what you say. What should I do?
I watch my connection with wireshark.
I did the spoofing one time but after it did not succeed I witched it back to default.Kalle
-
When you run tcpdump on the interface in pfSense you see eveything the driver is sending but that might not necessarily make it onto the wire.
By using a switch in between, mirroring the port and capturing on there you see what traffic is actually going back and forth.Steve