Yes I added the lines in the config. The PPP connection was established without problem and there are no errors in the log file.
I haven't had outage since than, therefore I couldn't test the reconnect part.

In the end I wrote a very tiny shell script to check the ZFS status and put it into /usr/local/libexec/nagios/check_zfs_status.sh

#!/bin/sh cmdzpool="/sbin/zpool" healthcheck=`$cmdzpool status -x | grep -c "all pools are healthy"` if [ $healthcheck -eq 1 ] then echo "ZFS Volumes OK" return 0 else echo "ZFS Volumes error." return 2 fi

It's not the ideal solution since the check has to be installed manually but it's better than nothing.

Ah it dawns on me there is one thing I should mention so someone else doesn't get caught.

ARP cache played a huge part in this and an accidentally left over rule in one of the firewalls as well.

The firewalls were basically competing to be the gateway so things would get weird like a vm would boot up with a gw then a while later change to another.

Once ARP cleared up, everything was fine.

Yes that should be fine but neither reached the speeds you were seeing to speedtest so it doesn't really give us any more information.
If you can connect the pfSense WAN to something running an iperf server directly and test to that it removes any sort of restriction reaching it so any limitation left much be in one of the pfSense interfaces.
As I said I do that sort of test all the time using another pfSense instance on the WAN. That way it hands out DHCP so the test device just works.

Steve

@johnpoz I block doubleclick also they have some bad bugs in that system. I am glad to see I am not the only one.

Thanks guys for your answers!

@cool_corona said in What information is Netgate collecting?:

@stephenw10 Then why is it uploaded to Netgate in the first place?

It's doing that because the user told the firewall they wanted it to do that.

It's for secure remote backups, and it's off by default and completely opt-in. Maybe they forgot they enabled it, or another user enabled it, but it was done by choice not by Netgate.

Hmm, you would not expect some minor packet loss to cause TCP connections to fail. You just see retransmissons.

Unless all of those failures were happening at the same time so it times out. That would take a while though.

This starts to look more like a duplicate IP or a packet loop. You can see that if you have a loop that's prevented by stp and it periodically resets.

Removing one link from the lagg entirely might prove that.

Steve

You may need to re-trigger it and check specifically then.
If there's nothing at all in the DHCP logs it probably didn't try to run the dhclient at all. That usually means the WAN NIC was unlinked at the time (the ONT was booting) but became linked before pfSense finished booting failing trigger the usual linkup script.

Steve

The system aliases for each interface (LANnet, WANnet etc) are only the actual interface subnet.

So often your ISP will provide your WAN IP and subnet something like 1.2.3.4/29 or maybe only a single IP if it's PPP connection. WAN net is only the IPs in that /29.

It's a common mistake with new users because many other firewalls with zone based filtering use the WAN 'zone' to mean the entire internet.

Traffic routing from LAN via WAN or OpenVPN would depend on the system routing tables since the LAN rules do nor have any policy based routing on them (a gateway set).
The system routing tables are usually updated by the OpenVPN client when it connects based on whatever the server passes it. Most commercial providers will pass a new default route. Often that's undesirable so you can set the OpenVPN client to ignore routes passed to it and use policy based routing instead. That's what I do.

Steve

@stephenw10 said in Can't update battery date in NUT:

Probably your UPS simply doesn't support it:

Seems you might be correct:

[2.6.0-RELEASE][admin@pfSense.localdomain]/root: upsrw apcups@localhost [battery.runtime.low] Remaining battery runtime when UPS switches to LB (seconds) Type: STRING Maximum length: 10 Value: 120 [input.sensitivity] Input power sensitivity Type: STRING Maximum length: 10 Value: medium [input.transfer.high] High voltage transfer point (V) Type: STRING Maximum length: 10 Value: 266 [input.transfer.low] Low voltage transfer point (V) Type: STRING Maximum length: 10 Value: 180 [ups.delay.shutdown] Interval to wait after shutdown with delay command (seconds) Type: STRING Maximum length: 10 Value: 20 [ups.delay.start] Interval to wait before (re)starting the load (seconds) Type: STRING Maximum length: 10 Value: 30 [2.6.0-RELEASE][admin@pfSense.localdomain]/root:

Yes, should be good now. Let us know if you see any further issues.

Steve

@marinsnb said in CGNAT and pfSense:

but my WAN gets a 100.xxx.xx.x IP according to my pfSense.

Yeah 100.64/10 or 100.64-127.x.x is cgnat range. If that is what your isp is using there not much you can do about that other than contacting to see if they can give you a actual public IP, possible more $$..

Do you also get a IPv6 - that should be a global address and public, I would hope they would do a prefix delegation of /56 or /48 even which would allow for not natting when doing IPv6.

Problem with cgnat, is even getting say a hurricane electric IPv6 tunnel isn't going to work.

Yes probably should be the CGNAT space, 100.64.0.0/10.

@cool_corona

Thank you -- can you point me to any online discussion/thread where this is discussed so I can follow along?

@bassit said in Any ideas? dns gone rouge.:

I hadn't changed anything in resolver or forwarder.

Well pfsense was just resolving - that is how it is out of the box. I resolve for like 10 some years now, never an issue.

What stopped working is your blocking it via your IPS is most likely..

Or you had an issue resolving "something" and noticed the log entries and went down a rabbit hole that has nothing to do with anything ;) If your just in IDS mode and monitoring and not blocking.

DNS (resolving) can fail when you can not talk to a ns in the path to getting to the authoritative NS for the domain your wanting to lookup.. Or can not talk to the authoritative ns for some reason, or dnssec fails, etc. and you have that enabled (its enabled out of the box).. But if yoru going to forward you should uncheck that.

When something fails to resolve, and your resolving - you need to figure out why.. dig +trace is good start to see what your having an issue talking too.. If your forwarding, and something fails - your at the mercy of why you didn't get an answer to where you forwarded too.

@johnpoz Done. Thanks for your help!

@st6 said in Connection Dropped After Gateway Down:

I asked here and people told me that this feature will only clear states of the link that becomes down don't affecting other links connection.

That's not true. It will kill states on all gateways when any gateway goes down. It's not enabled by default because it's very disruptive but can be helpful in some situations.
When that feature was added it was not possible to kill states by gateway so it was all or nothing.

Steve

@johnpoz wow thanks very much, i am new to pfsense so im not realy sure at what i am seeing, it seemed odd that something pre programmed would contain syntax that is wrong.

@stephenw10 - Your assumption is probably correct, the original install would have been done 5 years ago with whatever version was current at the time. And, I have not connected a monitor to it since. I also re-installed using the UEFI option and that's what it is in now.

Still does not explain why it never booted after being shutdown. Guess we will never know, but thanks again for all your input and time.

Greg