@stephenw10 that is insane.. I had a device that had a multicast mac set on it, company screw up.. It could work - but it could also cause some problems. It was a networking bridge for electric consumption meter, current cost by envi if I recall.. Had to be 10 years ago, I remember it working until I moved it something other than a dumb switch - then I ran into issues.. I remember having to do something with igmp snooping.

An update for anyone who may be experiencing this issue.

This issue is caused by ATT's RG firmware. The latency spikes and jitter are resolved on the BGW320-505 as of firmware 6.30.5.

This issue was somewhat widely discussed at /r/ATTFiber. Shame on ATT for taking 8+ months to release a firmware which fixed it. And I was only able to get the firmware update by working with a redditor who had a high-level engineering contact at ATT, who was able to MANUALLY push the firmware update to my device. Who knows when it would have rolled out to me...

Thank you to @stephenw10 for the help along the way.

@stephenw10 cheers. We shall wait!

I will say that I've only ever seen issues like that when using UFS without RAM disks enabled. Using ZFS or enabling RAM disks should prevent it for the vast majority of cases.

Graylog has taken care of this for me. Creating reports for top dst IP and ports

@mucip said in Internet lost in PfSense but VPN or everythingelse work well?:

the monitoring and what is used for?

And nice stats :

79f298d6-5237-4445-8459-f960de6df354-image.png

Monitoring the device in front of your pfSense, the ISP router, only tells you something about the cable between them 😊 and nothing much about your entire WAN connection.
Monitoring for example 8.8.8.8 tells you something about the 'quality' of the connection up until 8.8.8.8.
This will includes your entire 'ISP' WAN connection.

@stephenw10 said in Route pfsense itself over VPN.:

Hmm, well it sure seems like it's set to serial as primary console from what you're seeing. But as I say you should be able to see which is set at the vga console.

Success !!!

I made the mistake when creating a static route by typing ip address of openvpn server and then selecting the subnet. Selecting the subnet was a mistake that caused pfsense to be stuck at boot. I created new static route by typing the openvpn server address, chose the appropriate WAN and saved the setting. After reboot, pfsense booted just fine. I created additional static routes for the remaining vpn clients and everything just worked. Rebooted once again, no issues. Then i selected openvpn client as a default gateway and that was it.

All tailscale clients are now going through vpn, and all vpn clients connect without any issues after reboot.

Thank you very much Stephen. All this would not be possible without your help. Im marking this thread as resolved.

Cheers.

Nice! That does imply some ARP issue. You shouldn't really have to do that. But if you do keep that in place you should add it as a system Tunable:

https://docs.netgate.com/pfsense/en/latest/config/advanced-tunables.html

@mvikman oh sorry - I must of hit reply on wrong post, corrected ;) thanks

Good idea about redmine, I will look to see if anything in there already - and not make the suggestion

Supplement, for future reference if someone encounters the same problem.

I did some tests later and ruled out pf, but the problem was the same.
The final result was that the esxi virtual machine would check the mac match by default. If it does not match the mac in the vmx configuration file, the communication will fail. Customizing the mac on the gui can only set a fixed prefix,

Reference:

https://community.broadcom.com/vmware-cloud-foundation/discussion/custom-mac-address-in-esx-4#bm65eebd63-587b-41e1-8108-b951b7ef03d0

And because the new version of esxi parameter definition checkMACAddress is invalid

ethernet1.checkMACAddress = "false"

I don't want to enable promiscuous mode in the vds port group, so the final solution is to add a network card to modify the vmx configuration of the virtual machine to define two items:

ethernet1.addressType = "static" ethernet1.address = "10:2d:3c:40:55:63"

Reference:

https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-networking/GUID-F9243FED-F081-498F-B4A9-EF950292AF77.html

Of course, modify mac from the system layer,
1, such as pfsense, modify /etc/inc/interfaces.inc Add mwexec ("/sbin/ifconfig vmx0 ether 10:2d:3c:40:55:63"); Updates will be overwritten
2, Windows system settings network card mac
and then enable the three items (Promiscuous Mode, MAC Address Changes, Forged Transmits) on vds. It is feasible and communication is possible. I did not adopt this solution

@VioletDragon said in pfSense Auto Backup:

Do you know if this will backup configs for like pfBlockerng, Snort etc?

You've missed somehow what pfSense is.
Ok, its a firewall router. Like many out there, With tonnes of extras.
The main difference with other firewalls is that is is for 99,9 % Web interface driven, and it has just one (1) config file. This file contains everything.

Just read it : export it and open it in a text editor. You'll get the picture quickly.

No worries, glad you were able to get back up and running. 👍

@CatSpecial202 there is another thread going over this topic actually..

587 is explicit use of tls, ie it would make a non secure connection and then use starttls to to convert to an encrypted session. While port 465 is implicit connection only, so no that checkbox would prob not work on port 587

here is the thread where that is being discussed, kind of as a side topic

https://forum.netgate.com/topic/190885/empty-message-id-in-smtp-test-email

@matt0023 you can use either 587 or if like @Gertjan mentioned you can use 465.. I just changed mine to 465 and working.. I need to do some more research, seems there is a rfc that calls for implicit port use of 465

https://datatracker.ietf.org/doc/html/rfc8314#page-7

It is desirable to migrate core protocols used by MUA software to Implicit TLS over time, for consistency as well as for the additional reasons discussed in Appendix A. However, to maximize the use of encryption for submission, it is desirable to support both mechanisms for Message Submission over TLS for a transition period of several years. As a result, clients and servers SHOULD implement both STARTTLS on port 587 and Implicit TLS on port 465 for this transition period. Note that there is no significant difference between the security properties of STARTTLS on port 587 and Implicit TLS on port 465 if the implementations are correct and if both the client and the server are configured to require successful negotiation of TLS prior to Message Submission.

Either way the submission of the email and the auth would and should be encrypted - so from that point of view either or works.. It comes down to which one where you sending supports, and what your client supports I guess.

And you should be using tls 1.3 for sure - which you can see from posted header info that is currently used by pfsense notification via email.. I have currently moved mine to 465, I doubt gmail is going to stop use of that port no matter what any rfcs - atleast for many man years..

@stephenw10

Thank you so much

@Gertjan Thank you, This helps!

Yup, check for watchdog errors from the realtek driver. If you see any you definitely need the alternative driver.

For reference when coming from 2.7.0 you almost always need to run certctl rehash in order to see the update. That's fixed in 2.7.1 but it appears you may not actually have been on that.

Apologies for my late reply!

The IP does change from time to time, but usually over a long period (a few months). I've received such notices from the ISP for many months too, though I'm not sure if it was the same or not when they first started emailing me.

That said, I never thought someone else having the IP before could have caused it. It's the only plausible explanation that I've come across and could well explain it.

I did think to contact my ISP, but they're totally useless at the best of times. Will be switching to another provider within a couple of weeks, so will see if it continues after that.

Also, just a thank you to everyone who offered help 🙂

No, it is because my streamer is not from an official Spotify Connect authorized vendor. Instead, they use the open-source Librespot client library, which can be flakey depending on its implementation.