@bmeeks Thank you so much. I did send you the full log separately. Please let me know if I can get you anything else. And you needn’t do any more. I’m more than happy with what I’ve learned. Cheers!

Ah ja ok stimmt, die DNS Regel auf Localhost mache ja Sinn, aber nur wenn das Ziel !This Firewall ist.

@arjay Not NAT, but outbound NAT.
Did you add that?

@robinh but that is not your webgui port, so that you ha proxy... Restart your web gui, most likely it didn't restart..

I get even more and more the feeling that either:

a) my ISP is doing something weird or
b) my VDSL Modem is doing weird stuff.

I just noticed that even Playing CS:GO is not working anymore. After a couple of seconds I get the Message not possible to official Servers with following log:

Refreshing ping measurements SDR RelayNetworkStatus: avail=OK config=OK anyrelay=OK (Refreshing ping measurements) SteamNetworkingSockets lock held for 5.6ms. (Performance warning.) ServiceThread,SteamDatagramClientThinker::Think,EnsureDataCenterRoutesValid,ThinkPingProbes,CreateServerDataForCluster(x10),SendUDPacket(x10) This is usually a symptom of a general performance problem such as thread starvation. Host_WriteConfiguration: Wrote cfg/config.cfg Ping measurement completed Ping location: mlx1=14+1,mst1=19+1/20+1,fra=/22+1,ams=/25+1,lhr=/32+1,vie=/33+1,par=/37+1,mad=/40+1,waw=/42+1,mny1=85+8/86+8,iad=/91+8,mmi1=124+12/119+8 SDR RelayNetworkStatus: avail=OK config=OK anyrelay=OK (Refreshing ping measurements) Ping measurement complete after 5.0s. Sending sample to GC ams: 25ms via mlx1 (front=14ms, back=11ms) can: 185ms via tsnu (front=149ms, back=36ms) canm: 193ms via tsnu (front=149ms, back=44ms) cant: 184ms via tsnu (front=149ms, back=35ms) canu: 183ms via tsnu (front=149ms, back=34ms) dfw: 132ms via mny1 (front=85ms, back=47ms) lhr: 32ms via mlx1 (front=14ms, back=18ms) mam1: 25ms via mlx1 (front=14ms, back=11ms) mas1: 93ms via mny1 (front=85ms, back=8ms) mat1: 104ms via mny1 (front=85ms, back=19ms) mch1: 105ms via mny1 (front=85ms, back=20ms) mdc1: 92ms via mny1 (front=85ms, back=7ms) mdf1: 122ms via mny1 (front=85ms, back=37ms) mfr1: 22ms via mst1 (front=19ms, back=3ms) mla1: 149ms via mny1 (front=85ms, back=64ms) mln1: 32ms via mlx1 (front=14ms, back=18ms) mlx1: 14ms via direct route mmi1: 119ms via mny1 (front=85ms, back=34ms) mny1: 86ms via direct route mpx1: 157ms via mny1 (front=85ms, back=72ms) msa1: 139ms via mny1 (front=85ms, back=54ms) msj1: 150ms via mny1 (front=85ms, back=65ms) msl1: 110ms via mny1 (front=85ms, back=25ms) mst1: 20ms via direct route par: 37ms via mst1 (front=19ms, back=18ms) pwg: 183ms via tsnu (front=149ms, back=34ms) pwj: 150ms via tsnu (front=149ms, back=1ms) pwu: 158ms via tsnu (front=149ms, back=9ms) pww: 170ms via tsnu (front=149ms, back=21ms) pwz: 177ms via tsnu (front=149ms, back=28ms) sea: 150ms via mny1 (front=85ms, back=65ms) sha: 175ms via tsnu (front=149ms, back=26ms) sham: 174ms via tsnu (front=149ms, back=25ms) shat: 174ms via tsnu (front=149ms, back=25ms) shau: 169ms via tsnu (front=149ms, back=20ms) shb: 174ms via tsnu (front=149ms, back=25ms) sto2: 49ms via mlx1 (front=14ms, back=35ms) tsn: 150ms via tsnu (front=149ms, back=1ms) tsnm: 149ms via tsnu (front=149ms, back=0ms) tsnt: 149ms via tsnu (front=149ms, back=0ms) tsnu: 149ms via direct route tyo1: 267ms via msj1 (front=160ms, back=107ms) Host_WriteConfiguration: Wrote cfg/config.cfg Started tracking Steam Net Connection to =[A:1:2737553413:22553]:0, handle ceb4929 [#216746281 SDR server steamid:90168859682390021 vport 0] Requesting session from mlx1#71 (188.42.190.100:27047). Ping = 414 = 14+400 (front+back). [#216746281 SDR server steamid:90168859682390021 vport 0] Requesting session from mst1#57 (151.106.18.227:27041). Ping = 419 = 19+400 (front+back). [#216746281 SDR server steamid:90168859682390021 vport 0] Selecting mlx1#71 (188.42.190.100:27047) as primary. (Ping = 414 = 14+400+0 (front+interior+remote).) [#216746281 SDR server steamid:90168859682390021 vport 0] Selecting mst1#57 (151.106.18.227:27041) as backup #1 (Ping = 419 = 19+400+0 (front+interior+remote).) Already have a ticket for server 'steamid:90168859682390021' with older expiry 1675206188. Discarding and replacing with new ticket expiring at 1675206254 Received Steam datagram ticket for server steamid:90168859682390021 vport 0. Host_WriteConfiguration: Wrote cfg/config.cfg [#216746281 SDR server steamid:90168859682390021 vport 0] problem detected locally (4001): Timeout; remote problem. Rx age server (never) relay 0.4s Steam Net connection #216746281 SDR server steamid:90168859682390021 vport 0 problem detected locally, reason 4001: Timeout; remote problem. Rx age server (never) relay 0.4s **** Unable to localize '#GenericConfirmText_Label' on panel descendant of 'PopupManager' Closing Steam Net Connection to (unknown), handle ceb4929 (2001 Matchmaking failed. We never heard from gameserver) Summary of connection to #216746281 SDR server steamid:90168859682390021 vport 0: End-to-end connection: closed due to problem detected locally, reason code 4001. (Timeout; remote problem. Rx age server (never) relay 0.4s) Remote host is in data center 'fra' Current rates: Sent: 0.0 pkts/sec 0.0 K/sec Recv: 0.0 pkts/sec 0.0 K/sec Ping:414ms Max latency variance: ???ms Est avail bandwidth: 1024.0KB/s Bytes buffered: 0 Lifetime stats: Totals Sent: 21 pkts 6,597 bytes Recv: 0 pkts 0 bytes No ping distribution available. (1 samples) No connection quality distribution available. (0 measurement intervals) Latency variance histogram not available No rate stats received from remote host No lifetime stats received from remote host Primary router: mlx1#71 (188.42.190.100:27047) Ping to relay = -1 Current rates: Sent: 2.0 pkts/sec 0.6 K/sec Recv: 2.2 pkts/sec 0.0 K/sec Quality: 100% (Dropped:0.00% WeirdSeq:0.00%) Bytes buffered: 0 Lifetime stats: Totals Sent: 21 pkts 6,597 bytes Recv: 21 pkts 204 bytes Recv w seq: 21 pkts Dropped : 0 pkts 0.00% OutOfOrder: 0 pkts 0.00% Duplicate : 0 pkts 0.00% SeqLurch : 0 pkts 0.00% No ping distribution available. (0 samples) No connection quality distribution available. (1 measurement intervals) Latency variance histogram not available No rate stats received from remote host No lifetime stats received from remote host Backup router: mst1#57 (151.106.18.227:27041) Ping = -1+-1=-2 (front+back=total) Removing Steam Net Connection for =[A:1:2737553413:22553]:0, handle ceb4929 [#216746281 SDR server steamid:90168859682390021 vport 0] Discarding inactive session mst1#57 (151.106.18.227:27041). ConnectionShutdown [#216746281 SDR server steamid:90168859682390021 vport 0] Discarding inactive session mlx1#71 (188.42.190.100:27047). ConnectionShutdown

While die PCAP ( CSGO.zip ) shows incoming and outgoing Traffic ... To be 100% sure I just ordered a new VDSL Modem...

@stephenw10 said in Perplexing Problem with PFSense:

@itworxnz said in Perplexing Problem with PFSense:

It's always the same two blocks out of seven that seem to cause it, but everyone is affected.

If the router/gateway went down everyone would be affected but the different hosts in the same subnet would still be able to connect to each other. Can we assume that isn't case?

Still need that questions answering to determine what sort of problem you are dealing with. And I would still do this:

When this happens if you run a pcap somewhere do you see anything incoming?

This doesn't seem like a bad cable to me or a bad switch port. Those would only effect devices connected to them. For something to take down the entire subnet across multiple switches such that no traffic can move across the network at all it pretty much has to be a flood of some sort.

But if things can still ping other local hosts just not the local gateway I'd be looking for a rogue dhcp server or something doing ARP poisoning perhaps.

You should really be using VLANs to separate these user groups out. That would prevent something like a rogue dhcp server affecting everyone.

Steve

@johnpoz Agreed, I too hope IoT catch up soon on security related stuff. Many nice gadgets only have wifi, and as is, I don't feel entirely comfortable using that for IoT. Of course it can and imho should, be zoned in contained vlans, but just the fact your wifi is offering your network to anyone who (can) listen, is not comforting, but very convenient.
I've learned a lot on this exercize, enough to wanting to read more - WPA2 to WPA3 was indeed a big leap, and perhaps time for me to re-evaluate wifi for my purposes... :) (for now still excluding IoT)

@stephenw10

I have 3 x Netgate 3100 appliances. 2 live and one spare. One of the live ones is located in a distant datacenter so upgrading it remotely is too risky.

Typically I upgrade all 3 firewalls only about once per year when I have other reasons to travel to the dc. I import config to the spare one and just physically swap them around followed by some testing. If anything goes wrong then I just swap them back.

Unless the issue comes back I'll wait for the next major release with the first follow up update.

Longer route, more hops. Generally more chances to lose packets.

1.1.1.1 is an anycast address so you see replies from whatever is logically closest to you.

Steve

@ftani Now it starts to make sense, except the two block private networks rules under IOT (and the other VLANs). Those rules only belong on the WAN side, which you already have...

I GOT IT !

enabled MSS clamping to 1440.

So settings for Wemag, if anyone is reading this post :

WAN:
-> DHCPv6
-> MTU 1492
-> prefix /64

LAN:
leave untouched (...well, apart from the ipv6 setting - tracking WAN and so on- )

GIF:
MTU 1472
MSS clamping: 1440

now everything seems to work as it should. tbc.
Thank u @JKnott and @stephenw10

@stephenw10 said in CGNAT UPnP Issue Advice:

Yeah set an override or enable the STUN external IP detection.

UPnP can still work in that situation if the upstream router is forwarding traffic. So if you set the pfSense as the DMZ IP in your ISP router for example.

Steve

For this scenario, where UPnP isn't actually used for anything towards external servers/devices, STUN might work as a way to remove the errors.

It might also work for e.g. a gaming scenario, at least if the mobile router has a public IP, (I'll make sure to test that). But in this case the mobile router is behind CG-NAT, so it might not work for gaming.

What I don't understand though, is why does miniupnp give this error and refuses to do it's job if the WAN IP is from the private IP range?
If the upstream router places pfsense in DMZ, it should still work!

I have tested this and it does actually work fine if you can "fool it"...

My failover goes over LTE and the mobile router has a public IP but doesn't do bridging. It does however have DMZ and most importantly, it allows me to set any IP on the LAN interface. If I set it to a public IP, UPnP on pfsense works perfectly fine, giving me Open NAT on all the games I throw at it, double NAT and all. Other routers, like Ubiquiti edgerouter, also work, but they do it even if WAN has a private IP...

The problem that you run into when doing it this way, is that it breaks the Dynamic DNS setup, since it will now take the fake WAN IP and not use the "Check IP Service".

I see three simple things that we need here.

Provide an override selection to prevent miniupnp to check for private IP on the WAN interface. Introduce Gateway Group into the External Interface selection for UPnP, so it can follow the default gateway in a failover scenario, or allow multi select not only for Internal interfaces. Not really a necessity if 1 & 2 are in place but still a good idea to have the option to force "Check IP service" regardless of the WAN IP.

You can see it has a default route at the top of the table and I would guess that it would would not have shown that before. It might show in logs still but it probably won't tell you anything.
If it happens again check the routing table before making any gateway changes. I doubt it will though.

Steve

@motivio lets get that pcap started on pfsense.
Not sure how often it's querying for snapchat but let it run until the alert in pfblocker comes up.
Make sure count is set to 0
Stop the capture
Download the capture
Open the capture
search for the string in the capture. Edit > Find Packet > Set to string

0a9cbe25-36eb-4bb1-9944-8306efaa8b03-image.png

@alcamar said in HAProxy Weiterleitung zum nextcloud-Server:

Kann das hier dokumentieren, falls es ähnliche unbedarfte wie mich in Zukunft gibt.

Warum nicht? Nachdem der Threadtitel schon darauf hinweist, könnten Leute das finden.

Allerdings ist deine Konstellation wohl eher eine Seltenheit. Wie gesagt, üblicherweise läuft ein Webserver heutzutage nicht in einem virtuellen Verzeichnis.
Wenn HAproxy würde ich alle Anfragen einfach weiterleiten lassen und den Rest den Backendserver machen lassen. Dafür gibt es jede Menge Anleitungen.

Ich kämpfe aber noch mit Zertikaten beim CALdav. Eigentlich müsste nur die pfsense Zertifikate jonglieren, oder?

So wäre es wünschenswert. Funktioniert leider nicht immer. Ich weiß aber nicht, wie das bei Nextcloud ist. Meine betreibe ich nicht hinter einem Proxy.

Aber bezüglich DAV und HAproxy habe ich schon Threads gesehen. Aber ich denke, hier würdest du mit den beiden Suchbegriffen im Netz rascher brauchbare Ergebnisse finden als hier.
Die könnte man dann auf die Konfiguration in der pfSense GUI "übersetzen".

Deinen Punkt hinsichtlich Sicherheit des Ports 443 habe ich mir für die nächsten Überlegungen vorgemerkt.

Wenn du es geschafft hast, dass HAproxy die Anfragen in das virtuelle Verzeichnis von Nextcloud leitet und keine anderen zulässt, sollte es soweit eh sicher sein. Abgesehen natürlich, dass dir klar sein muss, dass die Nextcloud im Internet steht und damit ordentliche Zugangspasswörter braucht und aktuell gehalten werden muss.

@orcape
es wäre schon mal interessant ein tcpdump laufen zu lassen, wie @Bob-Dig schon schreibt hört sich das nach einem Client Problem an.

Allerdings weiß ich auch nicht was der WG Client macht wenn keine Paket Antwort kommt,
evtl. aller paar Sekunden eine neue FQDN IP Auflösung?
Darüber habe ich mir noch gar keine Gedanken gemacht, ist aber ein guter Punkt.

Wir setzen WG nur auf Smartphones ein, es ist halt super für Roaming (Wifi+LTE) und vor allem für den Akku verbraucht.

Alles was Strom hat bekommt OpenVPN ;)

@jimp There’s the button!! I knew there had to be a button/box for this, just couldn’t find it… Thanks so much!

@rh128 that is good news! yeah not good idea to just pull the power plug on pfsense. You running ZFS - that is suppose to be better than UFS..

@jc1976 said in MTU bug:

when i adjusted the mtu, i arrived at 1472 by plugging my laptop directly into the modem and running the commands until it stopped fragmenting, and then i used that number input the mtu size in pfsense, so i was able to adjust it.

On the WAN interface, you have to use the MTU your ISP uses. Otherwise, some frames may be discarded. Do you know where the fragmenting is happening? It could be anywhere between you and the destination. You can determine that by looking at the source address of the ICMP messages. Fragmentation is the mechanism to get around the different MTUs and is entirely normal, though these days Path MTU Detection is often used and is mandatory with IPv6. Also, what was fragmenting? On Linux PMTUD is generally used for everything and with TCP on Windows.

@stephenw10 the one in the picture above

@joshgreyz: Are you aware of the ability to upgrade your Community Edition to pfSense+?

As a quick test I confirmed that an older export does fail with OpenVPN 2.6.0 but I exported a .p12 from a snapshot using the cert manager set to 'high', without a password, and dropped that into the OpenVPN config folder with the right name and that worked.

So that's another option, but it's a bit cumbersome.

@froek said in XG 7100 vlan dhcp configuration problem:

Thank you Ryan - you got me pointed in the right direction!

You're welcome :)

User Intel Quick Assist on 71er Site and SafeXcel on the ARM Site.
User a good DH Group like 19, 20, 21.
Use wan optimized Stuff to push Data over VPN, not SMB, it designed is lan ony.

I run my NAS Backups over the Tunnel, with the Upload limiting around about 50MBit/s.
This Speed is no problem for the 21er, System Load 8-9%, Interrupt 18%.

And yes, if you use AES GCM with SafeXcel on ARM, you got stuck after som Time with the entire IPsec Stack.

@coffeeman007 Oh my bad - plex on the brain I guess.. Sorry!! doh!!

@stephenw10

This works in a MS Active Directory, via LDAP. My goal is connect to our Google Workspace LDAP.

The pfSense Authentication and Captive Portal works, but Squid, not.

I have a change "Squid Authentication Method" to Local and doesn't autenticate.

@mystique_ OSPF is pretty simple to set up.

Enable it and add the interfaces to area 0 and you're done.

One generally sets interfaces that are to be in the OSPF database that are not intended to communicate with other OSPF routers to passive.

That's generally all that HAS to be done to get the IGP working and exchanging routes.

@heper OK. I realized my ISP also has iperf3 server listening so I tried that instead of speedtest. I attached a virtio vtnet interface to pfsense and made a better comparison with single vs. parallel flows, IPv4 vs. IPv6, and coming from physical port (ix) vs. virtio (vtnet). WAN is always physical port.

You had right, I think it is related to tx/rx queue issue you also linked above.

My test results show a single flow (ix or vtnet) can support around 5Gb/s on my system (packet filtering enabled). If I enable parallel, ix reaches to 9Gb/s, it does not matter IPv4 (NAT) or v6 (no NAT), and it consumes around 70% CPU (4 cores). However, when using vtnet with parallel flow, neither throughput nor CPU use changes and it is still around 5-6Gb/s (similar to single flow). It is actually very good for a single flow (as good as physical) and CPU consumption is not different (only a few percents higher maybe).

@stephenw10

Probably the appliance I’d get if shopping for one. IF I had it to do over instead of a nice albeit older Dell i7 for free and then add the cost of several Intel i225 NIC’s, I’d do this appliance. The cost of the NIC’s about the same as this appliance.

The new gen processor while not today’s i7 my guess is it outperforms the early gen i7 in the old Dell.

https://www.amazon.com/dp/B0B6J2ZKTM/?coliid=I2ZTQKSXOF5P6&colid=B016TSAJFGNR&ref_=lv_ov_lig_dp_it&th=1

@bmeeks

Yeah, got catch! I checked PfBlocker and the cron job starts at the top of the hour (see screenshot below). I probably should change that to on the half hour 00:30 so they don't collide. I'll try that and report back :)

screenshot4.jpg

@johnpoz said in Static IP - MAC mapping inside DHCP dynamic pool - how to?:

You trying to use .77, .88, .99 for your whatever equipment is not something I can say I have ever seen in some 30 years in the biz.. Common practice I have seen used countless companies is use either the first part or the last part of the range for statics or reservations.

With all my appreciation to Your knowledge and value here in community, but:

If You do not see some solution before (even with wide contacts and hundreds of installations), - not mean the solution is wrong or not have a reason.

Many peoples (even hi-educated Ingeneers with degrees) not thinking and analyzing, just doing like “googling for solution- copy solution - if working, go to next task or pub”. So, very possible that a You see a thousands of persons who “just copying nor thinking”.
And than we all see thousands of data leaking, DB hacking, etc... Just because even in a Enterprise sysadmins not thinking...

More than this, I pretty sure a You not see much sysadmins who make proper equipment and wires labeling with QRcodes (where one tap on iPad/iPhone open the sheet with all data about this equipment/ cable or open Augmented Reality plane, like this
Is this hard to implement - definitely NO. People just not thinking....

Like only few people on this forum pay attention to rapidly growing of QUIC protocol implementing. And one day BOOM! And all here start to realizing that old-fasion filtration come close to the end, because more and more ISP implementing QUIC on their core, more and more mobile apps start to using QUIC, more web browsers come with QUIC enabled by default...

Back to topic: I STRONGLY SURE that remembering the fact that .99,.88,.77 “double numbers” are own company stable equipment ARE EASY that anything else.
(Because as SysAdmin I need to remembering also A LOT of other things...)

@johnpoz haha yes I did use the wrong name I have a Netgear switch and a netgate router. Thanks for your suggestion. I will have to research some more ! I think my issue is my lack of research. I might have gotten into something that was beyond my understanding but I do think the pf software is quite a sophisticated piece to everything. Having a parameter firewall, VPN, Snort, Proxies etc it was definitely worth the purchase. I will have to learn more about networking haha. Cheers.

Any idea how you ended up with Python 3.9 on there? That isn't in the 22.05 repos.

I use PVE. Everything is very stable and no problems.

@viragomann said in pfsense HAproxy HTTPS frontend to HTTP backend (Server hat kein SSL laufen):

Wenn es zu dem Hostnamen keine alternativen Backends gibt, kann der Health Check auch ganz entfallen. Der macht eher bei einem Failover Sinn. Ansonsten hilft es nur, dem Client schneller eine Antwort zu liefern, falls es nicht reagiert.

Das zum Einen, zum anderen hilft es aber auch gleich zu sehen ob ein Problem technisch vom Proxy oder Server stammt. Liefert der Proxy wenigstens die Seite, dass kein Backend verfügbar ist, dann weiß man wenigstens schon grob wo man suchen muss 😃

@andyrh
Only bigger than 65xx i have played w. was 12K
MPLS rulz there

@dbx said in Access Webserver on openvpn client (site-to-site):

Ive checked the DNS using the diagnostic tool on the server endpoint and it does resolve to the remote private ip

The point is what IP the browser is using.
That the DNS resolution is working, says sadly nothing. If the browser uses DoH (DNS over HTTPS) he requests a public DNS server and doesn't care about your local DNS settings.

You can check this out in the browsers debugging mode (F12) and look, which IP it is requesting.

You can also capture the traffic on pfSense on the client facing interface. Enter the clients IP into the IP filter and state port "80|443" (means OR) and try to access the web server.
Then look, which IP it is requesting. But you will see some noise there.
However, you can search for the web servers private IP and the public IP.

@dbx said in Access Webserver on openvpn client (site-to-site):

you did also mention previously that there is some special settings on the client side.

The special settings, I meant, are the firewall rules. That you have to ensure that a pass rule on the VPN interface (not group) is applied to the forwarded traffic.

My current outbound NAT rule has:

Interface: SERVER_VPNV4
Source: Client LAN Subnet
NAT Address: SERVER_VPNV4 address
Source Port, Destination, and Destination Port and NAT Port all as *

This rule makes commonly no sense for a site-to-site VPN.
Such masquerading is needed, when you configure a VPN client for a public VPN service.

In a site-to-site you route the traffic to the remote site by entering the remotes network in the VPN settings on both sites.

Das müsste doch auch mit IPsec im Tunnel Mode gehen, wenn man Mobike einsetzt.

Dann müssen, so wie ich das verstanden habe auf die WAN GW Group und man müsste diese mit der richtigen Stickyness einsetzen.

Aber ja viele Wege führen nach Rom, aber muss man da unbedingt hin?

@rcoleman-netgate I'm supposed to be patient when I'm troubleshooting? 🤣 thanks for letting me know.

Welche Version ist im Einsatz, hier kam ja vor kurzem jemand mit einer 2.3 oder 2.4 an.

Es gab Probleme in bestimmten Versionen mit der Log Komprimierung, die sollte man abschalten.

Ein CLI Terminal kopiert das markierte automatisch, kenne das nicht anderes.
Ansonsten einfach printable output in ein Log schreiben.

Was die Hardware angeht, ja würde mal Zeit für was neueres, gibt ja auch die IPU Reihe, wenn du dem Hersteller treu bleiben willst.
Da geht es auch eher um aktuelle Features wie AES usw. Das gibts bei deiner CPU von 2011 noch alles nicht.
Wie schnell ist das Inet und gibt es des PPPoE bei dir?