Port Forwarding stopped working after upgrading to 2.8.0

[comet424]

ok i not able to post my logs right
here is my one drive and i posted it in a text file
log file

and what also drives me crazy is if i google and i goto a link i get this stupid googleservices and it wont let me goto the damn link is there anything in pfsense can fix that problem
constant bs i tell ya

here is a screen shot for the vpn gateway fail over
so i used to have both to tier 1 in the past but i watched another video they said
you do Tier 1 and Tier 2 so you have the usa first and if it fails it moves to 2nd one..

i also enabled the interface for new york vpn but i still get those gateway errors... do you think its killing the php stalling it from all those errors of the tier 1 no gateway?
as i made this this way so i can switch back and forth from usa to canada if needed but mostly stick on the canada one

[stephenw10]

The logs look like you're seeing packet loss on the PPPoE WAN and then obviously seeing similar loss on all the VPNs that are over the WAN.
That in turn is triggering other processes for each gateway event.

The first thing I would do there is set an external monitoring IP on the WAN so you're not reliant on the ISP gateway responding to pings.

Are you routing all traffic via that VPN gateway group?

[comet424]

@stephenw10

so i have my network split so some IPs go out the VPN and others go out the WAN

so the WAN has a built in monitor IP the 2 vpns i set the monitor ips from setups setting up PIA

and what does the monitor IP actually do?
and what ip should i set it?

so you mentioned to change the monitor ip from the isp whats the purpose of that.. for like when i mention it seems to stall out and then i can sometimes ping 1.1.1.1 but i cant ping google.ca or any dns name just ip address..

and here is the screen shot

[comet424]

and i dont know why the one vpn says dynamic both should say dynamic in the settings but one replaces the dynamic with an ip

[comet424]

@stephenw10
i added some new logs

i set the Wan monitor to 1.0.0.1 i couldnt set it to 1.1.1.1 and i lost interent nothing worked.. i could ping 1.1.1.1 from the desktop but i couldnt ping google.ca

i had to reboot pfsense

so i got a log of the wan monitor ip 1.0.0.1 and one after i rebooted same link as i sent ya

[stephenw10]

The gateway monitoring serves two purposes. It generates data for the WAN quality logging in Status > Monitoring so you can see how your WAN varies over time. It is used to detect a connection that has failed (depending how that is defined) so in a gateway group it can be added or removed; like you have with the VPN group.

Since you only have one real WAN you don't need a failover action with that so you could set Disable Gateway Monitoring Action on that. That way it won't trigger unnecessary service starts when there is packet loss on it. You need that action on the VPN gateways in order to failover between them but you could set the thresholds higher so 20% packet loss doesn't trigger it if you regularly see that.

The NY VPN shows as 'dynamic' because it's not connected. That gateway IP is pushed by the server when it connects.

1.0.0.1 should be valid as a monitoring IP, I would expect that to work fine normally.

How do you have DNS configured? Is it setup to only use the VPN gateways for example? That would explain why you could ping an IP but not resolve anything if the VPN is reconnecting.

Also when you set those monitoring IPs pfSense adds a static route to them via the gateway they're on. So when testing connectivity from a LAN side client you should avoid any of those IPs in order to not get static routed via that.

[comet424]

@stephenw10
ok so
1.. i added 2 more logs from last night where it was locking up my desktop to pfsense or internet and seems to be that fail over

2... i set the 1.0.0.1 for the wan at the moment and i can try that 20% threshold

3.. i enabled the 2nd vpn for fail over to see if it locks up

3.. i must have a Routing issues on my Desktop i use like 192.168.0.49 for No VPN and 192.168.0.151 to be on the VPN as i have it split vpn and no vpn ips... and i got vlans

now what i mentioned before on my desktop if i try to search on Homedepot.ca and i get error from there site saying something went wrong... now i can test on my ubuntu VM on my dmz vlan 192.168.40.x which uses WAN and it can search fine... so then i went and tested on my laptop i connect to my LAN and search same item on home depot and i get that same error something went wrong... but if i connect laptop which running windows 11 like the desktop then the search works fine... as i thought maybe its a windows 11 vs ubuntu issue but it wasnt

as for DNS this is what i got setup but i dunno if its the Routing Bypass Policy causing the issue?

but here is what i got
and the 103.x.x.x DNS thats i still got in there from nordvpn i can delete those i guess i didnt yet
so i thinking i got couple issues that routing you meentioned that could be killing php or i thinking comp not powerful enough and its just stalling out
but cpu usage at moment is just 7%

but here is some screen shots maybe i configured things wrong
and i wish on the DNS page you could set like 1.1.1.1 for WAN interface but then keeps VPN and WAN seperate so there is no vpn leakage but it doesnt work that way.. i do have a no wan egreess too

but here is my screen shots. you probably see something i configured wrong
![pfsense 8.png]
(/assets/uploads/files/1752588584425-pfsense-8.png)

oh so when i have 1.1.1.1 1.0.0.1 monitoring ips dont test those ips for pining ping some other ip as it always knows that route there so gives a false postivie i take it

[comet424]

i going to try tommorow or later tonight i going to use one of my backup servers with a amd 3700x i think
i going to install pfsense on a ssd and import the current config and then set it to the 2 nics i got in there and test it,. to see if it locks me out too like i been experiencing in case my 1.5 4 core cpu isnt powerful anymore

[comet424]

so here is searching home depot on the LAN this is with or without behind the vpn

and this here is same search HD on the DMZ network

so definitely i got something configured wrong

[dennypage]

@stephenw10 said in Port Forwarding stopped working after upgrading to 2.8.0:

1.0.0.1 should be valid as a monitoring IP, I would expect that to work fine normally.

FWIW, Cloudflare appears to have had an ICMP hiccup on the West coast yesterday from approximately 14:50 to 15:50 PT. Both 1.0.0.1 and 1.1.1.1 experienced significant on again / off again loss during that time.

[stephenw10]

@dennypage said in Port Forwarding stopped working after upgrading to 2.8.0:

Cloudflare appears to have had an ICMP hiccup

Yup, good point. That could be causing confusion troubleshooting.

But @comet424 how do you have DNS setup for LAN clients? Are they using pfSense (Unbound) for DNS? If so how is Unbound configured? Forwarding to the DNS servers you have configured? Those will only be available via the VPNs. You should remove the NordVPN servers if you're not using NordVPN.

[comet424]

@stephenw10
sorry i ment to get back to you yesterday

so.. if you mean if cloudflare is doing that woth home depot where it shows something wrong thats been going on for a year so i figured it was a vpn issue but when i transfered from nordvpn to pia vpn i still had that issue.. also if i search in google and goto home depot webpage it will load part of the page but it wont display like the prices or anything like that.. also have the same issue with Canadian Tire and some other sites.. i figured it was like a double nat thing or something..

2nd.. my pfsense locked me out again i added another log file to the link when i got back in 10 min after it locked me out well froze me out i wasnt able to ping 192.168.0.1 from the desktop couldnt ssh ... but i got the log file and saved it

but i could ssh in Unraid and i could ssh into pfsense no problem i tried the restart the php-frm that didnt do anything and i couldnt ping google.ca from the shell command but i could ping 1.1.1.1 but weird thing is i couldnt access from my desktop on the 192.168.0.49 ip yet my unraid is on 192.168.0.3 and i could ssh in.. so its like it buggers up the windows dns or whatever wierd but i forgot to test my vm ubuntu on the dmz 192.168.40.x

and i did get a chance to test it on the 8 core cpu gaming motherboard i had stuff to do and didnt get to test it to see if the comp is too slow

but to address your dns stuff

for 192.168.0.x i use 192.168.0.1 as dns
for my IOT 192.168.20.x dns 192.168.20.1
my camera network 192.168.10.x dns 192.168.10.1
my management ip network 192.168.30.x dns 192.168.30.1
my dmz network 192.168.40.x 192.168.40.1

now the management port network i disable the internet as i was having issue of files and internet working through it and not the 192.168.0.1 i use the management port network to be able to WAke on Lan as you cant wake on lan Fiber which some of my comps are 10g fiber network

so ya i ended up having to reboot pfsense just to get it working it seems if it stalls or kills php or so its basiclly useless it doesnt come back to working vpn or not vpn clients trying to reconnect my openvpn connections dont wanna re connect and if they do you cant ping crap.. but when you do a reboot its working again..

so i not sure if you can see anything from the logs but its getting frustrating tommorow for sure i going to run pfsense on my gaming comp to see if its crapping out on there too.. cuz its getting stupid like i dunno why DMZ network can access internet fine like home depot displays fine but LAN doesnt

and with the laptop if i connect to the LAN homedepot will give that error something wrong everytime or wont load pages right.. but you connect to the dmz side and it has no issues. as i was figuring it was just the desktop pc issues but no happens same way on 2 comps.. but basiclly my lan and DMZ use the same in the rules you see nothing special... unless the bypass routing is causing issues or of the pf blockNG and if it has to go after the routing bypass policy as i read that has to be first when splitting wan an vpn to go different routes..

sorry if i confusing too i have dyslexia so it sounds right in my head but may confuse people at times.. so bear with me too...

oh ya also i wasnt able to access 192.168.0.1 through firefox as a docker on unraid it locked out the gui but i was still able to ssh from unraid... but once i rebooted and such i could also gui pfsense from the firefox on the unraid.. its weird how i could ssh from unraid gui but not from windows cmd shell when it froze up or so.. like its a windows hater lol

[comet424]

i also figure maybe cuz of my 3mb/s internet connection where i supposed to get 5mbit on dsl when it maxes out and bad phone line i get that it stalls out and messes up pfsense and it just craps out pfsense

so that fiber 1gb cant come soon enough sometime between now and the end of the year as maybe it wont crash with the 1gb but that cant solve the home depot example i did thats gotta be something miss configured i got

[stephenw10]

@comet424 said in Port Forwarding stopped working after upgrading to 2.8.0:

so.. if you mean if cloudflare is doing that woth home depot where it shows something wrong

No I meant if you are testing connectivity against 1.1.1.1/1.0.0.1 that might have returned false failures yesterday because Cloudflare was having issues. But it could also have applied to cloudflare proxied sites.

@comet424 said in Port Forwarding stopped working after upgrading to 2.8.0:

but i could ssh in Unraid and i could ssh into pfsense no problem

You couldn't ssh into pfSense directly but you could ssh into Unraid and then ssh into pfSense from there?
That sounds more like either pfSense blocking your IP or you have some routing error or IP conflict perhaps.

[comet424]

@stephenw10 ah ok learn something new everyday

ya so on the desktop
i couldnt ssh i couldnt ping i couldnt gui the pfsense.. and the log file you can see i happened about 10 min from the end of that log file

but ya i could ssh from the desktop into unraid.. and then unraid ssh into pfsense and thats when i did the php-frm restart but that did nothing

then i tried Firefox a docker app i have in unraid and i wasnt able to gui the page it was still locked out.. it took 10 min before desktop could gui pfsense and firefox app under unraid could also but for 10 min couldnt

is there any routing i can look for do the rules and nat look ok
i gave up trying to install pfsense on my gaming comp.. pfsense is not compatible with Asus Tuf X570 i spent all day trouble shooting bios and everything it will not boot properly off usb.. i even tried installing pfsense on a HD from unraid set it up and plugged it in.. oh no craps right out..

yet i tried your nemis's opnsense oh it has no issues installing on Asus x570 tuf gaming motherboard... i figured it was usb issues so i tried 3 different 32gb usbs no still craps out mid running on the usb.. opnsense likes asus and pfsense hates asus... i even tried disabling network card in bios and pulled out my hba card and my 10gigtek nic and still wasnt able to install pfsnese it boots partially and then just locks up and gives you a prompt for keyboard panic mode

so i couldnt test out if a better processor would help..

in the 80s i struggled with token ring and dip switchs and coaxal networking and slow butt 1200 baud modem.. today i struggle OS doesnt install on a gaming

comp lol time sure flies

is the pfblockng ok to be before or after the routing policy does any of the rules look wrong.

here some cell phone screen caps cell sucks.. but i spent all day and just got fed up trying.. to try a different comp asus tuf x570 gaming 32 gig ram and i think its a 5600G cpu

[stephenw10]

That's a known issue with the upstream firmware API. See: https://redmine.pfsense.org/issues/16237

Pretty easy workaround shown there until it's fixed.

[comet424]

@stephenw10
ah ok i read that link i not sure how you disable what in the bios

ill give it a try tommorow
create a file
/boot/loader.conf.local

and place in it
hint.iwm.0.disabled="1"

and thats all i need to do ?
not sure how you interupt the boot proccess
and then you type
set hint.iwm.0.disabled="1"
boot

right now it locks up the comp when it boots so i ended up saying i had enough of it today and had a drink lol...

so is it a freebsd issue or is it a pfsense issue?

and i never got the routing to not stop crapping out desktop was locked out again. makes you wanna drink too lol

and is there a time frame when they fix it.. i guess the next release of pfsense 6 months down the road? but like 2.8.1 probablly

[stephenw10]

@comet424 said in Port Forwarding stopped working after upgrading to 2.8.0:

create a file
/boot/loader.conf.local

and place in it
hint.iwm.0.disabled="1"

and thats all i need to do ?

Yes. Do that in 2.7.2 before upgrading.

@comet424 said in Port Forwarding stopped working after upgrading to 2.8.0:

not sure how you interupt the boot proccess

When you see the loader menu with the 3 second countdown press ESC to reach the loader prompt:

    _ __  / _|___  ___ _ __  ___  ___      _                
   | '_ \| |_/ __|/ _ \ '_ \/ __|/ _ \   _| |_              
   | |_) |  _\__ \  __/ | | \__ \  __/  |_   _|             
   | .__/|_| |___/\___|_| |_|___/\___|    |_|               
   |_|                                                      
                                                            
                                                                              
 ╔════ Welcome to Netgate pfSense Plus ════╗      __________________________  
 ║                                         ║     /                       ___\ 
 ║  1. Boot Multi user [Enter]             ║    |                      /`     
 ║  2. Boot Single user                    ║    |                     /    :-|
 ║  3. Escape to loader prompt             ║    |      _________  ___/    /_ |
 ║  4. Reboot                              ║    |    /` ____   / /__    ___/ |
 ║  5. Cons: Serial                        ║    |   /  /   /  /    /   /     |
 ║                                         ║    |  /  /___/  /    /   /      |
 ║  Options:                               ║    | /   ______/    /   /  _    |
 ║  6. Kernel: default/kernel (1 of 1)     ║    |/   /          /   / _| |_  |
 ║  7. Boot Options                        ║        /          /___/ |_   _| |
 ║  8. Boot Environments                   ║       /                   |_|   |
 ║                                         ║      /_________________________/ 
 ║                                         ║                                  /
 ╚═════════════════════════════════════════╝
                                                                               
Exiting menu!


Type '?' for a list of commands, 'help' for more detailed help.
OK 

[comet424]

@stephenw10 i wish they explained it better like you did.. they so criptied like engineer writtings

so i having issues installing pfsense i setup the dhcp and it then checkjing for netgate servers but in the end it fails it cant detect any netgate servers to install pfsense

is netgate servers down to install pfsense... i wish you could just install it and not need to download it .. like a usb installer with pfsense 2.7.2 already on the usb or the 2.8.

i even tried shutting off my dsl modem.. for 20 min and also set in pfsense installer dns to 1.1.1.1 but it will not get past the netgate servers cant be fetched..
do you know if they down or so

i did find a iso of 2.7.2 so i got that installed on the gaming comp but i cant import the 2.8.0 configuration file into 2.7.2 so i guess ill have to wait till comp can update to 2.8.0 as there is no iso of 2.8.0 which sucks as it would help right now to get things back up and running