1 Gig Fios and PFSense

jbeez

@sstatjm two suggestions,

first in the wan interface https://pfsense/interfaces.php?if=wan at the bottom are two checkboxes, they shouldn't be causing an issue but you have a strange issue, so uncheck the blocking of private networks. If that doesn't solve it its easy to revert

second, you can add your firewall rule in the floating tab to have it process before all other rules. your rule looks good as posted but I'm not sure where you have it in the loading order

Are you able to pcap the traffic with verizon on your wan interface to see what's happening after that 5 minutes?

Can you post how your WAN interface is configured? Here are the relevant sections from mine:

If you want to ssh in, I just looked at my /var/db/dhclient.leases.igb0 here's an example of my last entry

lease {
  interface "igb0";
  fixed-address {redacted}.95;
  option subnet-mask 255.255.255.0;
  option routers {redacted}.1;
  option domain-name-servers 71.242.0.12,71.250.0.12;
  option domain-name "verizon.net";
  option dhcp-lease-time 7200;
  option dhcp-message-type 5;
  option dhcp-server-identifier 96.227.128.1;
  renew 5 2023/1/13 14:21:34;
  rebind 5 2023/1/13 15:06:34;
  expire 5 2023/1/13 15:21:34;
}

sstatjm

@jbeez

Gertjan

@sstatjm said in 1 Gig Fios and PFSense:

Pass all a new rule on the WAN ....

Remove this WAN rule as it should not and will not repair or explain your issue.
It's pfSense that "connects to" the upstream gateway through the WAN.
Never ever some outside source should be able to connect to your WAN so it can access some WAN (or LAN) hosted service.

The exception would be a natted port, like a VPN hosted on pfSense. That WAN rule would only accept connection using UDP, using port 1194. The answering process would be (only) the VPN server running on pfSense.

There are not much details in this thread, so I'll add my own interrogations.
What if the DHCP handshake isn't done correctly ?
My own ISP uses DHCP, packets should be encapsulated in a VLAN channel '835'. Special DHCP options should be added so my ISP 'login' and ISP 'password' are (encoded) send with the initial DHCP-REQUST command. If not done so, I get an IP and a degraded connection, I guess to re negotiate the access ?

Your ISP router will work because ( ? ) it will update itself, and adapt if a new login procedure is needed. So its normal it would work just fine.
Btw : it isn't the ISP that updates the ISP router. It will be the ISP router that requests if an update is available, and update itself if that is the case.

Again, I mention my own experiences. I can't help with what your ISP 'wants' so you get a good connection.
The connection (the link) is fine, as when you use the ISP router, all is well.
So I tend to say : your ISP router does something that pfSense doesn't and that's why the ISP closes the door.

Btw : don't even think of calling your ISP about this.
Because you know what they will say "use OUR router, others are not supported by us (your router is supported by you)".
The path to the solution is : Go 'Google' and look who is using your ISP and is also using pfSense.

sstatjm

@gertjan When I just got the service I had no issues for weeks and then one day. I started to experience packet loss and then no internet. If all else fails I will just put the ip assign to pfsense from the router and put it in DMZ.
I notice that DCHP updates itself every hour when i was going thru a log i found when logged into the router

Gertjan

@sstatjm said in 1 Gig Fios and PFSense:

When I just got the service I had no issues for weeks and then one day

What works yesterday just fine can change tomorrow.
As I explained above : what if the ISP changes the connect procedure ?
You'll agree with me that something has changed ;)
You, as a end user, won't notice anything of all this because the ISP can 'instruct" (update) your ISP router (that is : the router will ask for an update, and get one if it's available).
The result is that your connection always works when you use the ISP router.

The thing is : pfSense can't know what changed unless you changed settings.

jbeez

@gertjan said in 1 Gig Fios and PFSense:

@sstatjm said in 1 Gig Fios and PFSense:

Pass all a new rule on the WAN ....

Remove this WAN rule as it should not and will not repair or explain your issue.
It's pfSense that "connects to" the upstream gateway through the WAN.
Never ever some outside source should be able to connect to your WAN so it can access some WAN (or LAN) hosted service.

The exception would be a natted port, like a VPN hosted on pfSense. That WAN rule would only accept connection using UDP, using port 1194. The answering process would be (only) the VPN server running on pfSense.

There are not much details in this thread, so I'll add my own interrogations.
What if the DHCP handshake isn't done correctly ?
My own ISP uses DHCP, packets should be encapsulated in a VLAN channel '835'. Special DHCP options should be added so my ISP 'login' and ISP 'password' are (encoded) send with the initial DHCP-REQUST command. If not done so, I get an IP and a degraded connection, I guess to re negotiate the access ?

Your ISP router will work because ( ? ) it will update itself, and adapt if a new login procedure is needed. So its normal it would work just fine.
Btw : it isn't the ISP that updates the ISP router. It will be the ISP router that requests if an update is available, and update itself if that is the case.

Again, I mention my own experiences. I can't help with what your ISP 'wants' so you get a good connection.
The connection (the link) is fine, as when you use the ISP router, all is well.
So I tend to say : your ISP router does something that pfSense doesn't and that's why the ISP closes the door.

Btw : don't even think of calling your ISP about this.
Because you know what they will say "use OUR router, others are not supported by us (your router is supported by you)".
The path to the solution is : Go 'Google' and look who is using your ISP and is also using pfSense.

As far as I know he's on the same ISP as I am, and there is no VLAN encapsulated traffic happening there. a GPON connection goes to an ONT device, and then hands off to either MOCA or an rj45 ethernet for data. As long as his ONT is provisioned for data over the ethernet port it does a standard DHCP request/response proceedure.

I suggested the wan rule as a troubleshooting step to see if the dhcp packets during the renewal phase were getting caught up. I have several unanswered questions still so I can't help much further here.

jbeez

@sstatjm What network interfaces are you using(hardware). Do you have any log messages? Maybe a hardware issue on the pfsense box?

Do you have anything else to test with?

sstatjm

@jbeez Running on a ASRock H370M-ITX motherboard with a Intel(R) Core(TM) i5-9500 CPU, NICs are Dual Intel LAN.
Dont have anything else at the moment to test with.

Currently I am getting - dpinger 99776 WAN_DHCP 192.168.1.1: sendto error: 13

2 those every sec

jbeez

@sstatjm In system-> Routing. WAN_DHCP ->

See if the issue persists. Not sure what it can do if your gateway goes down, you don't have a backup WAN link right?

jbeez

Also... any dpinger messages in my log about WAN_DHCP show the public IP of my Verizon FiOS upstream gateway. Your log message is showing an RFC1918 address.

Physically are you connecting just your wan nic right to your ONT? You aren't bridging those interfaces in pfsense at all are you?

mikeinnyc

WAN_DHCP Gateway ??? with your PFS Box, you plugged it in and just got a working WAN IP DHCP lease right away...until they changed it

Most likely it's not your PFSense router. It's most likely that Verizon has changed your Public IP DHCP block along with the DNS servers that Verizon uses to resolve. You did nothing wrong so don't worry. I am assuming you are willing to have your IP address changed X times per month in exchange for a lower-rate nonstatic IP. Verizon "may" change your IP faster than Toliet Paper in the NYC Subways but only if 1) you get flagged by pattern high burst traffic usage or by multiple open ports for longer than normal periods. 2) OR, It could just be your power failed and it changed the "WAN IP and the DNS servers."

I know it's hard but try to pay up and Go Static IP if you are running more than cameras outside specifically file sharing aka web hosting. Otherwise, you'll be unplugging that ONT box and trying to get a new lease may be hourly even with ddns. Best of luck and I hope you get a DHCP ... static lease for at least one year.

sstatjm

@jbeez at this point I am just exhausted. I am currently connected to the verizon router that's why you are seeing that ip address.
To get a stable connection I have to connect to the verizon router. the moment i connect directly the ONT all hell breaks lose and service drops. Yes, i did call them to release the DCHP lease........... 4hrs and 45 mins later. Back to square one.

jbeez

@mikeinnyc said in 1 Gig Fios and PFSense:

WAN_DHCP Gateway ??? with your PFS Box, you plugged it in and just got a working WAN IP DHCP lease right away...until they changed it

Most likely it's not your PFSense router. It's most likely that Verizon has changed your Public IP DHCP block along with the DNS servers that Verizon uses to resolve. You did nothing wrong so don't worry. I am assuming you are willing to have your IP address changed X times per month in exchange for a lower-rate nonstatic IP. Verizon "may" change your IP faster than Toliet Paper in the NYC Subways but only if 1) you get flagged by pattern high burst traffic usage or by multiple open ports for longer than normal periods. 2) OR, It could just be your power failed and it changed the "WAN IP and the DNS servers."

I know it's hard but try to pay up and Go Static IP if you are running more than cameras outside specifically file sharing aka web hosting. Otherwise, you'll be unplugging that ONT box and trying to get a new lease may be hourly even with ddns. Best of luck and I hope you get a DHCP ... static lease for at least one year.

idk if this is the same everywhere, but I'm in the Phila area, and my verizon wan IP has changed maybe three times in the last 2yrs, its very stable for me.

Again I don't work at verizon, IDK what they do, how consistent it is, but I can speak to my personal experience, I also have some unifi gateways and edgerouters on maybe 4 or 5 other verizon networks that I admin and they are also stable, although most of them are in this area as well.

sstatjm

@jbeez It was a frustrating night last night. What I ended up doing was going back to the verizon provided router and connect my pfsense to it. I put that ip address into the DMZ. So far that is working. But now another issue has arise from all this now my upload speed is about a 1/3 of the 1gig speed advertised.

This is just fun!!

mikeisfly

For what it's worth, I have Verizon FiOS and Gigabit internet and I have no issues. I'm connected directly to the ONT. I would check to make sure your Fiber is clean. You may be able to have Verizon come out and check the connections. If possible, use other hardware like a traditional router just to see if it maybe an issue with your hardware.

dma_pf

@sstatjm

@jbeez said in 1 Gig Fios and PFSense:

idk if this is the same everywhere, but I'm in the Phila area, and my verizon wan IP has changed maybe three times in the last 2yrs, its very stable for me.
Again I don't work at verizon, IDK what they do, how consistent it is, but I can speak to my personal experience, I also have some unifi gateways and edgerouters on maybe 4 or 5 other verizon networks that I admin and they are also stable, although most of them are in this area as well.

I'm also in the northeast US. Been on FIOS for about 3 years and have never had any issues connecting pFsense directly to their ONT. IP address assignments have also been very "sticky" with only a few changes over the years.

I do have pfsense set to only use IPv4...no IPv6. I've never used Verizon's DNS servers. I don't use any monitoring on the gateway. Here's my interface and gateway.

I've never had to mess around with any other settings to get FIOS to work.

With the FIOS router attached to the ONT you can have Verizon run some diagnostics to try to rule out any issues behind the ONT. This sounds like this is probably not the case.

There's probably something in pfsense that's setup wrong or not working correctly. I would suggest making a local backup of your current pfsense configuration (Diagnostics/Backup & Restore), then do a reset to factory defaults (Diagnostics/Factory Defaults), then reboot and see how the connection works.

sstatjm

@dma_pf What version of pfsense are you running?

dma_pf

@sstatjm 2.6

sstatjm

@dma_pf ok cool thanks. I just rebuild it and so far its holding up good. 26 mins and counting
So now what's next!!! anybody have any suggestions on what I should keep?

mikeinnyc

@jbeez I worked at the other Blue cable company. VZ tech guys had some Fked up stories. Both company employees eventually shared the same employment multiple times back and forth. It's like the Philly cheese steak cooks immigrated to NYC and then back to Philly and then told stories. No, the grass is not greener! Don't ever call tech and tip them off is all that I'll say. "Their job is to collect more revenue on the quota chain you just don't know it." One can go months straight under the radar...until someone does their admin network monitoring job. Watch this Video and you'll laugh

Stay low my friends and never call in for anything if you don't have to. Just open up that ONT yourself on the side of your house. Unplug that black box for a few minutes then plug it back in and close the ONT unit. P.S. this is not good tech advice at all... I ate too many lead paint chips!

For the entrepreneur, you certainly can use a dual wan but the primary should be business static and one of them can be a residential DHCP but not both. Plus, google will index higher-ranking static IPs over ever-changing residential IPs. A simple way to check is to go to Business Static IP check Type = business or residential?

I just want to hear a few stories of customers winning on DHCP leases. Meaning maybe it changes once per year. The problem is just when you think you won they change the IP address more than normal. Now back to paint chips.