multiple machines some sites won't load first time especially google sites



  • Here's my setup and a little background. I'm new to learning firewalls, pfsense is my first one, as a consequence I keep blowing up the home fios connection and my family has threatened to kill me, so what i've done is leave the fios router plugged in and i've plugged my pfsense in as a dmz'd dhcp client on the fios router.

    Using two separate win 10 machines plugged in directly to the fios router, I successfully tested their connectivity against various sites websites

    when I plug into my pfsense and try to access those same sites it seems most work 99% except for google sites. google.com, mail.google.com, gmail.com, drive.google.com, docs.google.com. The google sites consistently fail to load until i refresh the browsers (chrome and firefox), sometimes I need to refresh 2 and 3 times.

    iv'e tried leaving the pfsense wan dns server entries blank, I've also tried setting them to several different dns addresses including 8.8.8.8, 8.8.4.4, 1.1.1.1, 198.153.194.1, 204.117.214.10, 129.250.35.250.

    In chrome I've tried clearing the cache, clearing the HSTS cache, i've reset my network connection by running

    arp -d *
    netsh int ip reset
    netsh winsock reset
    ipconfig /flushdns
    ipconfig /release
    ipconfig /renew

    and rebooting both the pfsense and computers multiple times. I've restored to factory defaults, restored backups, nothing seems to work.

    nslookup yields odd results, right now its failing to resolve google sites

    nslookup
    Default Server: pflookup.local
    Address: 192.168.35.1

    food.net
    Server: pflookup.local
    Address: 192.168.35.1

    Non-authoritative answer:
    Name: food.net
    Address: 3.235.229.168

    food.com
    Server: pflookup.local
    Address: 192.168.35.1

    Non-authoritative answer:
    Name: food.com
    Addresses: 52.20.42.213
    35.175.52.53

    google.com
    Server: pflookup.local
    Address: 192.168.35.1

    *** pflookup.local can't find google.com: Server failed

    docs.google.com
    Server: pflookup.local
    Address: 192.168.35.1

    *** pflookup.local can't find docs.google.com: Server failed

    drive.google.com
    Server: pflookup.local
    Address: 192.168.35.1

    *** pflookup.local can't find drive.google.com: Server failed

    task.com
    Server: pflookup.local
    Address: 192.168.35.1

    Non-authoritative answer:
    Name: task.com
    Addresses: 165.160.15.20
    165.160.13.20

    cnn.com
    Server: pflookup.local
    Address: 192.168.35.1

    Non-authoritative answer:
    Name: cnn.com
    Addresses: 2a04:4e42::323
    2a04:4e42:400::323
    2a04:4e42:600::323
    2a04:4e42:200::323
    151.101.1.67
    151.101.129.67
    151.101.193.67
    151.101.65.67

    but 30 seconds later it works

    google.com
    Server: pflookup.local
    Address: 192.168.35.1

    Non-authoritative answer:
    Name: google.com
    Addresses: 2607:f8b0:4006:806::200e
    142.250.64.78

    drive.google.com
    Server: pflookup.local
    Address: 192.168.35.1

    Non-authoritative answer:
    Name: drive.google.com
    Addresses: 2607:f8b0:4006:810::200e
    172.217.10.78

    docs.google.com
    Server: pflookup.local
    Address: 192.168.35.1

    Non-authoritative answer:
    Name: docs.google.com
    Addresses: 2607:f8b0:4006:818::200e
    172.217.3.110

    I've changed my dhcp range, i briefly tried dns forwarding but i screwed that all up and took down the whole network in the house, even things not connected to the pfsense

    in the fios router here are my wan dns servers

    71.243.0.12
    68.237.161.12

    I'm running 2.4.5-RELEASE-p1 (amd64)
    built on Tue Jun 02 17:51:54 EDT 2020
    FreeBSD 11.3-STABLE

    any clue what im doing wrong?

    thank you
    Ron



  • I have 2 theories on my problem.

    1> it appears as though my fios is not passing ipv6 traffic and i believe the google dns prefers ipv6

    2> double nat causing issues



  • @rterren said in multiple machines some sites won't load first time especially google sites:

    1> it appears as though my fios is not passing ipv6 traffic and i believe the google dns prefers ipv6

    nslookup will ask the DNS (your pfSense = 192.168.35.1) both an A and AAAA record for a zone.
    This will work fine over a IPv4 and/or a IPv6.

    @rterren said in multiple machines some sites won't load first time especially google sites:

    2> double nat causing issues

    You NATted some ports(s) in the FIOS and pfSense.
    These NAT ruls only get used for incoming traffic, does not impact outgoing DNS traffic.

    By default, pfSense will resolve. This means it doesn't use upstream DNS resolvers or forwards like the ones from your ISP, or even commercial " 8.8.8.8, 8.8.4.4, 1.1.1.1, 198.153.194.1, 204.117.214.10, 129.250.35.250" (they use/sell your request data).
    The resolver contacts directly the main Root name servers. These servers know all about everybody, because they are the "name part" of the domains names on the Internet. The nice thing about them is : no set-up is needed, it will work out of the box. I advise you strongly not to change any DNS settings, neither entering any "DNS" IP manually.
    So :
    @rterren said in multiple machines some sites won't load first time especially google sites:

    iv'e tried leaving the pfsense wan dns server entries blank,

    doesn't need any effort. Just do nothing, and you'll be fine.

    On the General Setup page, these are the perfect settings :

    4ee5bcd6-de18-47f2-be4c-945f50799f87-image.png

    Even when when you hook up pfSense behind some other (ISP) router, your FIOS, it will obtain an IP on it's WAN interface as any other device (printer, PC, etc) you hooked up on the FIOS LAN. It will be an RFC1918 of course, an IP, from the LAN of the FIOS router..
    Just be sure that that WAN (pfSEnse) IP - network doesn't conflict with the pfSense LAN network.

    7e8b1f07-6be8-4538-b12e-d2d6d83b62a8-image.png

    You got this wrong :

    @rterren said in multiple machines some sites won't load first time especially google sites:

    as a consequence I keep blowing up the home fios connection

    normally, you should apply @home settings that are known to work.
    And when you make changes, be ready to
    Test - be able to ask the "what if ?" question and go back if needed.
    Use a backup of your config if needed.

    Fooling around with pfSense should be done @work. When things go down @work, you're not risking your live. It's the other way around : they'll pay you more when things go wrong.

    edit : I forgot to mention the reason why sometimes resolving works, and sometimes it doesn't.
    When multiple DNS servers are entered on the General settings page, they are used on a round robin base.
    If one of them doesn't work out, there will be no answer. The next request will use the next DNS in line, and have an answer.
    When you enter nothing here, the default 127.0.0.1 will get used (by pfSEnse itself == the resolver and the same resolver will also serve all pfSense LAN based devices). That will always work. That is, as long as the resolver has a free access to the main 13 Internet roots servers.

    Btw : your LAN devices should always stay in the "network" mode as they were when you bought them : DHCP activated, no static settings. A PC today, with a default W10, never needs any user intervention. If it does, you already have network issues.
    If you prefer that some LAN based devices always have the same IP, set up a static MAC lease for that PC on pfSense.

    Last but not least : pfSense is a router / firewall , pretty identical to any other router. There is no "Networking the pfSense way". All the SOHO firewall routers are the same. pfSense has more functionalities, which could make you think it is more complex. That's not true : just don't use the features you don't need / don't understand, and you'll be fine.



  • @Gertjan Wow thats an awesome write up thanks for all of it. I decided to just completly remove the FIOS router and everything is working great now. I would have done it @work but I don't have a job, thanks covid.. lol


Log in to reply