Traffic blocked randomly



  • Hi,
    I have an issue with PfSense:
    When I start a connection, everything is fine and then for mobile phone for example, when I stop using it and come back 5 minutes later, I ve got error timeout.
    I can't access to local network too
    It is the same on my computer.
    But the DHCP leases are still active because I let the DHCP leases last 2 hours (default time)

    When I get these error, I'm trying to ping IP Adresses and domain names such as 8.8.8.8 1.1.1.1 google.com and it fails.
    Seems that the firewall is blocking traffic randomly
    But I've created a pass firewall rules that let everything pass but it still doesn't work
    Thank you for your help
    Jules


  • LAYER 8 Global Moderator

    @jules13 said in Traffic blocked randomly:

    Seems that the firewall is blocking traffic randomly

    Yeah doesn't work that way ;)

    Out of the box if pfsense blocked traffic it would LOG it..

    If you can not even resolve google.com that points to connectivity issue either to pfsense itself from your client or for pfsense to internet, or maybe unbound is failing..

    pinging 8.8.8.8 removes unbound failing as a problem - but still you do not know if you can not reach pfsense or pfsense can not reach 8.8.8.8

    Can you ping pfsense IP when this happens? Access pfsense gui? Does the log or the gui show that your wan is offline or taking packet loss? Check the log did unbound recently restart? Or fail, etc. etc.



  • When It happens mostly on mobile devices such as IOS devices and sometimes on my laptop:
    ping 8.8.8.8 => Error
    ping google.com => Error
    ping 192.168.1.1 => Success
    Pfsense GUI => Error Network protocol Violation (firefox)
    Access NAS (local network) => Error

    Then I disconnect of my wifi N and reconnect to my wifi AC (WiFi access point connected to pfsense) or I renew DHCP Lease on my device and it works again few minutes
    The error appear when I stop using internet (device standby mode) and I reopen it.

    I stopped DNS resolver services and I use :
    1.1.1.1
    1.0.0.1
    9.9.9.9
    208.67.222.222
    208.67.220.220
    DNS server.

    Where can I access WAN logs (system logs? or dignostic)

    Thank you for your help
    Jules



  • @jules13 said in Traffic blocked randomly:

    I stopped DNS resolver services and I use :
    1.1.1.1
    1.0.0.1
    9.9.9.9
    208.67.222.222
    208.67.220.220

    And when you remove all these IP's, and go back to the initial situation, the one that you found when you installed pfSEnse what will happen ?
    I'll give you the answer : no more issues. I'll leave it up to you to draw your conclusions.

    You said you stopped the DNS Resolver. Then who does the DNS for pfSense itself ??

    @jules13 said in Traffic blocked randomly:

    Where can I access WAN logs (system logs? or dignostic)

    Except for some very typical situations (NAT, etc), no connections can be initiated from WAN. No need to log incoming connections on this interface yjat are blocked anyway. Not related to DNS.



  • @gertjan said in Traffic blocked randomly:

    And when you remove all these IP's, and go back to the initial situation, the one that you found when you installed pfSEnse what will happen ?
    I'll give you the answer : no more issues. I'll leave it up to you to draw your conclusions.

    You said you stopped the DNS Resolver. Then who does the DNS for pfSense itself ??

    I did a factory reset, and I still have no internet randomly.
    On my laptop I had timeout error with all website (including gui) and I couldn't ping 8.8.8.8 and google.com
    google.com => couldn't solve hostname
    8.8.8.8 => Timeout no answers.

    Thank you for your help
    Jules


  • LAYER 8 Global Moderator

    @jules13 said in Traffic blocked randomly:

    I stopped DNS resolver services and I use :

    So you have your clients directly access those name servers?

    You can for sure do that if you so desire.. But let me point out a flaw in your selection. Quad 9 filters, while 1.1.1.1 does not - opendns filters as well - but could be different than what quad9 filters.

    So now you run into a conundrum, you have non filtering dns, and filtering dns that filter differently... If you point your client to all of those you have no idea which one your client will actually query or get an answer from. So sites might be blocked, they might not be blocked.. This is a borked configuration out of the gate. If your going to point to external dns - then pick 1 company!! So you are sure your always going to get either unfiltered for filtered the same way from any of their dns you might query.

    When you say you try and access NAS - are you using an IP, how wold 1.1.1.1 resolve the IP of your local NAS? if your asking them for example.

    As to network protocol error when you try and access the gui? Yeah going to need more info - is that some error your browser issues.. Does it also give you the same error in say chrome?

    You will find all the logs - in such a crazy place ;)
    Status / System Logs

    As to factory reset and can't resolve google.. Can you get to the pfsense gui, can you ping pfsense IP? Is unbound even running? Does your isp block dns queries and only allows you to use their dns? What does the unbound log say - up the level of logging if need be in unbound, etc.

    Can you resolve pfsense own name from unbound? What does pfsense diag / dns lookup show when you ask it to resolve www.google.com ?



  • @johnpoz said in Traffic blocked randomly:

    You can for sure do that if you so desire.. But let me point out a flaw in your selection. Quad 9 filters, while 1.1.1.1 does not - opendns filters as well - but could be different than what quad9 filters.

    So now you run into a conundrum, you have non filtering dns, and filtering dns that filter differently... If you point your client to all of those you have no idea which one your client will actually query or get an answer from. So sites might be blocked, they might not be blocked.. This is a borked configuration out of the gate. If your going to point to external dns - then pick 1 company!! So you are sure your always going to get either unfiltered for filtered the same way from any of their dns you might query.

    As I realised with @Gertjan. I did an error, therefore I factory reset the PfSense router and the issue is still there

    As to network protocol error when you try and access the gui? Yeah going to need more info - is that some error your browser issues.. Does it also give you the same error in say chrome?

    It is a firefox issue, i'm trying to see what happen with chrome:

    You will find all the logs - in such a crazy place ;)
    Status / System Logs

    Thank you

    As to factory reset and can't resolve google..
    Can you get to the pfsense gui,

    It's the same as before: It can be working 10 minutes and then I have timeout errors

    Can you ping pfsense IP?

    Yes

    Is unbound even running?

    Seems so : in the log I've got :

    Mar 6 13:12:10 	unbound 	35451:0 	info: start of service (unbound 1.8.1). 
    

    Does your isp block dns queries and only allows you to use their dns?

    No, because I used opendns DNS with linksys router in the past

    What does the unbound log say - up the level of logging if need be in unbound, etc.

    Mar 6 13:39:43 	unbound 	44190:0 	info: reply from <co.> 156.154.100.25#53
    Mar 6 13:39:43 	unbound 	44190:0 	info: query response was REFERRAL
    Mar 6 13:39:43 	unbound 	44190:0 	info: processQueryTargets: ns2.radioline.co. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: skipping target due to dependency cycle (harden-glue: no may fix some of the cycles) ns2.radioline.co. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: new target ns1.radioline.fr. AAAA IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: sending query: ns2.radioline.co. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	debug: sending to target: <radioline.co.> 195.210.43.139#53
    Mar 6 13:39:43 	unbound 	44190:0 	debug: iterator[module 1] operate: extstate:module_state_initial event:module_event_pass
    Mar 6 13:39:43 	unbound 	44190:0 	info: iterator operate: query ns1.radioline.fr. AAAA IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: resolving ns1.radioline.fr. AAAA IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: finishing processing for ns1.radioline.fr. AAAA IN
    Mar 6 13:39:43 	unbound 	44190:0 	debug: validator[module 0] operate: extstate:module_state_initial event:module_event_moddone
    Mar 6 13:39:43 	unbound 	44190:0 	info: validator operate: query ns1.radioline.fr. AAAA IN
    Mar 6 13:39:43 	unbound 	44190:0 	debug: iterator[module 1] operate: extstate:module_wait_reply event:module_event_pass
    Mar 6 13:39:43 	unbound 	44190:0 	info: iterator operate: query ns2.radioline.co. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: processQueryTargets: ns2.radioline.co. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	debug: cache memory msg=68811 rrset=134872 infra=66387 val=44275
    Mar 6 13:39:43 	unbound 	44190:0 	debug: iterator[module 1] operate: extstate:module_wait_reply event:module_event_reply
    Mar 6 13:39:43 	unbound 	44190:0 	info: iterator operate: query fr. DNSKEY IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: response for fr. DNSKEY IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: reply from <fr.> 193.176.144.22#53
    Mar 6 13:39:43 	unbound 	44190:0 	info: query response was ANSWER
    Mar 6 13:39:43 	unbound 	44190:0 	info: processQueryTargets: fr. DNSKEY IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: sending query: fr. DNSKEY IN
    Mar 6 13:39:43 	unbound 	44190:0 	debug: sending to target: <fr.> 194.0.9.1#53
    Mar 6 13:39:43 	unbound 	44190:0 	debug: cache memory msg=69007 rrset=135938 infra=66639 val=44275
    Mar 6 13:39:43 	unbound 	44190:0 	debug: iterator[module 1] operate: extstate:module_wait_reply event:module_event_reply
    Mar 6 13:39:43 	unbound 	44190:0 	info: iterator operate: query fr. DNSKEY IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: response for fr. DNSKEY IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: reply from <fr.> 194.0.9.1#53
    Mar 6 13:39:43 	unbound 	44190:0 	info: query response was ANSWER
    Mar 6 13:39:43 	unbound 	44190:0 	info: finishing processing for fr. DNSKEY IN
    Mar 6 13:39:43 	unbound 	44190:0 	debug: validator[module 0] operate: extstate:module_wait_module event:module_event_moddone
    Mar 6 13:39:43 	unbound 	44190:0 	info: validator operate: query fr. DNSKEY IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: validated DNSKEY fr. DNSKEY IN
    Mar 6 13:39:43 	unbound 	44190:0 	debug: validator[module 0] operate: extstate:module_wait_subquery event:module_event_pass
    Mar 6 13:39:43 	unbound 	44190:0 	info: validator operate: query cache.radioline.fr. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: NSEC3s for the referral proved no DS.
    Mar 6 13:39:43 	unbound 	44190:0 	info: Verified that unsigned response is INSECURE
    Mar 6 13:39:43 	unbound 	44190:0 	debug: cache memory msg=69007 rrset=135938 infra=66639 val=45507
    Mar 6 13:39:43 	unbound 	44190:0 	debug: iterator[module 1] operate: extstate:module_wait_reply event:module_event_reply
    Mar 6 13:39:43 	unbound 	44190:0 	info: iterator operate: query ns2.radioline.co. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: sanitize: removing potential poison RRset: ns1.radioline.fr. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: response for ns2.radioline.co. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	info: reply from <radioline.co.> 195.210.43.139#53
    Mar 6 13:39:43 	unbound 	44190:0 	info: query response was ANSWER
    Mar 6 13:39:43 	unbound 	44190:0 	info: finishing processing for ns2.radioline.co. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	debug: validator[module 0] operate: extstate:module_state_initial event:module_event_moddone
    Mar 6 13:39:43 	unbound 	44190:0 	info: validator operate: query ns2.radioline.co. A IN
    Mar 6 13:39:43 	unbound 	44190:0 	debug: cache memory msg=69007 rrset=135938 infra=66639 val=45507 
    

    Can you resolve pfsense own name from unbound?

    How can I do this ?

    What does pfsense diag / dns lookup show when you ask it to resolve www.google.com ?

    I've got this:

    Name server 	Query time
    127.0.0.1	0 msec
    192.168.2.1	12 msec
    

    But I can acces GUI only when I have internet, because when I have the error I can"t access GUI


  • LAYER 8 Global Moderator

    @jules13 said in Traffic blocked randomly:

    192.168.2.1 12 msec

    Who is this? Is this upstream nat router in front of pfsense?

    Your log seems to show unbound working fine. What does your quality of connection in pfsense show you?

    query whatever name you called pfsense..
    simple nslookup or dig, or host or whatever your other fav dns query tool might be. My pfsense is called sg4860.local.lan

    $ dig sg4860.local.lan
    
    ; <<>> DiG 9.12.3-P1 <<>> sg4860.local.lan
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5577
    ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 4096
    ;; QUESTION SECTION:
    ;sg4860.local.lan.              IN      A
    
    ;; ANSWER SECTION:
    sg4860.local.lan.       3600    IN      A       192.168.9.253
    
    ;; Query time: 0 msec
    ;; SERVER: 192.168.9.253#53(192.168.9.253)
    ;; WHEN: Wed Mar 06 08:05:41 Central Standard Time 2019
    ;; MSG SIZE  rcvd: 61
    

    Nslookup

    > sg4860.local.lan
    Server:  sg4860.local.lan
    Address:  192.168.9.253
    
    Name:    sg4860.local.lan
    Address:  192.168.9.253
    

    But seems more like you have a local connectivity issue... Does this happen on client when your wired to pfsense vs wifi? Lets see pfsense quality graph

    0_1551881447133_qualitygraph.png

    This will show you if pfsense wan connection is going offline or having lots of packet loss, etc.



  • @johnpoz said in Traffic blocked randomly:

    @jules13 said in Traffic blocked randomly:

    192.168.2.1 12 msec

    Who is this? Is this upstream nat router in front of pfsense?

    Yeah I must have a double NAT configuration because ISP in France don't allows you to dc the box and there is no bridge mode.
    So I've got this:

    FTTH => ONT => ISP Router (192.168.2.1) => WAN PFSENSE (192.168.1.4) => LAN PFSENSE (192.168.1.1)

    Your log seems to show unbound working fine. What does your quality of connection in pfsense show you?

    Here is the graph, Seems fine

    0_1551881795850_Quality graph.PNG

    query whatever name you called pfsense..
    simple nslookup or dig, or host or whatever your other fav dns query tool might be. My pfsense is called sg4860.local.lan

    Nslookup

    > sg4860.local.lan
    Server:  sg4860.local.lan
    Address:  192.168.9.253
    
    Name:    sg4860.local.lan
    Address:  192.168.9.253
    

    NsLookup

    nslookup pfsense.localdomain
    Serveur :   pfSense.localdomain
    Address:  192.168.1.1
    
    Nom :    pfsense.localdomain
    Address:  192.168.1.1
    


  • @jules13 said in Traffic blocked randomly:

    FTTH => ONT => ISP Router (192.168.2.1) => WAN PFSENSE (192.168.1.4) => LAN PFSENSE (192.168.1.1)

    Are you sure the pfSense WAN is 192.168.1.4? If yes, then that config is invalid as you can't have the same network on WAN and LAN.

    Edit: also it's normal that the WebUI is very slow and the dashboard takes a long time to load without a working connection to the Internet. Also what NICs are you using for your pfSense installation?



  • Sorry I did a mistake, the wan IP of PfSense is 192.168.2.4 and it is on the DMZ of the ISP router

    I'm using the intel Gigabit ethernet card of the MB (asus rog b360)
    And I have a dual NIC intel : Intel PRO/1000 PT Dual Port Server Adapter

    WAN is MB
    LAN is Intel card
    In pfsense these are virtiO cards because of proxmox VM.



  • I've created a graph of Wan Traffic and it seems there is a problem as I have a continuous use of internet and the graph look like this:
    0_1551885487913_WAN traffic.PNG


  • LAYER 8 Global Moderator

    well your quality graph means nothing for your actual internet - because all that shows you is you have connectivity to the ips router in front of pfsense.

    In pfsense these are virtiO cards because of proxmox VM.

    So pfsense is a VM.. Yeah you need to mention these things out of the gate!!

    Proxmox at one time had issues with pfsense - not sure if still the case?
    https://docs.netgate.com/pfsense/en/latest/virtualization/virtualizing-pfsense-with-proxmox.html

    But!!
    WARNING: because the hardware checksum offload is not yet disabled, accessing pfSense WebGUI might be sluggish. This is NORMAL and is fixed in the following step.

    To disable hardware checksum offload, navigate under System > Advanced and select Networking tab. Under Networking Interfaces section check the Disable hardware checksum offload and click save. Reboot will be required after this step.

    Did you do this??



  • Yes I disabled hardware checksum offloading, Sorry I was sure I have said that it was a proxmox VM, but it is not the case.

    Yes I followed this tutorial to install PfSense


Log in to reply