Traffic blocked randomly

Jules13

Hi,
I have an issue with PfSense:
When I start a connection, everything is fine and then for mobile phone for example, when I stop using it and come back 5 minutes later, I ve got error timeout.
I can't access to local network too
It is the same on my computer.
But the DHCP leases are still active because I let the DHCP leases last 2 hours (default time)

When I get these error, I'm trying to ping IP Adresses and domain names such as 8.8.8.8 1.1.1.1 google.com and it fails.
Seems that the firewall is blocking traffic randomly
But I've created a pass firewall rules that let everything pass but it still doesn't work
Thank you for your help
Jules

johnpoz

@jules13 said in Traffic blocked randomly:

Seems that the firewall is blocking traffic randomly

Yeah doesn't work that way ;)

Out of the box if pfsense blocked traffic it would LOG it..

If you can not even resolve google.com that points to connectivity issue either to pfsense itself from your client or for pfsense to internet, or maybe unbound is failing..

pinging 8.8.8.8 removes unbound failing as a problem - but still you do not know if you can not reach pfsense or pfsense can not reach 8.8.8.8

Can you ping pfsense IP when this happens? Access pfsense gui? Does the log or the gui show that your wan is offline or taking packet loss? Check the log did unbound recently restart? Or fail, etc. etc.

Jules13

When It happens mostly on mobile devices such as IOS devices and sometimes on my laptop:
ping 8.8.8.8 => Error
ping google.com => Error
ping 192.168.1.1 => Success
Pfsense GUI => Error Network protocol Violation (firefox)
Access NAS (local network) => Error

Then I disconnect of my wifi N and reconnect to my wifi AC (WiFi access point connected to pfsense) or I renew DHCP Lease on my device and it works again few minutes
The error appear when I stop using internet (device standby mode) and I reopen it.

I stopped DNS resolver services and I use :
1.1.1.1
1.0.0.1
9.9.9.9
208.67.222.222
208.67.220.220
DNS server.

Where can I access WAN logs (system logs? or dignostic)

Thank you for your help
Jules

Gertjan

@jules13 said in Traffic blocked randomly:

I stopped DNS resolver services and I use :
1.1.1.1
1.0.0.1
9.9.9.9
208.67.222.222
208.67.220.220

And when you remove all these IP's, and go back to the initial situation, the one that you found when you installed pfSEnse what will happen ?
I'll give you the answer : no more issues. I'll leave it up to you to draw your conclusions.

You said you stopped the DNS Resolver. Then who does the DNS for pfSense itself ??

@jules13 said in Traffic blocked randomly:

Where can I access WAN logs (system logs? or dignostic)

Except for some very typical situations (NAT, etc), no connections can be initiated from WAN. No need to log incoming connections on this interface yjat are blocked anyway. Not related to DNS.

Jules13

@gertjan said in Traffic blocked randomly:

And when you remove all these IP's, and go back to the initial situation, the one that you found when you installed pfSEnse what will happen ?
I'll give you the answer : no more issues. I'll leave it up to you to draw your conclusions.

You said you stopped the DNS Resolver. Then who does the DNS for pfSense itself ??

I did a factory reset, and I still have no internet randomly.
On my laptop I had timeout error with all website (including gui) and I couldn't ping 8.8.8.8 and google.com
google.com => couldn't solve hostname
8.8.8.8 => Timeout no answers.

Thank you for your help
Jules

johnpoz

@jules13 said in Traffic blocked randomly:

I stopped DNS resolver services and I use :

So you have your clients directly access those name servers?

You can for sure do that if you so desire.. But let me point out a flaw in your selection. Quad 9 filters, while 1.1.1.1 does not - opendns filters as well - but could be different than what quad9 filters.

So now you run into a conundrum, you have non filtering dns, and filtering dns that filter differently... If you point your client to all of those you have no idea which one your client will actually query or get an answer from. So sites might be blocked, they might not be blocked.. This is a borked configuration out of the gate. If your going to point to external dns - then pick 1 company!! So you are sure your always going to get either unfiltered for filtered the same way from any of their dns you might query.

When you say you try and access NAS - are you using an IP, how wold 1.1.1.1 resolve the IP of your local NAS? if your asking them for example.

As to network protocol error when you try and access the gui? Yeah going to need more info - is that some error your browser issues.. Does it also give you the same error in say chrome?

You will find all the logs - in such a crazy place ;)
Status / System Logs

As to factory reset and can't resolve google.. Can you get to the pfsense gui, can you ping pfsense IP? Is unbound even running? Does your isp block dns queries and only allows you to use their dns? What does the unbound log say - up the level of logging if need be in unbound, etc.

Can you resolve pfsense own name from unbound? What does pfsense diag / dns lookup show when you ask it to resolve www.google.com ?

Jules13

@johnpoz said in Traffic blocked randomly:

You can for sure do that if you so desire.. But let me point out a flaw in your selection. Quad 9 filters, while 1.1.1.1 does not - opendns filters as well - but could be different than what quad9 filters.

So now you run into a conundrum, you have non filtering dns, and filtering dns that filter differently... If you point your client to all of those you have no idea which one your client will actually query or get an answer from. So sites might be blocked, they might not be blocked.. This is a borked configuration out of the gate. If your going to point to external dns - then pick 1 company!! So you are sure your always going to get either unfiltered for filtered the same way from any of their dns you might query.

As I realised with @Gertjan. I did an error, therefore I factory reset the PfSense router and the issue is still there

As to network protocol error when you try and access the gui? Yeah going to need more info - is that some error your browser issues.. Does it also give you the same error in say chrome?

It is a firefox issue, i'm trying to see what happen with chrome:

You will find all the logs - in such a crazy place ;)
Status / System Logs

Thank you

As to factory reset and can't resolve google..
Can you get to the pfsense gui,

It's the same as before: It can be working 10 minutes and then I have timeout errors

Can you ping pfsense IP?

Yes

Is unbound even running?

Seems so : in the log I've got :

Mar 6 13:12:10 	unbound 	35451:0 	info: start of service (unbound 1.8.1).

Does your isp block dns queries and only allows you to use their dns?

No, because I used opendns DNS with linksys router in the past

What does the unbound log say - up the level of logging if need be in unbound, etc.

Mar 6 13:39:43 	unbound 	44190:0 	info: reply from <co.> 156.154.100.25#53
Mar 6 13:39:43 	unbound 	44190:0 	info: query response was REFERRAL
Mar 6 13:39:43 	unbound 	44190:0 	info: processQueryTargets: ns2.radioline.co. A IN
Mar 6 13:39:43 	unbound 	44190:0 	info: skipping target due to dependency cycle (harden-glue: no may fix some of the cycles) ns2.radioline.co. A IN
Mar 6 13:39:43 	unbound 	44190:0 	info: new target ns1.radioline.fr. AAAA IN
Mar 6 13:39:43 	unbound 	44190:0 	info: sending query: ns2.radioline.co. A IN
Mar 6 13:39:43 	unbound 	44190:0 	debug: sending to target: <radioline.co.> 195.210.43.139#53
Mar 6 13:39:43 	unbound 	44190:0 	debug: iterator[module 1] operate: extstate:module_state_initial event:module_event_pass
Mar 6 13:39:43 	unbound 	44190:0 	info: iterator operate: query ns1.radioline.fr. AAAA IN
Mar 6 13:39:43 	unbound 	44190:0 	info: resolving ns1.radioline.fr. AAAA IN
Mar 6 13:39:43 	unbound 	44190:0 	info: finishing processing for ns1.radioline.fr. AAAA IN
Mar 6 13:39:43 	unbound 	44190:0 	debug: validator[module 0] operate: extstate:module_state_initial event:module_event_moddone
Mar 6 13:39:43 	unbound 	44190:0 	info: validator operate: query ns1.radioline.fr. AAAA IN
Mar 6 13:39:43 	unbound 	44190:0 	debug: iterator[module 1] operate: extstate:module_wait_reply event:module_event_pass
Mar 6 13:39:43 	unbound 	44190:0 	info: iterator operate: query ns2.radioline.co. A IN
Mar 6 13:39:43 	unbound 	44190:0 	info: processQueryTargets: ns2.radioline.co. A IN
Mar 6 13:39:43 	unbound 	44190:0 	debug: cache memory msg=68811 rrset=134872 infra=66387 val=44275
Mar 6 13:39:43 	unbound 	44190:0 	debug: iterator[module 1] operate: extstate:module_wait_reply event:module_event_reply
Mar 6 13:39:43 	unbound 	44190:0 	info: iterator operate: query fr. DNSKEY IN
Mar 6 13:39:43 	unbound 	44190:0 	info: response for fr. DNSKEY IN
Mar 6 13:39:43 	unbound 	44190:0 	info: reply from <fr.> 193.176.144.22#53
Mar 6 13:39:43 	unbound 	44190:0 	info: query response was ANSWER
Mar 6 13:39:43 	unbound 	44190:0 	info: processQueryTargets: fr. DNSKEY IN
Mar 6 13:39:43 	unbound 	44190:0 	info: sending query: fr. DNSKEY IN
Mar 6 13:39:43 	unbound 	44190:0 	debug: sending to target: <fr.> 194.0.9.1#53
Mar 6 13:39:43 	unbound 	44190:0 	debug: cache memory msg=69007 rrset=135938 infra=66639 val=44275
Mar 6 13:39:43 	unbound 	44190:0 	debug: iterator[module 1] operate: extstate:module_wait_reply event:module_event_reply
Mar 6 13:39:43 	unbound 	44190:0 	info: iterator operate: query fr. DNSKEY IN
Mar 6 13:39:43 	unbound 	44190:0 	info: response for fr. DNSKEY IN
Mar 6 13:39:43 	unbound 	44190:0 	info: reply from <fr.> 194.0.9.1#53
Mar 6 13:39:43 	unbound 	44190:0 	info: query response was ANSWER
Mar 6 13:39:43 	unbound 	44190:0 	info: finishing processing for fr. DNSKEY IN
Mar 6 13:39:43 	unbound 	44190:0 	debug: validator[module 0] operate: extstate:module_wait_module event:module_event_moddone
Mar 6 13:39:43 	unbound 	44190:0 	info: validator operate: query fr. DNSKEY IN
Mar 6 13:39:43 	unbound 	44190:0 	info: validated DNSKEY fr. DNSKEY IN
Mar 6 13:39:43 	unbound 	44190:0 	debug: validator[module 0] operate: extstate:module_wait_subquery event:module_event_pass
Mar 6 13:39:43 	unbound 	44190:0 	info: validator operate: query cache.radioline.fr. A IN
Mar 6 13:39:43 	unbound 	44190:0 	info: NSEC3s for the referral proved no DS.
Mar 6 13:39:43 	unbound 	44190:0 	info: Verified that unsigned response is INSECURE
Mar 6 13:39:43 	unbound 	44190:0 	debug: cache memory msg=69007 rrset=135938 infra=66639 val=45507
Mar 6 13:39:43 	unbound 	44190:0 	debug: iterator[module 1] operate: extstate:module_wait_reply event:module_event_reply
Mar 6 13:39:43 	unbound 	44190:0 	info: iterator operate: query ns2.radioline.co. A IN
Mar 6 13:39:43 	unbound 	44190:0 	info: sanitize: removing potential poison RRset: ns1.radioline.fr. A IN
Mar 6 13:39:43 	unbound 	44190:0 	info: response for ns2.radioline.co. A IN
Mar 6 13:39:43 	unbound 	44190:0 	info: reply from <radioline.co.> 195.210.43.139#53
Mar 6 13:39:43 	unbound 	44190:0 	info: query response was ANSWER
Mar 6 13:39:43 	unbound 	44190:0 	info: finishing processing for ns2.radioline.co. A IN
Mar 6 13:39:43 	unbound 	44190:0 	debug: validator[module 0] operate: extstate:module_state_initial event:module_event_moddone
Mar 6 13:39:43 	unbound 	44190:0 	info: validator operate: query ns2.radioline.co. A IN
Mar 6 13:39:43 	unbound 	44190:0 	debug: cache memory msg=69007 rrset=135938 infra=66639 val=45507

Can you resolve pfsense own name from unbound?

How can I do this ?

What does pfsense diag / dns lookup show when you ask it to resolve www.google.com ?

I've got this:

Name server 	Query time
127.0.0.1	0 msec
192.168.2.1	12 msec

But I can acces GUI only when I have internet, because when I have the error I can"t access GUI

johnpoz

@jules13 said in Traffic blocked randomly:

192.168.2.1 12 msec

Who is this? Is this upstream nat router in front of pfsense?

Your log seems to show unbound working fine. What does your quality of connection in pfsense show you?

query whatever name you called pfsense..
simple nslookup or dig, or host or whatever your other fav dns query tool might be. My pfsense is called sg4860.local.lan

$ dig sg4860.local.lan

; <<>> DiG 9.12.3-P1 <<>> sg4860.local.lan
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5577
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;sg4860.local.lan.              IN      A

;; ANSWER SECTION:
sg4860.local.lan.       3600    IN      A       192.168.9.253

;; Query time: 0 msec
;; SERVER: 192.168.9.253#53(192.168.9.253)
;; WHEN: Wed Mar 06 08:05:41 Central Standard Time 2019
;; MSG SIZE  rcvd: 61

Nslookup

> sg4860.local.lan
Server:  sg4860.local.lan
Address:  192.168.9.253

Name:    sg4860.local.lan
Address:  192.168.9.253

But seems more like you have a local connectivity issue... Does this happen on client when your wired to pfsense vs wifi? Lets see pfsense quality graph

This will show you if pfsense wan connection is going offline or having lots of packet loss, etc.

Jules13

@johnpoz said in Traffic blocked randomly:

@jules13 said in Traffic blocked randomly:

192.168.2.1 12 msec

Who is this? Is this upstream nat router in front of pfsense?

Yeah I must have a double NAT configuration because ISP in France don't allows you to dc the box and there is no bridge mode.
So I've got this:

FTTH => ONT => ISP Router (192.168.2.1) => WAN PFSENSE (192.168.1.4) => LAN PFSENSE (192.168.1.1)

Your log seems to show unbound working fine. What does your quality of connection in pfsense show you?

Here is the graph, Seems fine

0_1551881795850_Quality graph.PNG

query whatever name you called pfsense..
simple nslookup or dig, or host or whatever your other fav dns query tool might be. My pfsense is called sg4860.local.lan

Nslookup

> sg4860.local.lan
Server:  sg4860.local.lan
Address:  192.168.9.253

Name:    sg4860.local.lan
Address:  192.168.9.253

NsLookup

nslookup pfsense.localdomain
Serveur :   pfSense.localdomain
Address:  192.168.1.1

Nom :    pfsense.localdomain
Address:  192.168.1.1

Grimson

@jules13 said in Traffic blocked randomly:

FTTH => ONT => ISP Router (192.168.2.1) => WAN PFSENSE (192.168.1.4) => LAN PFSENSE (192.168.1.1)

Are you sure the pfSense WAN is 192.168.1.4? If yes, then that config is invalid as you can't have the same network on WAN and LAN.

Edit: also it's normal that the WebUI is very slow and the dashboard takes a long time to load without a working connection to the Internet. Also what NICs are you using for your pfSense installation?

Jules13

Sorry I did a mistake, the wan IP of PfSense is 192.168.2.4 and it is on the DMZ of the ISP router

I'm using the intel Gigabit ethernet card of the MB (asus rog b360)
And I have a dual NIC intel : Intel PRO/1000 PT Dual Port Server Adapter

WAN is MB
LAN is Intel card
In pfsense these are virtiO cards because of proxmox VM.

Jules13

I've created a graph of Wan Traffic and it seems there is a problem as I have a continuous use of internet and the graph look like this:
0_1551885487913_WAN traffic.PNG

johnpoz

well your quality graph means nothing for your actual internet - because all that shows you is you have connectivity to the ips router in front of pfsense.

In pfsense these are virtiO cards because of proxmox VM.

So pfsense is a VM.. Yeah you need to mention these things out of the gate!!

Proxmox at one time had issues with pfsense - not sure if still the case?
https://docs.netgate.com/pfsense/en/latest/virtualization/virtualizing-pfsense-with-proxmox.html

But!!
WARNING: because the hardware checksum offload is not yet disabled, accessing pfSense WebGUI might be sluggish. This is NORMAL and is fixed in the following step.

To disable hardware checksum offload, navigate under System > Advanced and select Networking tab. Under Networking Interfaces section check the Disable hardware checksum offload and click save. Reboot will be required after this step.

Did you do this??

Jules13

Yes I disabled hardware checksum offloading, Sorry I was sure I have said that it was a proxmox VM, but it is not the case.

Yes I followed this tutorial to install PfSense