Diagnostics ARP Table 504 Nginx Gateway Timeout

jimp

@lddj9090 said in Diagnostics ARP Table 504 Nginx Gateway Timeout:

@jimp
What do you mean by "disable DNS for the page"?

It will stop the page from using DNS to resolve hostnames. DNS resolution is almost certainly what is slowing it down.

Instead you'll just get a list of the MAC addresses and IP addresses without the hostnames.

johnpoz

@lddj9090 said in Diagnostics ARP Table 504 Nginx Gateway Timeout:

figure out if pfsense has a problem resolving devices in its mac table?

By why which IP/mac combo's don't show a name.. And where that could/should resolve, etc.

example

Here is snip of mine. There are a few that don't resolve - but most of them do.

[21.05-RELEASE][admin@sg4860.local.lan]/root: arp -a
? (192.168.6.106) at 8a:75:de:b2:b7:bd on igb2.6 expires in 1166 seconds [vlan]
? (192.168.6.104) at 64:79:f0:d6:60:50 on igb2.6 expires in 1186 seconds [vlan]
? (192.168.6.105) at 38:80:df:f4:23:43 on igb2.6 expires in 1194 seconds [vlan]
sg4860.wguest.local.lan (192.168.6.253) at 00:08:a2:0c:e6:20 on igb2.6 permanent [vlan]
Alexa-Kitchen.local.lan (192.168.4.76) at a0:d0:dc:56:53:42 on igb2.4 expires in 1101 seconds [vlan]
Alexa-Bedroom.local.lan (192.168.4.77) at 40:a9:cf:c2:b0:99 on igb2.4 expires in 974 seconds [vlan]
alexa-show.local.lan (192.168.4.78) at b8:5f:98:0e:22:14 on igb2.4 expires in 653 seconds [vlan]
Alexa-cpu.local.lan (192.168.4.79) at b4:7c:9c:a8:d5:f0 on igb2.4 expires in 1198 seconds [vlan]
island2.local.lan (192.168.4.72) at 18:69:d8:39:e4:23 on igb2.4 expires in 944 seconds [vlan]
? (192.168.4.203) at 9a:4c:a7:b3:8f:31 on igb2.4 expires in 1000 seconds [vlan]
island1.local.lan (192.168.4.71) at 18:69:d8:3a:03:cd on igb2.4 expires in 1194 seconds [vlan]
nightstand.local.lan (192.168.4.65) at 68:57:2d:98:38:84 on igb2.4 expires in 981 seconds [vlan]
office1.local.lan (192.168.4.61) at 68:57:2d:98:39:14 on igb2.4 expires in 900 seconds [vlan]
sg4860.wpsk.local.lan (192.168.4.253) at 00:08:a2:0c:e6:20 on igb2.4 permanent [vlan]
office2.local.lan (192.168.4.62) at 68:57:2d:98:36:83 on igb2.4 expires in 1136 seconds [vlan]
office3.local.lan (192.168.4.63) at 68:57:2d:98:28:22 on igb2.4 expires in 925 seconds [vlan]
end-table.local.lan (192.168.4.56) at d4:a6:51:cf:c5:45 on igb2.4 expires in 928 seconds [vlan]
guestroom.local.lan (192.168.4.57) at 18:69:d8:39:f7:e9 on igb2.4 expires in 1001 seconds [vlan]
S30.local.lan (192.168.4.89) at 88:3f:4a:f0:cb:9c on igb2.4 expires in 1115 seconds [vlan]
hall1.local.lan (192.168.4.58) at d4:a6:51:d1:a2:8a on igb2.4 expires in 1055 seconds [vlan]
hall2.local.lan (192.168.4.59) at d4:a6:51:d2:0b:d4 on igb2.4 expires in 909 seconds [vlan]
D2.local.lan (192.168.4.52) at 18:69:d8:3a:08:d8 on igb2.4 expires in 931 seconds [vlan]
D3.local.lan (192.168.4.53) at 18:69:d8:39:d9:f4 on igb2.4 expires in 922 seconds [vlan]
D4.local.lan (192.168.4.54) at 18:69:d8:39:f2:ca on igb2.4 expires in 1198 seconds [vlan]
D5.local.lan (192.168.4.55) at 18:69:d8:39:ed:ba on igb2.4 expires in 1194 seconds [vlan]

I haven't gotten around to setting up some of them with reservations, and those do not resolve. I will fix that at some point, etc.

But if you had hundreds of these that do not resolve - and its trying then sure that could cause a delay in the output..

The simple solution is prob just do as shown and disable the dns part of arp table. See if I do with -n -a, none of mine resolve

[21.05-RELEASE][admin@sg4860.local.lan]/root: arp -n -a
? (192.168.6.106) at 8a:75:de:b2:b7:bd on igb2.6 expires in 1183 seconds [vlan]
? (192.168.6.104) at 64:79:f0:d6:60:50 on igb2.6 expires in 1169 seconds [vlan]
? (192.168.6.105) at 38:80:df:f4:23:43 on igb2.6 expires in 1140 seconds [vlan]
? (192.168.6.253) at 00:08:a2:0c:e6:20 on igb2.6 permanent [vlan]
? (192.168.4.76) at a0:d0:dc:56:53:42 on igb2.4 expires in 1049 seconds [vlan]
? (192.168.4.77) at 40:a9:cf:c2:b0:99 on igb2.4 expires in 1167 seconds [vlan]
? (192.168.4.78) at b8:5f:98:0e:22:14 on igb2.4 expires in 456 seconds [vlan]
? (192.168.4.79) at b4:7c:9c:a8:d5:f0 on igb2.4 expires in 1158 seconds [vlan]
? (192.168.4.72) at 18:69:d8:39:e4:23 on igb2.4 expires in 1048 seconds [vlan]
? (192.168.4.203) at 9a:4c:a7:b3:8f:31 on igb2.4 expires in 1006 seconds [vlan]
? (192.168.4.71) at 18:69:d8:3a:03:cd on igb2.4 expires in 997 seconds [vlan]
? (192.168.4.65) at 68:57:2d:98:38:84 on igb2.4 expires in 1084 seconds [vlan]
? (192.168.4.61) at 68:57:2d:98:39:14 on igb2.4 expires in 1004 seconds [vlan]
? (192.168.4.253) at 00:08:a2:0c:e6:20 on igb2.4 permanent [vlan]
? (192.168.4.62) at 68:57:2d:98:36:83 on igb2.4 expires in 939 seconds [vlan]
? (192.168.4.63) at 68:57:2d:98:28:22 on igb2.4 expires in 1028 seconds [vlan]
? (192.168.4.56) at d4:a6:51:cf:c5:45 on igb2.4 expires in 1032 seconds [vlan]
? (192.168.4.57) at 18:69:d8:39:f7:e9 on igb2.4 expires in 1106 seconds [vlan]
? (192.168.4.89) at 88:3f:4a:f0:cb:9c on igb2.4 expires in 918 seconds [vlan]
? (192.168.4.58) at d4:a6:51:d1:a2:8a on igb2.4 expires in 858 seconds [vlan]
? (192.168.4.59) at d4:a6:51:d2:0b:d4 on igb2.4 expires in 1013 seconds [vlan]
? (192.168.4.52) at 18:69:d8:3a:08:d8 on igb2.4 expires in 1035 seconds [vlan]
? (192.168.4.53) at 18:69:d8:39:d9:f4 on igb2.4 expires in 1026 seconds [vlan]
? (192.168.4.54) at 18:69:d8:39:f2:ca on igb2.4 expires in 1001 seconds [vlan]
? (192.168.4.55) at 18:69:d8:39:ed:ba on igb2.4 expires in 997 seconds [vlan]

lddj9090

@jimp

After deleting some DNS Override entries, the GUI ARP table shows up without any problems. Seem to have fixed the issue!

Thanks a lot for the outstanding support.

johnpoz

Well your PTR zone there - yeah if you were asking some other NS to resolve the ptrs - and it didn't know them or you couldn't actually talk to that NS then yeah you could have issues when your trying to resolve hundreds of IPs to their names..

lddj9090

@johnpoz
Exactly, I think not beeing able to resolve the IP's caused the problem in the end. Thanks for your help mate!

lddj9090

Alright, to clarify for other users who may have problems with this:

Configuration under DNS Resolver -> General Settings -> Domain Overrides looked like:

Domain:
18.16.172-in.addr.arpa (our VLAN)

Lookup Server IP Address:
172.16.18.1 (Pfsense interface for this VLAN)

The reason I configured it like this was, I wanted that clients from that VLAN do not use an outside lookup server. Instead, using our Pfsense itself.

After deleting this entry, the ARP table shows up nicely.

johnpoz

What did you think that would do?

I wanted that clients from that VLAN do not use an outside lookup server. Instead

That makes no sense.. How would that of redirected say a client wanting to use 8.8.8.8 for dns??

You wanted a dns redirect..

https://docs.netgate.com/pfsense/en/latest/recipes/dns-redirect.html

You were telling pfsense, hey if you want to look up what this IP is - ask yourself? Well duh ;) How would that redirect a client to use pfsense vs googledns? Could create a loop I would think? But I think your issue is different than the other poster.. who had public IP in his arp table.

lddj9090

@johnpoz

When I tried lets say from VLAN 14 to nslookup a client in the 18 VLAN, I wanted pfsense to handle this as it is DNS/DHCP for the 18 VLAN.
So I thought I had to make this entry in the config. After deleting it, I was still able to resolve IP's in the 18 VLAN.

But I am not sure how domain override entries affect loading ARP entries. Have you got an idea?

Test wise I added the domain override entry again, and again ARP wouldn't load up.

johnpoz

My guess to why that problem would cause you an issue with arp, is it was causing some sort of loop/issue with actually doing the PTR request.. So it was causing timeouts when the arp table tried to resolve your local IPs to names.. Which would cause delay in the arp table being displayed.

The other users issue was related to trying to resolve PTR for large number of public IPs.. Which may or may not of resolved causing the delay.

Normally doing PTRs for local IPs should be pretty instant, since it would be just pfsense asking its local unbound for them. Or some other local NS authoritative for that IP range.. But yeah if there is issue talking to whatever NS it would resolve these IPs, or there is a LOT of them that are not local and have to query some external NS, that could cause delay in the table to be displayed.

lddj9090

This post is deleted!

lddj9090

@johnpoz
Your explanation sounds logical and makes sense.

How to investigate this any further? As pfsense itself is DNS for the VLAN 18 interface and therefore should be able to resolve those IPs.

johnpoz

I would pick an IP form your network - does it resolve? If not - why?

Pfsense is only going to be able to resolve stuff it has record for. Be it via dhcp registration, static registration, or you actually created a record.

Pretty much all my ips resolve because most of my network is reserved, and I register the static registrations in dns. But there are a few that do not..

In the past, and maybe even still there are issues "depending" on registration of dhcp leases into dns because of restarts of unbound..

even if the records do not exist.. if the NS is local - it should come back almost instant... For example

$ dig -x 192.168.9.44 @192.168.9.253

; <<>> DiG 9.16.18 <<>> -x 192.168.9.44 @192.168.9.253
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 45780
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;44.9.168.192.in-addr.arpa.     IN      PTR

;; AUTHORITY SECTION:
168.192.in-addr.arpa.   10800   IN      SOA     localhost. nobody.invalid. 1 3600 1200 604800 10800

;; Query time: 0 msec
;; SERVER: 192.168.9.253#53(192.168.9.253)
;; WHEN: Wed Jul 14 09:48:16 Central Daylight Time 2021
;; MSG SIZE  rcvd: 113

Here I did a PTR lookup for something I know there is no record for it. NX is sent pretty much instant.

lddj9090

@johnpoz
There are some ips which do not resolve as they are out of dhcp range and not static or the device doesn't tell a hostname. But almost all ips resolve proper.

If domain override exists with the lookup server pointing to itself, it causes the problem. Guess pfsense gets into a loop "ask itself many times" for the entries it can't resolve.