Access Point Stops responding to ICMP commands at night
-
Its happening every night at two different properties running both Pfsense.
The Access Points Stops responding to Ping requests. Things starts becoming normal at day time when very few people are using Wi-Fi.
Scenario. Router Pfsense
Switches : Unmanged Dlinks (around 7 different switches, a mix of gigabit and non gigabyte switches.Access Points: Dlink DAP 2553 around 7 One Cisco 1200 and One proxim 700
at one property it just started two days ago otherwise all APs were up for long time .
Note sure what causing APs not to reply or might be a broadcasting storm . any thoughts ? -
to add more. I went to the property and found a funny situation. I was able to login to the Wireless Aps at the spot but NAT is not working for some of the access points.
After some times NAT or internal access started working for some of the Aps but not for others so its just keep moving from one Ap to another.Not sure if some thing is bad or problem with pfsense. this pfsense is a dual WAN load balancer.
-
Some cheap access points have a tendency to flake out for off-subnet requests, ignoring the fact they have a default gateway configured or just not using it. Based on your description, that's my guess. Why it's not consistent, hard to say. Packet capture on the firewall would confirm or deny that.
Could potentially be some other general network problem, which a packet capture would also show.
-
thanks CMB
very annoying issue. I am able to ping the AP locally but cannot access the web interface and after few minutes I can. This problem is driving me crazy as nothing changed in the network. All switches and Aps are turned off and turned back on just to make sure no power cycle issue.
To capture packet should I be using WAN interface or LAN? I guess LAN as AP is on the LAN interface. Also what exactly I will be looking in the the packet capture logs.
So I selected LAN interface started capture packet and tried to login to one AP IP 192.168.10.50 which I was not able to access via Web interface and results are below here.00:30:18.154650 d4:be:d9:d9:4f:bb > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.50 tell 192.168.10.205, length 46
00:30:18.440592 3c:74:37:a2:bc:98 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.205 (ff:ff:ff:ff:ff:ff) tell 192.168.10.50, length 46
00:30:33.776836 3c:74:37:a2:bc:98 > 00:19:b9:07:6a:e9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 128, id 37583, offset 0, flags [none], proto ICMP (1), length 29)
192.168.10.50 > 192.168.10.1: ICMP echo request, id 4861, seq 19, length 9
00:30:33.776889 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 43: (tos 0x0, ttl 64, id 39061, offset 0, flags [none], proto ICMP (1), length 29, bad cksum 0 (->4cc7)!)
192.168.10.1 > 192.168.10.50: ICMP echo reply, id 4861, seq 19, length 9
00:30:33.870988 3c:74:37:a2:bc:98 > 00:19:b9:07:6a:e9, ethertype IPv4 (0x0800), length 76: (tos 0x0, ttl 128, id 37584, offset 0, flags [none], proto UDP (17), length 62)
192.168.10.50.53892 > 192.168.10.1.53: [udp sum ok] 52606+ A? api.facebook.com. (34)
00:30:34.080514 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 116: (tos 0x0, ttl 64, id 52616, offset 0, flags [none], proto UDP (17), length 102, bad cksum 0 (->177b)!)
192.168.10.1.53 > 192.168.10.50.53892: [udp sum ok] 52606 q: A? api.facebook.com. 2/0/0 api.facebook.com. CNAME star.c10r.facebook.com., star.c10r.facebook.com. A 31.13.77.55 (74)
00:30:34.307721 3c:74:37:a2:bc:98 > 00:19:b9:07:6a:e9, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 37585, offset 0, flags [none], proto TCP (6), length 48)
192.168.10.50.49836 > 31.13.77.55.80: Flags, cksum 0xb01e (correct), seq 3054773045, win 65535, options [mss 1360,nop,nop,sackOK], length 0
00:30:34.466699 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 84, id 0, offset 0, flags [DF], proto TCP (6), length 48, bad cksum 0 (->efa9)!)
31.13.77.55.80 > 192.168.10.50.49836: Flags [S.], cksum 0x24ea (correct), seq 862789194, ack 3054773046, win 14600, options [mss 1460,nop,nop,sackOK], length 0
00:30:34.495030 3c:74:37:a2:bc:98 > 00:19:b9:07:6a:e9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 128, id 37586, offset 0, flags [none], proto TCP (6), length 40)
192.168.10.50.49836 > 31.13.77.55.80: Flags [.], cksum 0x8ab6 (correct), seq 1, ack 1, win 65535, length 0
00:30:34.508115 3c:74:37:a2:bc:98 > 00:19:b9:07:6a:e9, ethertype IPv4 (0x0800), length 1414: (tos 0x0, ttl 128, id 37587, offset 0, flags [none], proto TCP (6), length 1400)
192.168.10.50.49836 > 31.13.77.55.80: Flags [P.], cksum 0x9b74 (correct), seq 1:1361, ack 1, win 65535, length 1360
00:30:34.510156 3c:74:37:a2:bc:98 > 00:19:b9:07:6a:e9, ethertype IPv4 (0x0800), length 1218: (tos 0x0, ttl 128, id 37588, offset 0, flags [none], proto TCP (6), length 1204)
192.168.10.50.49836 > 31.13.77.55.80: Flags [P.], cksum 0x4211 (correct), seq 1361:2525, ack 1, win 65535, length 1164
00:30:34.616726 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 84, id 5428, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->da7d)!)
31.13.77.55.80 > 192.168.10.50.49836: Flags [.], cksum 0x45a6 (correct), seq 1, ack 1361, win 16320, length 0
00:30:34.616741 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 84, id 5429, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->da7c)!)
31.13.77.55.80 > 192.168.10.50.49836: Flags [.], cksum 0x367a (correct), seq 1, ack 2525, win 19040, length 0
00:30:34.935232 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 1414: (tos 0x0, ttl 84, id 5430, offset 0, flags [DF], proto TCP (6), length 1400, bad cksum 0 (->d52b)!)
31.13.77.55.80 > 192.168.10.50.49836: Flags [.], cksum 0xc437 (correct), seq 1:1361, ack 2525, win 19040, length 1360
00:30:34.935249 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 189: (tos 0x0, ttl 84, id 5431, offset 0, flags [DF], proto TCP (6), length 175, bad cksum 0 (->d9f3)!)
31.13.77.55.80 > 192.168.10.50.49836: Flags [P.], cksum 0xed18 (correct), seq 1361:1496, ack 2525, win 19040, length 135
00:30:34.935265 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 938: (tos 0x0, ttl 84, id 5432, offset 0, flags [DF], proto TCP (6), length 924, bad cksum 0 (->d705)!)
31.13.77.55.80 > 192.168.10.50.49836: Flags [P.], cksum 0x776f (correct), seq 1496:2380, ack 2525, win 19040, length 884
00:30:34.935279 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 84, id 5433, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->da78)!)
31.13.77.55.80 > 192.168.10.50.49836: Flags [F.], cksum 0x2d2e (correct), seq 2380, ack 2525, win 19040, length 0
00:30:35.115443 3c:74:37:a2:bc:98 > 00:19:b9:07:6a:e9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 128, id 37589, offset 0, flags [none], proto TCP (6), length 40)
192.168.10.50.49836 > 31.13.77.55.80: Flags [.], cksum 0x80d9 (correct), seq 2525, ack 2381, win 63156, length 0
00:30:35.167484 3c:74:37:a2:bc:98 > 00:19:b9:07:6a:e9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 128, id 37590, offset 0, flags [none], proto TCP (6), length 40)
192.168.10.50.49836 > 31.13.77.55.80: Flags [F.], cksum 0x80d8 (correct), seq 2525, ack 2381, win 63156, length 0
00:30:35.263936 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 84, id 0, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->efb1)!)
31.13.77.55.80 > 192.168.10.50.49836: Flags [.], cksum 0x2d2d (correct), seq 2381, ack 2526, win 19040, length 0
00:30:44.870134 00:19:b9:07:6a:e9 > 3c:74:37:a2:bc:98, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 243, id 29096, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->df08)!)
31.13.77.55.80 > 192.168.10.50.49836: Flags [R.], cksum 0x80d5 (correct), seq 1, ack 2526, win 0, length 0 -
I'm guessing for pings, you can ping from LAN and not from WAN, via Diag>Ping. That would confirm the AP isn't routing correctly. That may be because of the APs being buggy, or it could be an IP conflict on the gateway IP, which would cause them to respond to the wrong device.
Starting a capture on LAN, filtering on one of the AP's IPs, and trying to access its management interface when it isn't working, will show what's happening at least from the firewall's perspective.
Check your system log for IP conflict logs too, though I'd expect you would have much bigger problems than just the APs occasionally inaccessible if you had a conflict on the gateway IP.
-
"The Access Points Stops responding to Ping requests."
So how does that have anything to do with pfsense?? If you have device that does not respond to ping from a device on the same segment - that has nothing to do with pfsense.
Now if you were saying it answers ping from devices on segment, but devices on another segment can not ping it. IE traffic was going through pfsense - ok then.
But in your case it seems you have a flaky AP, I don't see how pfsense or nat comes into play at all.
-
John you did not read the whole story. Apparantly it seems like it has nothing to do with Pfsense but it has definitely some thing. If it is a buggy AP then it should happen with just one AP not all of them. There are not conflicts in Gateway either, I guess Pfsense is not routing the request to the right AP when the traffic volume is high.
Every thing worked fine with the old nomadix router-
Any how I will try to further investigate.
-
I read the whole post, how is pfsense not routing anything when your pinging from the same segment??
Pfsense does not come into play if your on the same segment. Where did you mention any other segments? And how does Nat have anything to do with a ping from devices on the same segment?
What is this suppose to mean?
" but NAT is not working for some of the access points. "More than happy to help you figure out your issue - but from the info given I don't see how pfsense works into the equation other than its in the same building on the network the AP are on??
Could you lay out a basic diagram of your network, are there other segments at play? Other than this 192.168.10 ?? What is the mask on that network? is it /24 or does pfsense have more than 1 lan interface? With say 192.168.10.1/27 so that 192.168.10.50 would be on a different segment/vlan?
What are you natting? An AP does not NAT - if the wireless devices are true AP or wireless routers being used as AP, then they don't NAT anything.
Your thread might be better suited to general section, since it seems your just trying to work out a general networking issue, and not anything to do with pfsense that I can see.
-
Unless you're running out of states, there is 0 difference on where things get routed dependent on traffic volume. If you're out of states, things won't get routed as it'll stop accepting new connections. The fact it's intermittent rules out general config issues. It'd never work if the config were wrong. So that leaves general network issues. I still think a conflicting IP on the gateway IP seems like a likely culprit based on your description. Maybe a device you control like still having the old firewall up, maybe someone is plugging in some misconfigured device with a static 192.168.1.1 IP on it (if you're using the default subnet) which you'd likely see in the system log, amongst a range of other possibilities.
-
I figured out many days ago but just posting it here if some one faces the same situation.
pfsense was assigning APs addresses to the clients, so the APs were going down and I had IP conflicts, I separated the DHCP part from APs IPs and by using subnetting increased the useable IPs.
-
So you mean on pfsense you had a scope of say 192.168.1.100-200 and the APs had static IPs of say 192.168.1.150?
So some dhcp client would come on get an IP of 192.168.1.150 from pfsense?
So this could cause issue with pinging the AP ip or accessing its gui interface over http. But it should of had little to do with other clients connecting to the network in general or even using the AP. Unless these AP were not actual AP and were say natting, Some other client should of been able to use the wireless or wired network just fine
The only point of the AP ip in actual AP use would be to access the AP directly for config, it has nothing to do with connectivity in general to the network.
When you say AP, that normally means a device that bridges traffic from wired network to the wireless. Its IP is not even used in this conversation between a wireless client and wired network.
