Clients can't connect, DHCP log showing 'unknown subnet wrong network'

[ttblum]

Hello,

I recently swapped in a router running 2.2.2 at a site.

Ever since then, all laptops that arrive here after being used at a different site cannot get DHCP leases, and cannot connect at all.  Only after I type 'ipconfig /release' and 'ipconfig /renew' do they get a DHCP address.

The other site, let's call Site B, has subnet xxx.yyy.zzz.0/24.  This site, Site A, has subnet xxx.yyy.jjj.0/24.  The laptops are able to connect fine when they go back to Site B.

I am logging this at Site A:

Jul 2 08:32:39 dhcpd: DHCPREQUEST for xxx.yyy.zzz.8 from mm:mm:mm:mm:mm:mm via re2: wrong network.
Jul 2 08:31:21 dhcpd: DHCPINFORM from xxx.yyy.zzz.8 via re2: unknown subnet for client address xxx.yyy.zzz.8

Has anyone ever seen anything like this?

[doktornotor]

What retarded OS that is on the laptops?

[ttblum]

The laptops are running Windows 7.  This is the only site where it's happening.

[doktornotor]

@ttblum:

This is the only site where it's happening.

Do you have some insane lease time set there? Other than that, go to sniff the packets.

[ttblum]

I don't have a lease time set, I'm assuming it's the default 7200.

Do I need an explicit rule to allow dhcp traffic to the lan interface?  I didn't have this.

Is there some way to reproduce this without physically taking a device somewhere else and coming back?

[doktornotor]

@ttblum:

Do I need an explicit rule to allow dhcp traffic to the lan interface?  I didn't have this.

No, unless you relay DHCP somewhere else. Then again, there are firewall logs, so look there to see if anything (such as broadcasts) gets blocked!

As for ways to reproduce Windows stupidities, there are better venues, such as https://social.technet.microsoft.com/Forums/

[johnpoz]

If you run dhcp server on an interface, pfsense will auto create rules to allow dhcp to get to the server.  These rules will not be shown in the gui, but will be above all rules you put in.. So there really is no way for the user to block themselves from getting dhcp lease if they are running dhcp server on that interface.

Are these laptops wireless by chance?  Are they using the same SSID in both of these sites?

Any client that tries to renew a lease and it does not get an answer should auto drop the lease and ask for new one via discover..  You really should not have to release and renew manually.

Is it possible windows gots something borked up in their interpretation of the dhcp protocol - sure it is..

I would do a sniff on your interface your dhcp server is running on, diag packet capture and use port 67 or 68 as your filter so you only get dhcp traffic.  Then capture exactly what is happening is the client sending his old IP in option 50 in the discover packet?  Is he never sending a discover packet?

Very helpful if you could post this capture to look at.

I can try and duplicate your problem when I get home on my network..  And will post up a sniff with either your duplicated problem or how it should work.  Simple enough to duplicate your setup with just fire up old wireless router off the shelf at home use a different network that my current wireless network is using and connect to one and then other other.  But like to know if your using the same SSID in both locations.

[doktornotor]

Been duplicated by many people before, apparently… a random thread: https://windowsforum.com/threads/laptop-must-use-ipconfig-release-and-renew-to-change-networks.112633/

[ttblum]

The DHCP server at 'Site B' is configured with lease time of 1468800 .

Would this length cause any problems?

[doktornotor]

You mean 17 days?  :o Very possibly (as also noted on the thread linked above.) The clients will basically will not even attempt to renew until half of the lease time elapsed.

[johnpoz]

Here is the thing.. If client requests an address that the server does not know about - it should send a dhcpnak telling the client not to use that address.  And then a dhcpdiscover would be sent and the dhcp server can issue a address in its scope.

http://linux.die.net/man/5/dhcpd.conf
If the server finds the address the client is requesting, and that address is available to the client, the server will send a DHCPACK. If the address is no longer available, or the client isn't permitted to have it, the server will send a DHCPNAK. If the server knows nothing about the address, it will remain silent, unless the address is incorrect for the network segment to which the client has been attached and the server is authoritative for that network segment, in which case the server will send a DHCPNAK even though it doesn't know about the address.

So the discussion is it the client or the server at fault?  Is dhcp in pfsense not sending nak?  And you would hope the client that does not get a lease renewal should send discover on its own.

Once I have some coffee will fire up my windows 7 laptop and another wifi network and actual sniff this.

https://support.microsoft.com/en-us/kb/167014
DHCP client may fail to obtain a DHCP-assigned IP address

So question is what exactly is the ip range and scope at site 1 and what is it at site 2..  dhcp server should really send a nak so that the client can then send a discover.

edit:  Ok awake enough to fire off a dhclient request for specific address that does not match up to pfsense dhcp server scope..  So as you can see send a request for 192.168.0.110, which pfsense on this interface 192.168.9.0/24 clearly it sends dhcpnak saying hey you can not use that address.  Client then sends discover asking hey can I have an address..  dhcp server says sure use 192.168.9.7, client then sends request to that offer and says screw that send me 192.168.0.110, which again sends a NAK

So from this simple test sure seems like pfsense dhcp is sending NAK when it should.. But in my test client dhclient didn't care was told to ask for specific IP..  I can fire up windows and see what it does..  But this was faster and easier..  And points to maybe client not paying attention to nak like it should.  Or something in the query that would cause dhcp server not to seed the nak?

dhcptest.pcap

[ttblum]

I changed site B to a 7200 lease time.

A laptop was successfully moved from one office to the other today without any DHCP issues.

I forgot to mention that site A and site B are linked by a point to point T1.  I collected some traffic into a packet capture, and it is showing occasional packets from site B's DHCP server are entering site A's network.

I don't see a setting to be able to turn off DHCP relay on the T1 routers (they are separate from the pfSense).  I changed DHCP max hops to 0 on both T1 routers.

[johnpoz]

"and it is showing occasional packets from site B's DHCP server are entering site A's network."

Why would that be??  Wouldn't your firewall (pfsense) be in front of your router..  How would dhcp traffic be going across your point to point?

Your lease time really shouldn't matter..  Does it take longer than 2 hours to move from site A to site B?  So the lease would be expired by time the laptop got to the other location?

Can you draw up how these sites are connected - I don't see how in a normal setup dhcp packets would be getting between sites - unless you had helper/relay setup?

[ttblum]

Both sites have a point to point T1 directly connecting them, and each also has an internet connection.

Each site has two separate routers - a pfSense router facing the internet, and another router facing the other office.  The pfSense firewall rules are set up to route the traffic to the either the internet, or to the other router, depending on where the traffic is headed.

The other routers have no such setting as dhcp-relay, but since it's happening I'm assuming they are doing it.

After changing 'DHCP max hops' from 4 to 0 on these I'm not seeing any more leaked DHCP traffic from the other side.

[johnpoz]

so your point to point routers are behind pfsense?  So how would clients behind pfsense ever see dhcp from the other site??

So example this is how I picture your setup from your description

[ttblum]

Your picture shows the routers in front of the pfSense, not connected directly to the LAN as they are.

Since each LAN has two different gateways there is a direct path between LANs that does not involve the pfSense.

So, in other words, I didn't find a rogue DHCP server, I found a rogue DHCP relay agent.  I'm still monitoring to see if the laptops can connect OK now.

This is a legacy network I inherited, so yes I will be changing it.

[johnpoz]

Great that you got your issue sorted, and lan having 2 gateways is PITA setup..  Someone didn't know what they were doing would be my guess.

I bring up why something X is setup a specific way at work, and comes down to the guy before was an idiot ;)  So I use that same argument when setting something up at work or going over a design with colleagues – so when the next guy looks at this is he going to think you were an idiot?  Next guy not always aware of time constraints, budgets that lead to short cuts..  And none of which are really valid excuses for shit setup anyway ;)

So when doing something I like to check my work by thinking hey is the guy after me that looks at this going to think I was a complete moron or what? ;)  Document, Document, Document - and if you do something that is odd ball document why..  So even when you come back to look at it a few months later you don't think to yourself WTF was I thinking ;)  Oh yeah this is why we had to do it that way when you look in the docs...