VoiP LAN device stuck on DHCP renewal



  • Hi, I am using pfSense on industrial Intel Core I3 PC with with 4x1G LAN. Currently is just PPPoE and LAN with DHCP and DNS resolver- pretty basic configuration and I have snort running.
    I have one device: VoiP controller connected to pfSense via 8 port 1G LAN manageable switch which give me a problem with DHCP renewal. Once I started using it with pfSense, I've had VoiP device goes offline every 2 hours or 7200 seconds, which is default max lease time. As temporary solution, I have extended max lease time to 86400 seconds or 24 hours and device going offline and must be restarted once a day. All other things I am sure done correctly including firewall rules and port forward, PureNAT e.t.c. since I have device calling both ways and staying online within active DHCP lease time. I've tried static IP out of DHCP range with same no success.
    I have dozen other devices which are OK in current setup as I see. Before that, VoiP controller being working few years with DD-WRT router and stays online for few weeks and I have same Internet provider all the time.

    What I see from system logs, VoiP controller have no problem renew lease many time within 24 hours, but then it suddenly stuck in cycle DHCP OFFER->DHCP DISCOVER ->DHCP OFFER->DHCP DISCOVER->DHCP OFFER->DHCP DISCOVER

    I am wiresharked it with DD-WRT and it looks the same except it gets IP from DD-WRT after lease expired but nothing from pfSense. I've tried DD-WRT lease time 5, 20, 120 min and 24 hours and it works.

    The only clue I have: may be pfSense response to fast for VoiP 100Mbit controller and no any settings for slower device support in pfSense.
    I see same DHCP behaviour reported by other users with other devices which makes me think there is something in pfSense makes it not fully compatible with broad range of devices.
    Any help or ideas much appreciated.



  • Capture the DHCP traffic for that device and post here.  Is that DD-WRT there all the time, providing DHCP?  I doubt it's pfSense responding "too fast", as it will still be limited by the hardware.  Even if pfSense is on a GB switch, the VoIP device connection to the switch will only be 100 Mb, so the switch will buffer the packets until they can be sent at the lower rate.  Since that's a managed switch, you should be able to configure it for port mirroring, so that you can connect a computer running Wireshark to it, to monitor only the traffic on the port the VoIP device is connected to.



  • Yes, DD-WRT give IP all the time after lease expired. I've done capture already using mirroring port, I need some time to extract logs into text file to post it here.
    Thanks.



  • Perhaps you should disable the DD-WRT DHCP server, until this is resolved.  You can have multiple DHCP servers, but you should ensure that they don't try to hand out the same address block.  However, once a lease has been established, the clients should be going back to the same server for renewals.



  • Sorry, I have run pfSense only now. DD-WRT disconnected. I've being using DD-WRT alone for test without pfSense, so no dual DHCP conflict issue here and modem is in bridge mode..



  • Here is moment when VoiP device goes offline with pfSense alone when lease expired:

    No.    Time        Source                Destination          Protocol Length Info
        190 1215.931621 0.0.0.0              255.255.255.255      DHCP    317    DHCP Discover - Transaction ID 0xbe67eb

    Frame 190: 317 bytes on wire (2536 bits), 317 bytes captured (2536 bits)
    Ethernet II, Src: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Internet Protocol Version 4, Src: 0.0.0.0 (0.0.0.0), Dst: 255.255.255.255 (255.255.255.255)
    User Datagram Protocol, Src Port: 68 (68), Dst Port: 67 (67)
    Bootstrap Protocol (Discover)

    No.    Time        Source                Destination          Protocol Length Info
        191 1215.931742 10.10.1.1            10.10.1.2            ICMP    62    Echo (ping) request  id=0x3035, seq=0/0, ttl=64 (no response found!)

    Frame 191: 62 bytes on wire (496 bits), 62 bytes captured (496 bits)
    Ethernet II, Src: IskraTra_01:02:03 (00:0e:c4:01:02:03), Dst: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22)
    Internet Protocol Version 4, Src: 10.10.1.1 (10.10.1.1), Dst: 10.10.1.2 (10.10.1.2)
    Internet Control Message Protocol

    No.    Time        Source                Destination          Protocol Length Info
        192 1216.948554 10.10.1.1            255.255.255.255      DHCP    342    DHCP Offer    - Transaction ID 0xbe67eb

    Frame 192: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
    Ethernet II, Src: IskraTra_01:02:03 (00:0e:c4:01:02:03), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Internet Protocol Version 4, Src: 10.10.1.1 (10.10.1.1), Dst: 255.255.255.255 (255.255.255.255)
    User Datagram Protocol, Src Port: 67 (67), Dst Port: 68 (68)
    Bootstrap Protocol (Offer)

    No.    Time        Source                Destination          Protocol Length Info
        193 1218.928157 0.0.0.0              255.255.255.255      DHCP    317    DHCP Discover - Transaction ID 0xbe67eb

    Frame 193: 317 bytes on wire (2536 bits), 317 bytes captured (2536 bits)
    Ethernet II, Src: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Internet Protocol Version 4, Src: 0.0.0.0 (0.0.0.0), Dst: 255.255.255.255 (255.255.255.255)
    User Datagram Protocol, Src Port: 68 (68), Dst Port: 67 (67)
    Bootstrap Protocol (Discover)

    No.    Time        Source                Destination          Protocol Length Info
        194 1218.928257 10.10.1.1            10.10.1.2            ICMP    62    Echo (ping) request  id=0x3035, seq=0/0, ttl=64 (no response found!)

    Frame 194: 62 bytes on wire (496 bits), 62 bytes captured (496 bits)
    Ethernet II, Src: IskraTra_01:02:03 (00:0e:c4:01:02:03), Dst: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22)
    Internet Protocol Version 4, Src: 10.10.1.1 (10.10.1.1), Dst: 10.10.1.2 (10.10.1.2)
    Internet Control Message Protocol

    No.    Time        Source                Destination          Protocol Length Info
        195 1219.929553 10.10.1.1            255.255.255.255      DHCP    342    DHCP Offer    - Transaction ID 0xbe67eb

    Frame 195: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
    Ethernet II, Src: IskraTra_01:02:03 (00:0e:c4:01:02:03), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Internet Protocol Version 4, Src: 10.10.1.1 (10.10.1.1), Dst: 255.255.255.255 (255.255.255.255)
    User Datagram Protocol, Src Port: 67 (67), Dst Port: 68 (68)
    Bootstrap Protocol (Offer)

    No.    Time        Source                Destination          Protocol Length Info
        196 1226.937692 0.0.0.0              255.255.255.255      DHCP    317    DHCP Discover - Transaction ID 0xbe67eb

    Frame 196: 317 bytes on wire (2536 bits), 317 bytes captured (2536 bits)
    Ethernet II, Src: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Internet Protocol Version 4, Src: 0.0.0.0 (0.0.0.0), Dst: 255.255.255.255 (255.255.255.255)
    User Datagram Protocol, Src Port: 68 (68), Dst Port: 67 (67)
    Bootstrap Protocol (Discover)

    No.    Time        Source                Destination          Protocol Length Info
        197 1226.937795 10.10.1.1            10.10.1.2            ICMP    62    Echo (ping) request  id=0x3035, seq=0/0, ttl=64 (no response found!)

    Frame 197: 62 bytes on wire (496 bits), 62 bytes captured (496 bits)
    Ethernet II, Src: IskraTra_01:02:03 (00:0e:c4:01:02:03), Dst: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22)
    Internet Protocol Version 4, Src: 10.10.1.1 (10.10.1.1), Dst: 10.10.1.2 (10.10.1.2)
    Internet Control Message Protocol



  • It looks like that VoIP device is ignoring the DHCP offers.  After the offer, the client requests the offered address.  Are you mirroring the port it's connected to?

    BTW, if you filter on DHCP, you won't get those pings cluttering things up.



  • Yes, I know and I can't capture same situation with DD-WRT.
    Actually this capture done by pfSense "packet capture" function and opened by wireshark. Port mirroring I was using with DD-WRT setup.

    "BTW, if you filter on DHCP, you won't get those pings cluttering things up." - not sure what you mean. I ran capture full detail on purpose do not miss anything.



  • If you're trying to resolve a problem with a protocol, it generally helps to capture only that protocol.  Packet Capture will allow you to do that.



  • @JKnott:

    If you're trying to resolve a problem with a protocol, it generally helps to capture only that protocol.  Packet Capture will allow you to do that.

    OK, understood your point, thanks. Here is same moment with DD-WRT captured with switch mirrored port and Wireshark but VoiP controller remains online after following sequence:

    –----------------------------------------------------------------------------------------------------------------------------------------
    No.    Time          Source                Destination          Protocol Length Info
        584 740.807538000  0.0.0.0              255.255.255.255      DHCP    302    DHCP Discover - Transaction ID 0xbe67eb

    Frame 584: 302 bytes on wire (2416 bits), 302 bytes captured (2416 bits) on interface 0
    Ethernet II, Src: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Internet Protocol Version 4, Src: 0.0.0.0 (0.0.0.0), Dst: 255.255.255.255 (255.255.255.255)
    User Datagram Protocol, Src Port: 68 (68), Dst Port: 67 (67)
    Bootstrap Protocol (Discover)

    No.    Time          Source                Destination          Protocol Length Info
        585 740.837757000  D-LinkIn_e1:f2:9a    Broadcast            ARP      60    Who has 192.168.70.102?  Tell 192.168.70.1

    Frame 585: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) on interface 0
    Ethernet II, Src: D-LinkIn_e1:f2:9a (c0:a0:bb:e1:f2:9a), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Address Resolution Protocol (request)

    No.    Time          Source                Destination          Protocol Length Info
        586 741.835987000  D-LinkIn_e1:f2:9a    Broadcast            ARP      60    Who has 192.168.70.102?  Tell 192.168.70.1

    Frame 586: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) on interface 0
    Ethernet II, Src: D-LinkIn_e1:f2:9a (c0:a0:bb:e1:f2:9a), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Address Resolution Protocol (request)

    No.    Time          Source                Destination          Protocol Length Info
        587 742.891015000  D-LinkIn_e1:f2:9a    Broadcast            ARP      60    Who has 192.168.70.102?  Tell 192.168.70.1

    Frame 587: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) on interface 0
    Ethernet II, Src: D-LinkIn_e1:f2:9a (c0:a0:bb:e1:f2:9a), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Address Resolution Protocol (request)

    No.    Time          Source                Destination          Protocol Length Info
        588 743.424168000  169.254.255.1        255.255.255.255      DHCP    342    DHCP Offer    - Transaction ID 0xbe67eb

    Frame 588: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits) on interface 0
    Ethernet II, Src: D-LinkIn_e1:f2:9a (c0:a0:bb:e1:f2:9a), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Internet Protocol Version 4, Src: 169.254.255.1 (169.254.255.1), Dst: 255.255.255.255 (255.255.255.255)
    User Datagram Protocol, Src Port: 67 (67), Dst Port: 68 (68)
    Bootstrap Protocol (Offer)

    No.    Time          Source                Destination          Protocol Length Info
        589 743.424341000  VoiP_aa_00_bb_22    Broadcast            ARP      60    Gratuitous ARP for 192.168.70.102 (Request)

    Frame 589: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) on interface 0
    Ethernet II, Src: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Address Resolution Protocol (request/gratuitous ARP)

    No.    Time          Source                Destination          Protocol Length Info
        590 743.424353000  VoiP_aa_00_bb_22    Broadcast            ARP      60    Gratuitous ARP for 192.168.70.102 (Reply)

    Frame 590: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) on interface 0
    Ethernet II, Src: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Address Resolution Protocol (reply/gratuitous ARP)

    No.    Time          Source                Destination          Protocol Length Info
        591 743.424575000  192.168.70.1          192.168.70.102        ICMP    62    Echo (ping) request  id=0x24d2, seq=0/0, ttl=64 (no response found!)

    Frame 591: 62 bytes on wire (496 bits), 62 bytes captured (496 bits) on interface 0
    Ethernet II, Src: D-LinkIn_e1:f2:9a (c0:a0:bb:e1:f2:9a), Dst: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22)
    Internet Protocol Version 4, Src: 192.168.70.1 (192.168.70.1), Dst: 192.168.70.102 (192.168.70.102)
    Internet Control Message Protocol

    No.    Time          Source                Destination          Protocol Length Info
        592 743.424768000  0.0.0.0              255.255.255.255      DHCP    314    DHCP Request  - Transaction ID 0xbe67eb

    Frame 592: 314 bytes on wire (2512 bits), 314 bytes captured (2512 bits) on interface 0
    Ethernet II, Src: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Internet Protocol Version 4, Src: 0.0.0.0 (0.0.0.0), Dst: 255.255.255.255 (255.255.255.255)
    User Datagram Protocol, Src Port: 68 (68), Dst Port: 67 (67)
    Bootstrap Protocol (Request)

    No.    Time          Source                Destination          Protocol Length Info
        593 743.479401000  169.254.255.1        255.255.255.255      DHCP    342    DHCP ACK      - Transaction ID 0xbe67eb

    Frame 593: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits) on interface 0
    Ethernet II, Src: D-LinkIn_e1:f2:9a (c0:a0:bb:e1:f2:9a), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Internet Protocol Version 4, Src: 169.254.255.1 (169.254.255.1), Dst: 255.255.255.255 (255.255.255.255)
    User Datagram Protocol, Src Port: 67 (67), Dst Port: 68 (68)
    Bootstrap Protocol (ACK)

    No.    Time          Source                Destination          Protocol Length Info
        594 743.662369000  VoiP_aa_00_bb_22    Broadcast            ARP      60    Who has 192.168.70.1?  Tell 192.168.70.102

    Frame 594: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) on interface 0
    Ethernet II, Src: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Address Resolution Protocol (request)

    No.    Time          Source                Destination          Protocol Length Info
        595 743.662570000  D-LinkIn_e1:f2:9a    VoiP_aa_00_bb_22    ARP      60    192.168.70.1 is at c0:a0:bb:e1:f2:9a

    Frame 595: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) on interface 0
    Ethernet II, Src: D-LinkIn_e1:f2:9a (c0:a0:bb:e1:f2:9a), Dst: VoiP_aa_00_bb_22 (00:25:f6:00:bb:22)
    Address Resolution Protocol (reply)



  • OK, understood your point, thanks. Here is same moment with DD-WRT captured with switch mirrored port and Wireshark but VoiP controller remains online after following sequence:

    You're doing it again.  Please filter on DHCP, so that you don't get all that clutter.  A DHCP sequence should be easy to follow.  If you filter on it, you'll see the discover, offer, request and ack all in order, without anything else.  With all that ARP & ICMP stuff in there, I now have to search for only the DHCP stuff.  If you're trying to solve a DHCP problem, you should be focused only on DHCP.  You'd only be interested in the ARP traffic, if you're trying to find out why offered addresses are being rejected.  In this instance, the ARPs are used for duplicate address detection. which could cause an offered address to be rejected.  IN that capture, I see 12 packets, of which only 4 are relevant.

    Also, what's this about with the link local address?  Does that D-Link have the address 192.168.70.1?  Or 169.254.255.1?

    No.    Time          Source                Destination          Protocol Length Info
        593 743.479401000  169.254.255.1        255.255.255.255      DHCP    342    DHCP ACK      - Transaction ID 0xbe67eb

    The dhcp offer and ack, from the D-Link have a source address of 169.254.255.1, but other traffic uses 192.168.70.1.  In fact, there shouldn't be any DHCP traffic with link local addresses, as they're only supposed to be used when no DHCP server is available.

    Also, I hope you're capturing at the same point.  That is a mirror port of the managed switch.  If you're not doing that you're adding variables that may affect the results.

    Incidentally, a while ago, I bought a cheap managed switch just for this purpose.  I have port 2 mirrored to port 1 and I can then insert the switch between 2 devices and plug a computer running Wireshark into port 1 to monitor the traffic.

    BTW, when I use Wireshark, I use both capture and display filters as needed to get the desired results.  I much prefer Wireshark to Packet Capture, as it's much more flexible and can be viewed in real time.  However, I have used Packet Capture many times and exported to Wireshark for analysis, as it displays more detail.

    Also, it would be helpful to attach the actual capture file, as it shows more detail than what you show.



  • @JKnott:

    OK, understood your point, thanks. Here is same moment with DD-WRT captured with switch mirrored port and Wireshark but VoiP controller remains online after following sequence:

    You're doing it again.  Please filter on DHCP, so that you don't get all that clutter.  A DHCP sequence should be easy to follow.  If you filter on it, you'll see the discover, offer, request and ack all in order, without anything else.  With all that ARP & ICMP stuff in there, I now have to search for only the DHCP stuff. 
    Also, what's this about with the link local address?  Does that D-Link have the address 192.168.70.1?  Or 169.254.255.1?

    No.    Time          Source                Destination          Protocol Length Info
        593 743.479401000  169.254.255.1        255.255.255.255      DHCP    342    DHCP ACK      - Transaction ID 0xbe67eb

    The dhcp offer and ack, from the D-Link have a source address of 169.254.255.1, but other traffic uses 192.168.70.1.  In fact, there shouldn't be any DHCP traffic with link local addresses, as they're only supposed to be used when no DHCP server is available.

    Also, it would be helpful to attach the actual capture file, as it shows more detail than what you show.

    I posted that fragment on purpose and 169.254.255.1 is mystery for me too. Router here is 192.168.70.1. Most of DHCP traffic with DD-WRT and pfSense is like: DHCP Request ->DHCP ACK between lease expiration. I was trying to find clue when renewal is not successful or have any hiccup which are very rare in DD-WRT case.

    Looks like I need to repeat capture with pfSense and Wireshark+mirrored port. The good news are it's very repetitive failure.
    Thanks.



  • Here is attached files:
    1. filtered DHCP capture with pfSense and Wireshark+mirrored port. Lease time set to 3600 sec in pfSense
    2. pfSense DHCP syslog at the same time

    Packets 3, 5, 8, 9 - VoiP soft restart by dealing restart code from attached phone. Looks like legitimate DHCP renewal.
    Somewhere after packet #592 VoiP controller going offline and indicator changing from GREEN to RED and disconnected from SIP. In this mode I can restart it again by dealing restart code from attached phone.

    [pfSen+VoiP fail capture txt.txt](/public/imported_attachments/1/pfSen+VoiP fail capture txt.txt)
    [pfSense DHCP log xls.xls](/public/imported_attachments/1/pfSense DHCP log xls.xls)



  • Perhaps I'm missing, but those look entirely normal, other than somewhat short interval for a 3600 sec.  lease time.  I see the initial discover, offer, request & ack, followed by request & ack for each renewal.  What is strange, however, is the ack is going to the broadcast address and not the assigned IP.  Incidentally, I had asked you to attach the actual capture file, not a text interpretation of it.  The reason I did was, as I mentioned, the capture file contains more detail than the text.  For example, with the capture file, I could examine what the client sent to the server in the request.  Did it actually request the offered address?  Or somehow requested 255.255.255.255?  I can't tell from the text file.  Regardless, that ack to the broadcast address is wrong.



  • Sorry, I am bit hesitant to attach complete capture file and post online. There some MDNS and other garbage from Linux PC I don't know yet how to completely stop when wiresharking from Linux side. So here skimmed DHCP only version as you prefer as I understood. All UDP, ARP, ICMP, SIP filtered out.
    I am really curious because of 2 reasons: Should I continue with my VoiP provider and can I full rely on pfSense for future projects and network expansion.
    Thanks.

    [pfSen+VoiP DHCP 3600s lease01.pcapng](/public/imported_attachments/1/pfSen+VoiP DHCP 3600s lease01.pcapng)



  • There some MDNS and other garbage from Linux PC I don't know yet how to completely stop when wiresharking from Linux side.

    This is where creative use of capture filters comes in.  You could create an filter that allows DHCP and also filters on the client and server MAC addresses.  For example, it might be something like port 67 or port 68 and (ether host <mac1>or ether host <mac2>)</mac2></mac1>, with <mac1>and <mac2>being replaced with the actual MAC addresses of the client and server.  This should result in only the desired captures.

    I can't speak about the VoIP provider, but pfSense is an excellent firewall that does much more than much of the competition.</mac2></mac1>



  • Understood, the key word is "creative". I need bit more practice in filters.
    Here is complete file in case.

    [pfSen+VoiP DHCP 3600s lease02.pcapng](/public/imported_attachments/1/pfSen+VoiP DHCP 3600s lease02.pcapng)



  • Option 50, in the request, shows the address 192.168.1.107 is being requested.  The ack shows the correct client address, but for some reason is being sent to the broadcast address.  I have no idea why it would do that.  On my own system, the ack is sent to the correct client address.  The problem with sending to the broadcast address is the client has no idea the ack is for it.  When the broadcast IP address is used, the broadcast MAC address is also used.  A proper ack is sent to the assigned IP address, using the client's MAC address.

    So, the question now becomes, why is the ack going to the broadcast address?



  • I have no idea. I become suspicious about broadcast address from the beginning after I've got mentioned failures and started capturing traffic.



  • I got another look on DHCP sequence and compared to other devices. In short, VoiP controller receiving what was requested in DHCP Discover poll. VoiP set Broadcast flag 0x8000 and pf sense replied to broadcast address accordingly. Other PC and devices do not set Broadcast flag and pfSense replying to machine IP address as mentioned by JKnott.
      The other thing, Wireshark shows End option missing in DHCP Discover which is normally (255) End.
    But most important, for some reason VoiP do not reply on received DHCP Offer basically not following protocol. Correct protocol handling happening during soft/hard restart only which make me pretty much sure it's VoiP device software problem.
    Thanks for tips.


 

© Copyright 2002 - 2018 Rubicon Communications, LLC | Privacy Policy