Changing hardware… issue with NAT 1:1?

  • Hello all,

    I've been having trouble with the hardware in my current firewall (watchdog timeout on a NIC, dropping packets).

    I want to move over to a new server, so I installed pfSense 1.2.2, reconfigured everything from scratch, including the Virtual IP's, 1:1's, rules, etc…
    Once the configuration was done, I moved the wires over thinking that's all I had to do. After checking two working servers, I thought all was well, until I checked my third, fourth, and fifth servers to find they had no external access. I double checked the internal IP's, NAT and found nothing wrong in the configuration. I disabled/re-enabled the network adapters on the servers, flushed DNS cache, and cleared the ARP cache. I was just about out of ideas until I changed the external 1:1 IP for one of the servers that wasn't working, and sure enough, it started working with the new 1:1 external IP!

    I had some prior engagements lined up, so I was running out of time messing with the new firewall, so I moved the wires back to the old firewall (nothing changed, I didn't even turn it off) to find that the two servers that worked right away on the new firewall stopped working, and the 3 that didn't work on the new firewall started working on the old firewall. Can anyone begin to explain this behavior? I talked to two different techs at my ISP (very good tech support, local ISP running an Ethernet over Copper connection, 24x7 human support) and they couldn't find anything wrong on their end. They did all they knew to do, unfortunately with no resolve.

    I really need to get this working on a new firewall with the same exact configuration and external IP's for the servers with as minimal downtime as possible. And if anyone is wondering, I did originally backup the entire configuration of my old firewall and import it to the new firewall, that didn't work, so I then redid the configuration from scratch. I also made sure none of the NICs in the old firewall had any cables plugged in before trying the new firewall.

        - Adam

  • the first thing that comes to mind is differing MAC addresses on a managed switch. did you flush the ARP table on the ISP's switch/router?

  • We don't have any managed switches here. I asked my ISP to clear the ARP table, and it seemed like they thought I was speaking in tongues. I never got a clear answer from them about that. Would that most likely be the cause, the ARP table on their end?

    Thanks for the reply!
       - Adam

  • I would concur that it sounds like cached ARP entries on the providers end. If you have a CPE router, power cycle it. If you just get an ethernet handoff, this is a pain. You just have to wait it out, or get someone smart from the ISP.

  • Wow, sorry about the late update! You guys were absolutely right, had a higher Tier tech clear the ARP cache before moving over to the new hardware, and sure enough it worked like a charm.

    Thanks for the help!
      - Adam