What clue did I miss?

[amnixed]

Hello Experts,

I had a bit of ipsec troubleshooting adventure. It took a long time for me to stumble upon what fixed the problem, although I don't know what the problem was. Here is what happened:

• Customer has 2 working ipsec tunnels

• Tunnel A <--> B

• Tunnel A <--> C

• At node B, VoIP provider installs Juniper SRX300 between SG-2440 and ISP.

• At node B, public IP address changed at VoIP provider's request.

• Reconfiguring ipsec at node A and at node B with new IP address: no connection.

• At node B, connecting SG-2440 directly to ISP, disconnecting Juniper: no connection.

• Fiddling with various (matching) settings both at node A and node B: no connection.

• Tunnel A <--> C works fine all along.

• Checking firewall rules, no blocking rule found.

• Poring over log files both at node A and node B

• Node B says "peer not responding, trying again"

• From node B, trying nmap -sU -p500 -Pn [node A]: 500/udp open isakmp

• From node B, trying nmap -sU -p4500 -Pn [node A]: 4500/udp open|filtered nat-t-ike

• Really desperate now.

• Set up ipsec with same parameters between node A and node D (not the same customer)

• Tunnel A <--> D: no connection

• Set up ipsec with same parameters between node B and node D (not the same customer)

• Tunnel B <--> D: CONNECTION!!!

• What now?

• At node A, replacing FW-7541/RELEASE 2.3_1 with SG-4860/2.4.4-RELEASE-p3

• Tunnel A <--> B: CONNECTION!!!

Some of you might find all this stumbling and bumbling amusing.

How would you go about this problem more efficiently?

What clue did I miss?

What was/is the problem?

It's really not satisfying to conclude that "if new IP address and a pass-through device at node B, replace router at node A"