Almost perfect multi WAN setup

dekopolis

I've looked on the forums and I've looked at several of the multi-WAN examples floating around, but I still can't figure this one out. All the numbers on this page are fictitious, but I kept them similar to make things as close to the original as possible.

Everything seemed to work just find, and then this…

Here is the basic problem:

When browsing the internet, website "subdomain.websiteinquestion.com" does not respond.
Here are the facts:
1. "subdomain.websiteinquestion.com" => 216.45.128.206
2. "www.websiteinquestion.com" => 216.45.128.212
3. pinging "subdomain.websiteinquestion.com" does not work from behind pf, but works fine from the pfsense diagnostic ping page.
4. pinging "www.websiteinquestion.com" works just fine
5. websiteinquestion.com's ip address is used by the same provider that provides the T1 (I know this because the owner of the site told me, and #6 pretty much confirms it.)
6. a traceroute to "subdomain.websiteinquestion.com" from diagnostic page shows only 4 hops
7. if i let traffic flow on default gateway, then issue goes away

Extra Facts:
1. In a previous attempt at this, I tried to use the DNS servers as the monitor ips for the failover pool. I use the first DNS for the cable side to monitor that connection and the first DNS from the T1 to monitor the WAN. The T1 monitor would fail sporadically. In other words, pinging the DNS would fail about half the time.
2. Pinging the DNS of the T1 from behind the pf in the setup below behaves the same way.

Here is my setup:

Thanks!!!

ktims

What's a traceroute to the IP that's not working, from behind pfSense, look like? From what you've described, I don't think the problem is your pfSense.

cmb

What pfSense version?

Perry

3. pinging "subdomain.websiteinquestion.com" does not work from behind pf, but works fine from the pfsense diagnostic ping page.

pfsense will use wan while lan uses wan2

Tests I would preform:
MTU check of WAN2 http://forum.pfsense.org/index.php/topic,13649.msg72930.html#msg72930
Enable DNS forwarder and let it use opendns while the DNS server forwards to pfSense http://pfsense.comuf.com/multiwan.html

dekopolis

Thanks for the response everyone!!!

ktims:

What's a traceroute to the IP that's not working, from behind pfSense, look like?

Traceroute to IP I can't ping on WAN2:
Traceroute to 216.45.128.206
1 * * * Request timed out.
2 7 ms 6 ms 6 ms ge-1-1-ur01.blah.blah.net [68.102.150.169]
3 7 ms 6 ms 6 ms te-9-1-ur02.blah.blah..net [68.102.148.102]
4 10 ms 11 ms 12 ms te-7-3-ar02.nblah.blah.net [68.102.148.225]
5 13 ms 11 ms 14 ms po-1-ar01.blah.blah..net [68.90.232.101]
6 12 ms 14 ms 15 ms te-0-3-0-1-cr01.blah.blah..net [68.102.90.133]
7 27 ms 30 ms 29 ms pos-1-8-0-0-cr01.blah.blah.t.net [68.102.85.17]
8 28 ms 28 ms 28 ms 64.132.130.249
9 49 ms 50 ms 48 ms 66.192.120.202
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
…Eventually goes over maximum hops....

Traceroute to IP I can ping on WAN2:
Traceroute to 216.45.128.212
1 * * * Request timed out.
2 8 ms 6 ms 6 ms ge-1-1-ur01.blah.blah.net [68.102.150.169]
3 8 ms 9 ms 6 ms te-9-1-ur02.blah.blah..net [68.102.148.102]
4 11 ms 11 ms 12 ms te-7-3-ar02.nblah.blah.net [68.102.148.225]
5 11 ms 11 ms 14 ms po-1-ar01.blah.blah..net [68.90.232.101]
6 12 ms 14 ms 15 ms te-0-3-0-1-cr01.blah.blah..net [68.102.90.133]
7 27 ms 30 ms 29 ms pos-1-8-0-0-cr01.blah.blah.t.net [68.102.85.17]
8 28 ms 28 ms 28 ms 64.132.130.249
9 75 ms 50 ms 48 ms 66.192.120.202
10 53 ms 52 ms 53 ms 216.45.128.212

cmb

What pfSense version?

This is pfSense ver. 1.2.2

Perry:

Tests I would preform:
MTU check of WAN2 http://forum.pfsense.org/index.php/topic,13649.msg72930.html#msg72930

I have checked the MTU on both WAN/WAN2 and 1472 worked, so I think MTU is ok. I also double checked the subnet masks assigned by the ISPs and they are correct.

Enable DNS forwarder and let it use opendns while the DNS server forwards to pfSense http://pfsense.comuf.com/multiwan.html

I enabled the DNS forwarder and put the OpenDNS servers on the general page. DHCP still gives out 192.168.0.2, but that DNS server now forwards to the pfSense box. I created two static routes just like the multiwan document in the link shows, but there no change.

Perry

Maybe Diagnostics -> Packet can give a hint on what's going on

dekopolis

Perry

Maybe Diagnostics -> Packet can give a hint on what's going on

I tried going under Diagnostics -> Packet Capture:
I captured on WAN2, filtered for 216.45.128.206, Packet length 1500, Full Detail.
During the capture I ran a ping to the IP…

All I get in the capture is the outgoing ECHO request, and I never see the reply.

How can improve this? In other words, how can I see if PF receives the reply, and if it does, how can I see what it does with it?

Thanks!

Perry

Can you connect to the site if your are directly connect to the cable modem?

dekopolis

Perry…After testing the line directly, it turns out that the owner of the site is blocking our public IP address. I've emailed the webmaster and so has our cable provider to let them know.

Thanks!!