Weird Blocking Issue.

[deanot]

I thought I was getting my head around PfSense.  I have one issue that is making my head hurt, I have been fighting this for days now.

My setup.

Watchguard Firebox x500 core, running 2.2.2 Nanobsd 4g

WAN1 -> PPoE DSL Modem 1
WAN2 -> PPoE DSL Modem 2
LAN1 -> Traffic routed to WAN1
LAN2 -> Taffic routed to WAN2

DHCP and DNS is coming out of a server, this server has access to both networks.

The thing that is killing me is this, when I cut over my network to use PfSense, it works great, well for a few minutes….  Traffic works fine on all interfaces, then after a few minutes my server stops resolving addresses, can't see the internet and generally will do nothing.

A reboot of the PfSense box will fix it, for a few more minutes.  This problem only occurs on WAN1, which my server is set to use as a default gateway.  I have to put in place the original router so I can get my server back online.  As you can imagine, when my server can't resolve an address, all of the other machines using it for DNS are unable to get the resolution required by it.

I have set up rules in the firewall to allow DNS to flow, but this is not the case, I can't ping an address like 8.8.8.8 (google server) when connected through PfSense, but I can if I use the other WAN.

Any thoughts on why this is happening? I thought maybe a network card flaking out, but these boxes have 4 or 8 ports in them, mine has just the 4, so I should imagine they use the same driver.

I am lost.  Hope someone can shed some much needed light on this for me.

Many thanks.

[tim.mcmanus]

Can you post screen shots of your firewall rules for all interfaces?

[deanot]

Thanks for the reply… here are the screenshots.  I am pretty much a noob at this, but slowly getting my tiny little brain around it.  The network numbering is going to change as I intend on setting up a VPN later.  So please ignore the lame range.

[deanot]

In another attempt to fix this, I changed my servers Gateway over to use the WAN2_PPPOE and it works.  I have reasons to have it using WAN_PPPOE, so this really can't change or my VOIP phone system will fail to work.

I do not understand why this WAN is doing this, WAN1 was the first to be setup and seemed to work fine in earlier testing.

[deanot]

Interestingly enough, I moved the machine I am using to use the gateway of WAN1 and it works fine.  Something is blocking my server from using WAN1, but it will use WAN2.  Still digging into this..

[deanot]

I seem to be making some progress, have a few more items to look into.

[deanot]

I think I found the issue, user input would be to blame.

Both PPoE connections use a password, it would seem I fat fingered the password in WAN1 more than once.  I have made sure time and time again it was correct, apparently every time but the last time was wrong, I don't know how that could have been seeing as both PPoE interfaces use the same passwords.

So, this issue is resolved.  Still confused as to how the password was wrong, several time….  Oh well, the wonders of the mind and fat fingering baffle me.

Appreciate the help.

[deanot]

Just an update on this, the password had nothing to do with it.  In fact I feel like an idiot, as the problem was a conflicting IP address.  I thought I changed the IP of the old router away from the new one, once done I changed the PfSense WAN1 IP.  Neither reported an issue, no logs on the PfSense end showed this either.  It has now been up and running for the past 4-5 hours without failing.