Random Disconnect Issues

bgbearcatfan · May 27, 2007, 7:22 AM

Hopefully this is where i should post my problem at.

Recently got my pfsense box up and running using the latest version: 1.2-BETA-1

The past couple days i noticed that the people on our network have been experiencing issues with their instant messengers randomly disconnecting, then reconnecting within the next few seconds. So i decided to pay attention to my messenger and experienced the same thing. So at first i figured it was specific to aol aim messenger. However a few times, i was able to test getting out to websites during the connection drops, and that failed as well, so it seems all connectivity drops anywhere between 20-40 seconds at a time. I check the instant messenger status logs, and match them up with the pfsense logs, and the timestamps match up perfectly. Here is what the pfsense logs are returning:

May 27 03:03:56 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:56 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:52 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:52 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:51 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:51 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:48 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:48 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:47 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:47 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:45 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:45 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:44 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:44 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:43 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:43 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:42 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:42 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:40 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:40 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:39 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:39 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:38 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:38 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:33 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:33 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:31 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:31 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:29 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:29 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:29 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:29 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:26 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:26 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:26 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:26 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!
May 27 03:03:24 kernel: arp: 72.49.xxx.xxx is on re2 but got reply from 00:13:1a:ff:1c:1a on re3
May 27 03:03:24 kernel: arp: 00:08:54:47:15:c4 is using my IP address 72.49.xxx.xxx!

During that exact time period, 03:03:24 - 03:03:56, all connectivity drops.

My current pfsense setup is:

Lan - 10.1.1.0
Wireless - 172.20.1.0
Wan - 72.49.xxx.xxx
OPT1(Wan2) - 72.49.xxx.xxx
OPT2(Wan3) - 72.49.xxx.xxx

All Lan and Wireless network connectivity is routed through OPT2(Wan3).

Let me know if you need any additional info.
Any help or suggesstions is much appreciated.

Perry · May 27, 2007, 7:45 AM

If aol aim messenger is equal to aim talk ( i don't use it myself :) ) you need to forward port 5190 tcp

AIM Talk 5190
AIM Video IM 443,1024-5000,5190 443,1024-5000,5190

from http://portforward.com/cports.htm

bgbearcatfan · May 27, 2007, 7:52 AM

Hi,

Thanks for the response. No they aren't using the talk feature just plain old text conversations. However, it is definitely more than just a instant messenger problem, since everything drops, including ftp transfers, website navigation, etc.

Perry · May 27, 2007, 8:47 AM

ok. that can be some faulty hardware. I've experienced that kind of problems with both a faulty isp modem and a wireless AP (broadcast storm).

See if you can't track/narrow it down with ping.

I've used following approach before.
Ping www.google.com from pfsense.
On clients you run konst pinger http://www.visualsoftru.com/pinger.asp
log the result to a file and schedule vmailer http://www.virdi-software.com/vmailer/ to send it to your email so it can be examined.

bgbearcatfan · May 27, 2007, 5:41 PM

I was guessing a hardware problem as well.

Just put in a ticket to my isp to have the modem replaced. I will give you all an update if that corrects the issue.

Thanks for the responses Perry.

cmb · May 27, 2007, 9:28 PM

The messages you're seeing are indicative of having multiple interfaces plugged into the same broadcast domain, which is not good. It looks more like you have some sort of misconfiguration than a hardware problem, IMO, though it could be a combination of both.

bgbearcatfan · May 28, 2007, 3:17 AM

Well i'm not sure what it could be. I don't have anything like traffic shaping, or load balancing or anything enabled.. All internal traffic is routed through the wan3 (opt2) interface. It's the basic pfsense setup besides the two additional wan connections… Is it a problem to have multiple wan connections coming from the same default gateway?

cmb · May 28, 2007, 3:57 AM

Oh, re2 and re3 are probably both WAN interfaces of yours aren't they? That changes things, and makes me lean towards hardware issues again.

bgbearcatfan · May 28, 2007, 3:59 AM

Yes, exactly. To be more descriptive:

xl0 - WAN

re0 - LAN

re1 - Internal Wireless

re2 - WAN2

re3 - WAN3

bgbearcatfan · May 28, 2007, 9:51 PM

A bit of an update.

I'm kinda of leaning away from it being a modem issue.. Last night i plugged my laptop directly into the modem using the usb connection, and it did not disconnect at all, while at the same time leaving my desktop connected normally to pfsense, and the connection dropped about 12 times..

cmb · May 28, 2007, 9:59 PM

Doesn't mean there isn't an issue with the Ethernet on your modem that isn't an issue with USB.

Are all your WAN interfaces going to the same DSL modem? Does this affect traffic regardless of which WAN it is directed to?

bgbearcatfan · May 29, 2007, 2:32 AM

Good point..

The disconnections happened regardless of which wan interface i routed traffic through.. I'll leave this alone until the new modem comes in.. thanks again

cmb · May 29, 2007, 3:59 AM

So they are all 3 going to the same DSL modem? Couldn't you just one use WAN if you only have one ISP? I'm curious if the problem would go away without a multi-WAN setup.

bgbearcatfan · May 29, 2007, 6:05 AM

well the multiwan setup is so that i can host a couple websites, and keep them all on the default port 80, instead of having to do something messy like www.secondwebsite.com:90, etc., That was my main reason for moving from an ipcop environment to pfsense. Right now, since everyone is gone, i plugged my desktop directly into the modem via a cat53 cable, so this will tell me if it's a modem issue or not.

Perry · May 29, 2007, 6:58 AM

Are your wan ip's dynamic? most multi ip setup from one provide there would be only one physical line to pfsense with multi ip's that one can assign with virtual ip in pfsense.

The reason i said you should ping from pfsense was to check the line out but still keep every thing connected.

bgbearcatfan · May 29, 2007, 6:12 PM

Ok,

After all last night, and early into today, i have not had one single disconnect issue when plugging directly into the modem, using both the usb port, and the ethernet port, so the modem appears to be functioning correctly.

As for your question Perry, yes, the wan ip's are dynamic. I have been unable to get logged into the gui while the connection dropped to see if pfsense can still ping out, but i will keep trying that.

sullrich · May 30, 2007, 12:00 AM

Upgrade to a recent testing snapshot: http://snapshots.pfsense.com/FreeBSD6/RELENG_1_2/updates/

bgbearcatfan · May 30, 2007, 3:35 AM

Alright i just upgraded to the lastest snapshot. Will keep you all updated.

Thanks again for all the great support. None of you all have to do this, so thanks again.

bgbearcatfan · May 30, 2007, 5:56 PM

Ok,

I just upgraded to the most recent testing snapshot available, and still getting the same thing as before. Did i miss a step in configuring pfsense for a multiwan setup? I figured i could just add the two additional wan interfaces, and then route the traffic how i want using firewall rules.

bgbearcatfan · Jun 3, 2007, 4:40 AM

Thought i would give an update, hopefully someone has a few more ideas for me to try. I took out the two nics that i was using for my multiple wans, and installed pfsense again using one interface for the lan, one for the wan, and one for my wireless network. After 2 days of this, i haven't had one disconnection issue. So the problem only occurs when running multiple wans.