Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    2 subnets, unintentional bridging = Intermittent slow WAN?

    Scheduled Pinned Locked Moved General pfSense Questions
    1 Posts 1 Posters 1.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      stevedemon
      last edited by

      Hi guys,

      I've recently replaced a CentOS box we've been using at work as our router with a PFSense 1.2.3 install on a AMD x64 Dual core machine with 3 NIC's. We've got two subnets, one for data 10.0.0.0/24 and one for voice 10.0.1.0/24. I replaced the Linux box because we were dropping our WAN connection about once every 2-4 weeks, without any explanation.

      Unfortunately after switching over to PFSense we are still experiencing such WAN outages, but more in the form of extremely low throughput and excessively high pings. This is especially rough on your VoIP phones as they are constantly talking to a 3rd party server and any disruption or high latency with our WAN connection drops all our phones.

      I've been looking at the system log which has ARP routing issues, which I suspect means my two subnets are being bridged somewhere. Here is our log from the last couple minutes:

      Aug 20 22:46:51	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:46:56	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:46:59	kernel: arp: 10.0.0.24 is on sk0 but got reply from 00:1b:fc:6e:bb:4c on sk1
      Aug 20 22:47:02	kernel: arp: 10.0.0.3 is on sk0 but got reply from 00:1f:c6:60:93:16 on sk1
      Aug 20 22:47:05	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:47:10	last message repeated 2 times
      Aug 20 22:47:10	kernel: arp: 10.0.0.241 is on sk0 but got reply from 00:22:15:ba:98:3b on sk1
      Aug 20 22:47:14	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:47:14	kernel: arp: 10.0.0.3 is on sk0 but got reply from 00:1f:c6:60:93:16 on sk1
      Aug 20 22:47:15	kernel: arp: 10.0.0.212 is on sk0 but got reply from 00:1f:c6:ca:20:db on sk1
      Aug 20 22:47:26	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:47:28	kernel: arp: 10.0.0.3 is on sk0 but got reply from 00:1f:c6:60:93:16 on sk1
      Aug 20 22:47:31	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:47:33	kernel: arp: 10.0.0.210 is on sk0 but got reply from 00:24:21:25:1e:e0 on sk1
      Aug 20 22:47:37	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:47:40	kernel: arp: 10.0.0.6 is on sk0 but got reply from 00:1e:58:ab:24:32 on sk1
      Aug 20 22:47:40	kernel: arp: 10.0.0.4 is on sk0 but got reply from 00:15:5d:00:07:01 on sk1
      Aug 20 22:47:40	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:47:42	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:47:43	kernel: arp: 10.0.0.241 is on sk0 but got reply from 00:22:15:ba:98:3b on sk1
      Aug 20 22:47:43	kernel: arp: 10.0.0.231 is on sk0 but got reply from 00:24:21:25:1e:fb on sk1
      Aug 20 22:47:43	kernel: arp: 10.0.0.215 is on sk0 but got reply from 00:24:21:25:2b:f2 on sk1
      Aug 20 22:47:44	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:47:49	last message repeated 3 times
      Aug 20 22:47:50	kernel: arp: 10.0.0.199 is on sk0 but got reply from 00:30:48:59:e1:d2 on sk1
      Aug 20 22:47:52	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:47:55	kernel: arp: 10.0.0.165 is on sk0 but got reply from 00:02:b3:d4:cf:2b on sk1
      Aug 20 22:47:57	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:48:05	last message repeated 4 times
      Aug 20 22:48:05	kernel: arp: 10.0.0.55 is on sk0 but got reply from 00:e0:18:96:b9:46 on sk1
      Aug 20 22:48:07	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:48:07	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:48:10	kernel: arp: 10.0.0.4 is on sk0 but got reply from 00:15:5d:00:07:01 on sk1
      Aug 20 22:48:10	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:48:10	kernel: arp: 10.0.0.55 is on sk0 but got reply from 00:e0:18:96:b9:46 on sk1
      Aug 20 22:48:16	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:48:18	kernel: arp: 10.0.0.199 is on sk0 but got reply from 00:30:48:59:e1:d2 on sk1
      Aug 20 22:48:19	kernel: arp: 10.0.0.4 is on sk0 but got reply from 00:15:5d:00:07:01 on sk1
      Aug 20 22:48:23	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:48:24	kernel: arp: 10.0.0.209 is on sk0 but got reply from 00:1a:92:94:00:ec on sk1
      Aug 20 22:48:28	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:48:34	kernel: arp: 10.0.0.3 is on sk0 but got reply from 00:1f:c6:60:93:16 on sk1
      Aug 20 22:48:35	kernel: arp: 10.0.0.3 is on sk0 but got reply from 00:1f:c6:60:93:16 on sk1
      Aug 20 22:48:36	kernel: arp: 10.0.0.15 is on sk0 but got reply from 00:1e:8c:86:68:79 on sk1
      Aug 20 22:48:41	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:48:59	last message repeated 7 times
      Aug 20 22:48:59	kernel: arp: 10.0.0.24 is on sk0 but got reply from 00:1b:fc:6e:bb:4c on sk1
      Aug 20 22:49:06	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:49:09	kernel: arp: 10.0.0.2 is on sk0 but got reply from 00:1f:c6:0b:50:49 on sk1
      Aug 20 22:49:10	kernel: arp: 10.0.0.241 is on sk0 but got reply from 00:22:15:ba:98:3b on sk1
      

      Questions:

      1. Is it likely that the PFSense ARP table is getting so mixed up that eventually our WAN connection is affected? And if so, is removing this potential bridge actually going to solve the problem? I haven't noticed any interruption to our LAN traffic, but perhaps PFSense or Free BSD is more sensitive than our switches are.

      2. Is Free BSD more sensitive to such ARP table problems than a Linux kernel would be for routing purposes?

      3. Shouldn't additional bridges on unmanaged switches that don't have STP support cause immediate failures of my network

      3. Are there any other system logs, say from the console, that I can look at to track down anything else that may be causing these issues?

      EDIT: FYI system.log has grown to 512K in 19 hours, if that gives any frame of reference for frequency

      1 Reply Last reply Reply Quote 0
      • First post
        Last post
      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.