Machines on bridged LAN interface cannot reach each other

DarkPhoenix · Mar 6, 2011, 9:30 PM

In simplified form topology is simple:
1 WAN interface
4 physical LAN interfaces, bridged into BRIDGE interface

Machine 1 attached to LAN interface cannot ping/reach using any other method another machine plugged into another physical LAN interface.

Router can be pinged from both machines. Both machines can be pinged from router.

Arp shows only MAC of router:

Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.1.1              ether   26:7d:88:1f:eb:71   C                     eth0

Which is weird, because on properly configured bridged interface arp should discover all machines. Any ideas how to fix it?

wallabybob · Mar 6, 2011, 9:58 PM

@DarkPhoenix:

Any ideas how to fix it?

First it would be good to figure out what is going on.

How about setting up a ping to go through the pfSense. Then check the firewall log to see if your ping is being blocked.

DarkPhoenix · Mar 7, 2011, 6:18 AM

@wallabybob:

How about setting up a ping to go through the pfSense. Then check the firewall log to see if your ping is being blocked.

Pings from machine 1 do not reach machine 2. I had rules in firewall for such experiment, allow traffic from BRIDGE subnet to *, and from BRIDGE subnet to BRIDGE subnet with logging (has higher priority) - no entries regarding blocked/passed pings were shown.

One of my friends explained that such packets shouldn't be shown in log as they shouldn't be routed/firewalled at all - because source and destination are on the bridged interfaces on the same subnet, and if they're on the same subnet on bridged interface - arp should show macs of all active machines. They're not shown in my case, and possibly here lies the problem.

stephenw10 · Mar 7, 2011, 12:54 PM

In pfSense, by default, all bridges are filtered so you should see stuff in the firewall logs if it's blocked.

Steve

wallabybob · Mar 7, 2011, 2:10 PM

@DarkPhoenix:

arp should show macs of all active machines.

I suspect that is not true. This is how I think ARP works: ARP WHOIS requests are sent to the broadcast MAC address but I believe ARP responses go to the MAC address of the sender of of the request. Hence machines will generally see only the ARP responses to their own requests.

Bridges however need to know the MAC addresses of all machines connected to them.

Based on my experience with DHCP and getting it working across a bridge I would say there are a few difference places where firewall rules can trip you up. Your bridge will have at least a couple of members, say em0 and igb0 for the sake of example. Lets say bridge0 is your LAN interface. Then bridge0 will have a default "allow any" firewall rule. But unless you specify otherwise, default "block all" firewall rules apply at the member interfaces. I can't recall if these rules have logging enabled. I suspect not.

After you mess with the firewall rules you may need to reset firewall states to get the rules to take effect.

stephenw10 · Mar 7, 2011, 2:43 PM

@wallabybob:

Then bridge0 will have a default "allow any" firewall rule. But unless you specify otherwise, default "block all" firewall rules apply at the member interfaces.

Hmm, interesting possibility. You'd hope that setting up a bridge would take care of that.

What about disabling bridge filtering? Even if just for a test it would be interesting.

Have a read through this thread about bridging lan-wifi. Do you have to use sysctls to disable filtering as is shown? My 2.0 box is down after I made one tweak too many! :P

Steve

DarkPhoenix · Mar 8, 2011, 6:46 AM

I already have couple of rules which should allow such traffic:

		 <rule><id><type>pass</type>
			<interface>opt4</interface>
			 <tag><tagged><max><max-src-nodes><max-src-conn><max-src-states><statetimeout><statetype>keep state</statetype>
			 <os><source>
				<network>opt4</network>

			 <destination><network>opt4</network></destination></os></statetimeout></max-src-states></max-src-conn></max-src-nodes></max></tagged></tag></id></rule> 
		 <rule><id><type>pass</type>
			<interface>opt4</interface>
			 <tag><tagged><max><max-src-nodes><max-src-conn><max-src-states><statetimeout><statetype>keep state</statetype>
			 <os><source>
				<network>opt4</network>

			 <destination><any></any></destination></os></statetimeout></max-src-states></max-src-conn></max-src-nodes></max></tagged></tag></id></rule>

Also, my full config file with users removed can be found in attachment.

@stephenw10:

What about disabling bridge filtering? Even if just for a test it would be interesting.

It seems to be disabled:

		 <tunable>net.link.bridge.pfil_bridge</tunable>
			<value>default</value>

(and default is 0)

In this topic it's claimed that bridged interfaces should be on the same broadcast domain. In another topic the only drawback is mentioned relatively to conventional switch - performance. Although both for 1.x series of pfSense.

config-gate.local-20110308090912.xml.txt

wallabybob · Mar 8, 2011, 7:52 AM

@DarkPhoenix:

In simplified form topology is simple:
1 WAN interface
4 physical LAN interfaces, bridged into BRIDGE interface

Machine 1 attached to LAN interface cannot ping/reach using any other method another machine plugged into another physical LAN interface.

Router can be pinged from both machines. Both machines can be pinged from router.

Please provide the output of pfSense command # ifconfig -a

Please provide the output of a ping attempt and traceroute attempt from Machine 1 to another machine plugged into another physical LAN interface. (Could it be a routing problem? a DNS problem?)

On my pfSense box, LAN is a bridge with two members: rl0 (wired) and ath0_wlan0 (wireless). I have no trouble communicating across the bridge. I wonder what's different about your system that causes the bridging to not work.

stephenw10 · Mar 8, 2011, 9:19 AM

@DarkPhoenix:

@stephenw10:

What about disabling bridge filtering? Even if just for a test it would be interesting.

It seems to be disabled:
		 <tunable>net.link.bridge.pfil_bridge</tunable>
			<value>default</value> 
(and default is 0)

Right but in the thread I linked to it is suggesting setting:
net.link.bridge.pfil_bridge = 1
net.link.bridge.pfil_members = 0

This seems to tie in with Wallabybob's suggestion of problems with 'block all' rules on member interfaces.

Steve

stephenw10 · Mar 8, 2011, 9:26 AM

Actually after reading through you config the difference between your setup and that of Wallabybob or in the thread I linked to is that you have LAN as a member of your bridge where as the others have LAN set AS the bridge.
It seems odd that your setup doesn't work especially since 2.0 was supposed to moving away from LAN and WAN having special status. However I wouls suggest reconfiguring it accordingly.

Steve