I have been using a pfSense 2.0.1 box in the office used mainly as a filtered bridge and for routing 1 other subnet dedicated to isolating and providing internet access to guests. It is configured as follows:
WAN: 192.168.0.252/24 (bridged to LAN)
LAN: 192.168.0.249/24 (bridged to WAN)
OPT1: 192.168.10.253/24 (guest wireless network)
Manual outgoing NAT set for the 192.168.10/24 subnet
Default GW: 192.168.0.254 (another pfSense box I use as firewall/gateway)
The traffic from OPT1 would get NATted to 192.168.0.252 and then get to internet through the default gateway, but have rules to prevent access to the 192.168.0.0 subnet, just as I wanted.
All this worked great. The problem started few days ago when we had a new VOIP phone system installed and I added a new port to the pfSense box:
Since then neither traffic from OPT1 or OPT2 can get to internet. They can get as far as 192.1680.0.0/24, but they are not using the 192.168.0.254 gateway even if it is set as default gateway:
[2.0.1-RELEASE][admin@pfbridge]/root(1): netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 192.168.0.254 UGS 0 229 em0 127.0.0.1 link#5 UH 0 131 lo0 192.168.0.0/24 link#4 U 0 7036 fxp0 192.168.0.249 link#1 UHS 0 0 lo0 192.168.0.252 link#4 UHS 0 0 lo0 192.168.10.0/24 link#2 U 0 141 xl0 192.168.10.253 link#2 UHS 0 0 lo0 192.168.30.0/24 link#3 U 0 2099 xl1 192.168.30.254 link#3 UHS 0 0 lo0 Internet6: Destination Gateway Flags Netif Expire ::1 ::1 UH lo0 fe80::%em0/64 link#1 U em0 fe80::20c:f1ff:fefc:f2f0%em0 link#1 UHS lo0 fe80::%xl0/64 link#2 U xl0 fe80::204:75ff:fe84:aca8%xl0 link#2 UHS lo0 fe80::%xl1/64 link#3 U xl1 fe80::204:75ff:fe7c:12f0%xl1 link#3 UHS lo0 fe80::%fxp0/64 link#4 U fxp0 fe80::20c:f1ff:fefc:f2f1%fxp0 link#4 UHS lo0 fe80::%lo0/64 link#5 U lo0 fe80::1%lo0 link#5 UHS lo0 ff01:1::/32 fe80::20c:f1ff:fefc:f2f0%em0 U em0 ff01:2::/32 fe80::204:75ff:fe84:aca8%xl0 U xl0 ff01:3::/32 fe80::204:75ff:fe7c:12f0%xl1 U xl1 ff01:4::/32 fe80::20c:f1ff:fefc:f2f1%fxp0 U fxp0 ff01:5::/32 ::1 U lo0 ff02::%em0/32 fe80::20c:f1ff:fefc:f2f0%em0 U em0 ff02::%xl0/32 fe80::204:75ff:fe84:aca8%xl0 U xl0 ff02::%xl1/32 fe80::204:75ff:fe7c:12f0%xl1 U xl1 ff02::%fxp0/32 fe80::20c:f1ff:fefc:f2f1%fxp0 U fxp0 ff02::%lo0/32 ::1 U lo0
Looking at the traffic in the logs, I see it accepted by OPT1 and OPT2, but it never gets to the default gateway.
In the logs this also started appearing every minute:
kernel: arpresolve: can't allocate llinfo for 192.168.0.254
Any suggestion would be appreciated.
The issue might be interface assignment. you have an xl0 and xl1. If you physically added an interface, then the new nic might have taken over as xl0. Try switching OPT1 and OPT2 interface assignments and restart. As in, make OPT1 use xl1 and OPT2 to use xl0. Get some trace routes together from a node behind each interface to see where the communication is breaking.
Thanks for the reply. I solved the kernel: arpresolve: can't allocate llinfo for 192.168.0.254 problem by changing the LAN interface from EM0 to DRIDGE0 (created when I bridged LAN and WAN). After that change I noticed the the default gateway disappeared from the routing tables, but was set correctly in the web UI, so I just went on the WAN interface page, clicked Save and Apply and the default GW was restored.
After that everything started working again as I wanted.