Routing issues

MaxPF

I have been using a pfSense 2.0.1 box in the office used mainly as a filtered bridge and for routing 1 other subnet dedicated to isolating and providing internet access to guests. It is configured as follows:

WAN: 192.168.0.252/24 (bridged to LAN)
LAN: 192.168.0.249/24 (bridged to WAN)
OPT1: 192.168.10.253/24 (guest wireless network)
Manual outgoing NAT set for the 192.168.10/24 subnet
Default GW: 192.168.0.254 (another pfSense box I use as firewall/gateway)

The traffic from OPT1 would get NATted to 192.168.0.252 and then get to internet through the default gateway, but have rules to prevent access to the 192.168.0.0 subnet, just as I wanted.

All this worked great. The problem started few days ago when we had a new VOIP phone system installed and I added a new port to the pfSense box:

OPT2: 192.168.30.254/24

Since then neither traffic from OPT1 or OPT2 can get to internet. They can get as far as 192.1680.0.0/24, but they are not using the 192.168.0.254 gateway even if it is set as default gateway:

[2.0.1-RELEASE][admin@pfbridge]/root(1): netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.0.254      UGS         0      229    em0
127.0.0.1          link#5             UH          0      131    lo0
192.168.0.0/24     link#4             U           0     7036   fxp0
192.168.0.249      link#1             UHS         0        0    lo0
192.168.0.252      link#4             UHS         0        0    lo0
192.168.10.0/24    link#2             U           0      141    xl0
192.168.10.253     link#2             UHS         0        0    lo0
192.168.30.0/24    link#3             U           0     2099    xl1
192.168.30.254     link#3             UHS         0        0    lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::1                               ::1                           UH          lo0
fe80::%em0/64                     link#1                        U           em0
fe80::20c:f1ff:fefc:f2f0%em0      link#1                        UHS         lo0
fe80::%xl0/64                     link#2                        U           xl0
fe80::204:75ff:fe84:aca8%xl0      link#2                        UHS         lo0
fe80::%xl1/64                     link#3                        U           xl1
fe80::204:75ff:fe7c:12f0%xl1      link#3                        UHS         lo0
fe80::%fxp0/64                    link#4                        U          fxp0
fe80::20c:f1ff:fefc:f2f1%fxp0     link#4                        UHS         lo0
fe80::%lo0/64                     link#5                        U           lo0
fe80::1%lo0                       link#5                        UHS         lo0
ff01:1::/32                       fe80::20c:f1ff:fefc:f2f0%em0  U           em0
ff01:2::/32                       fe80::204:75ff:fe84:aca8%xl0  U           xl0
ff01:3::/32                       fe80::204:75ff:fe7c:12f0%xl1  U           xl1
ff01:4::/32                       fe80::20c:f1ff:fefc:f2f1%fxp0 U          fxp0
ff01:5::/32                       ::1                           U           lo0
ff02::%em0/32                     fe80::20c:f1ff:fefc:f2f0%em0  U           em0
ff02::%xl0/32                     fe80::204:75ff:fe84:aca8%xl0  U           xl0
ff02::%xl1/32                     fe80::204:75ff:fe7c:12f0%xl1  U           xl1
ff02::%fxp0/32                    fe80::20c:f1ff:fefc:f2f1%fxp0 U          fxp0
ff02::%lo0/32                     ::1                           U           lo0

Looking at the traffic in the logs, I see it accepted by OPT1 and OPT2, but it never gets to the default gateway.

In the logs this also started appearing every minute:

kernel: arpresolve: can't allocate llinfo for 192.168.0.254

Any suggestion would be appreciated.

podilarius

The issue might be interface assignment. you have an xl0 and xl1. If you physically added an interface, then the new nic might have taken over as xl0. Try switching OPT1 and OPT2 interface assignments and restart. As in, make OPT1 use xl1 and OPT2 to use xl0. Get some trace routes together from a node behind each interface to see where the communication is breaking.

MaxPF

Thanks for the reply. I solved the kernel: arpresolve: can't allocate llinfo for 192.168.0.254 problem by changing the LAN interface from EM0 to DRIDGE0 (created when I bridged LAN and WAN). After that change I noticed the the default gateway disappeared from the routing tables, but was set correctly in the web UI, so I just went on the WAN interface page, clicked Save and Apply and the default GW was restored.

After that everything started working again as I wanted.