Do you have ppp connections with them?  T-1 or ethernet circuits?

Thanks For the Reply Let Me Try The Mentioned Options

thank you…I will try and return with my result...  I appreciate the time you took from your life to answer mt silly question.,

Last week i've disabled tier2 wan interface + removed the failover gw group.

the arpresolve errors still occur frequently (once a week). release/renew of the wan interface solves it.

Anyone have a clue ?

Dec  5 14:45:48 pfsense check_reload_status: Syncing firewall Dec  5 15:25:43 pfsense dhclient: EXPIRE Dec  5 15:25:43 pfsense dhclient: Deleting old routes Dec  5 15:25:43 pfsense dhclient: PREINIT Dec  5 15:25:43 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:25:43 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:25:43 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:25:43 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:25:43 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.G Dec  5 15:25:43 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:25:43 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW ..... Dec  5 15:25:52 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:25:52 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:25:52 pfsense apinger: ALARM: WAN_TELENET(8.8.8.8)  *** WAN_TELENETdown *** Dec  5 15:25:52 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:25:52 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW ..... Dec  5 15:26:02 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:26:02 pfsense check_reload_status: Reloading filter Dec  5 15:26:02 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW ..... Dec  5 15:26:07 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:26:07 pfsense php: : The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic' Dec  5 15:26:07 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:26:07 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:26:07 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:26:07 pfsense php: : ERROR!  PPTP enabled but could not resolve the $pptpdtarget Dec  5 15:26:07 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:26:08 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW .... Dec  5 15:26:15 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:26:15 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:26:16 pfsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.WAN.GW Dec  5 15:26:44 pfsense dhclient: FAIL Dec  5 15:26:47 pfsense dhclient: ARPSEND Dec  5 15:26:49 pfsense dhclient: ARPCHECK Dec  5 15:26:49 pfsense dhclient: BOUND Dec  5 15:26:49 pfsense dhclient: Starting add_new_address() Dec  5 15:26:49 pfsense dhclient: ifconfig em0 inet XXX.XXX.WAN.IP netmask 255.255.192.0 broadcast 255.255.255.255 Dec  5 15:26:49 pfsense dhclient: New IP Address (em0): XXX.XXX.WAN.IP Dec  5 15:26:49 pfsense dhclient: New Subnet Mask (em0): 255.255.192.0 Dec  5 15:26:49 pfsense dhclient: New Broadcast Address (em0): 255.255.255.255 Dec  5 15:26:49 pfsense dhclient: New Routers (em0): XXX.XXX.WAN.GW Dec  5 15:26:49 pfsense dhclient: Adding new routes to interface: em0 Dec  5 15:26:49 pfsense dhclient: /sbin/route add default XXX.XXX.WAN.GW Dec  5 15:26:49 pfsense dhclient: Creating resolv.conf Dec  5 15:26:49 pfsense check_reload_status: rc.newwanip starting em0 Dec  5 15:26:49 pfsense apinger: alarm canceled: WAN_TELENET(8.8.8.8)  *** WAN_TELENETdown ***

I have the same problem, when i write default wan on two of my (cant write default on three lan) three wan it works with failover with thoose two wan i tried to take away default wan but it still remain on one of the wan.

How did you write your roul in firewall? Can you post a dump?

i'm not quite sure how to set up rules to route return traffic.
You can probably try out using ospf. My experience has been that ospf will change the routing table.
This might help, http://forum.pfsense.org/index.php/topic,39328.0.html
Let me know if you end up getting it to work with or without ospf.
-E

I would go with vlans and not bother with another NIC.

thank's a lot for this constructive reply.

I already search for a simple way to do this and don't find it. I hoped that you find a easy trick to do the job but you made it the hard (and the bettest) way !

It seem that it is effectively too hard and too long to port and I will have to use a simplest and less elegant way to do it.

cmb,

thanks for checking things, seems i'm not so far off as i thought, LAN rules are supposed to do exactly as you clarified. I always reset states after that sort of changes.

I think I figured one thing out, my client got a second router machine (lancom) as secondary gateway via win-dhcp and might have used that most of the time, took it out of the dhcp optins for now.
Seems things are working fine now. Sometime you just need somebody more routined to tell you everything is right, that you can start searching elsewhere.

thanks again

@michaeljk:

Unfortunatly, there is no possibility to get something faster to communicate with the internet. I configured both 3 MBit WAN's with Load-Balancing which worked really fine, but unfortunately there are some services where we cannot use it (e.g. online banking and ICQ-Connection, because the IP can change on every connection).

I think the answer to this is to route all outbound https traffic and/or ICQ traffic through one of your gateways. In the Firewall Rules for LAN there are advanced options below; I think one is called gateways; and you can select a particular gateway to match the rule. Then any traffic matching https would go through the same gateway everytime, even if its busy; but its a lot better than getting signed out of banking sites.

@tludikar:

Dreamslacker… Amazing!  Thank you very much!

I got it up and running with no problems, and figured out where I went wrong originally.  But for some reason, I am able to get internet on all VLANs except VLAN3 (LAN).

Anyway, thanks very much!  Its greatly appreciated, and the instructions were spot on!

Check that your VLAN numbering scheme on both pfSense and the switch match up.  I was using VLAN30 in my example but you're referring to VLAN3.  So you must verify that both pfSense and switch use the same VLANs.

Oh and VLAN1 should not be used.

Check if you had accidentally edited the Default rule for LAN rather than duplicating it.
Check the following rules to ensure you did not edit the LAN allow rules by accident:

Firewall:Rules -> LAN -> "Default allow LAN to any rule"
Firewall:NAT -> "Auto created rule for LAN to WAN"

I've never been able to provide the necessary information to get this issue fix, but there is some ways to reduce the impact.

1. If you only have one wan connection
system -> routing -> edit -> enable Disable Gateway Monitoring

2. Kill apinger and start apinger
Install the cron package
/bin/pkill apinger
a second later
/usr/local/sbin/apinger -c /var/etc/apinger.conf

3.Part of this script can be used to only restart apinger if it's down

#!/bin/sh # ###### Modem & apinger monitor script for pfSense ######## #   # #  - Restart modem if connection is down   # #  - Restart apinger if modem connection is ok but apinger # #     status says it's down   # #  - Kill phone states if frozen   # #  - Kill phone states if they use the wrong connection    # #   # ############################################################ << Copyright        Copyright (C) 2011 Perry Mason crazypark2 (at) yahoo.dk All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1\. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2\. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Copyright monitorip='8.8.4.4' wanip='192.168.0.100' wan2ip='192.168.101.100' wan2gateway='192.168.101.1' local_voip_ip='192.168.44.100' provider_voip_ip='87.54.25.133' usbswitch='ugen0.2' attempt_file=/tmp/atempt.status off_switch_file=/tmp/offswitch.status # Write phone states to file /sbin/pfctl -s state | grep $local_voip_ip > /tmp/statetmp.status # Make sure apinger is running correctly awkrepley2=`awk '/'$monitorip'/ && /down/ {print "down"}' /tmp/apinger.status` if [ "${awkrepley2}" = "down" ]; then # Determent if connection is down or apinger has exit  ping -c2 $monitorip  if [ $? != 0 ]; then    echo "WAN2 Connection Unavailable, $monitorip Doesn't Answer To Ping Request." | logger    # Switch off modem in a sanely approach    [ ! -f "$attempt_file" ] && attempt=4 || attempt=$(cat "$attempt_file")    [ ! -f "$off_switch_file" ] && off_switch=3 || off_switch=$(cat "$off_switch_file")    echo "If this value $attempt is bigger then this value $off_switch switch off modem" | logger    attempt2=$(($attempt > $off_switch))    if [ $attempt2 = 1 ];then      off_switch2=$(($off_switch * 2))      echo $off_switch2 > "$off_switch_file"      echo "Restarting modem" | logger      /usr/sbin/usbconfig -d $usbswitch power_off      sleep 15      /usr/sbin/usbconfig -d $usbswitch power_on #sleep 9 #echo "ssssssssstart" | logger #need_pid=$(cat "/var/run/apinger.pid") #echo "pid nr $need_pid" | logger #truss -p $need_pid -o /tmp/truss\ $need_pid.out & tcpdump -ni vr0 -w /tmp/$need_pid.pcap & sleep 90 #/bin/pkill truss #/bin/pkill tcpdump #echo "eeeeeeeeend" | logger    else    attempt3=$(($attempt + 1))    echo $attempt3 > "$attempt_file"    fi  else    echo "Apinger marks WAN2 down but connection is available, so restart apinger" | logger    /bin/pkill apinger    sleep 1    /usr/local/sbin/apinger -c /var/etc/apinger.conf    [ -f "$attempt_file" ] && rm -f "$attempt_file"    [ -f "$off_switch_file" ] && rm -f "$off_switch_file"    echo "some files deleted" | logger    sleep 30    /usr/local/sbin/pfSctl -c 'filter reload'    sleep 1    /sbin/pfctl -k $local_voip_ip -k $provider_voip_ip  fi else # Kill VOIP phone states if in wrong state awkrepley3=`awk '/'$wan2ip'/ && /'$provider_voip_ip'/ && /SINGLE/ {print "down"}' /tmp/statetmp.status`  if [ "${awkrepley3}" = "down" ] ; then    /sbin/pfctl -k $local_voip_ip -k $provider_voip_ip    echo "states frozen kill them" | logger    fi # If WAN2 has recoved but VOIP states still uses WAN connection then kill them awkrepley4=`awk '/'$wanip'/ && /'$provider_voip_ip'/ {print "down"}' /tmp/statetmp.status`  if [ "${awkrepley4}" = "down" ] ; then # fix to redmine ticket #1508 didn't change so filter reload is still needed    /usr/local/sbin/pfSctl -c 'filter reload'    sleep 1    /sbin/pfctl -k $local_voip_ip -k $provider_voip_ip    echo "WAN2 is running again kill voip states" | logger    fi fi

Ok, I've worked out what was going on here…

I needed realease the NICs from Windows so it still had control over them and my traffic wasn't going through PFsese at all!

Also, from the host LAN I've changed the Default Gateway and DNS server addresses to the LAN setup in VMWare and PFsense.
This seems to be the only way I can route my traffic to the VM and out the two WANs for Load Balancing.

Here's the setup now

Windows XP > VMWare WorkStation 8 > PFSense - Two WANs

Windows XP Network:
LAN of my Network - 192.168.122.* Router1 DHCP
Wan One              - 192.168.122.* Router1 DHCP
WAN Two            - 192.168.0.*    Router2 DHCP

VMWare Network: - Set to Bridge my 3 NICs as follows:
Main GigaBit NIC - VNET0
WAN NIC 1        - VNET1
WAN NIC 2        - VNET2

PFSense is all setup with Gateways connecting, Load Balancing and Firewall configured.

Now, The only way I can think of connecting the Hosts' IP Traffic in to the VM and so out of PFSense via the two WANs is to Manually configure my Host NIC like this:

Static IP Address Config of Host LAN card:
IP Address          - 192.168.122.200
SubNet                - 255.255.255.0
Default GateWay  - 192.168.122.66
DNS Server          - 192.168.122.66

It is all working and I'm able to Download via uTorrent through both the WANs at the same time - Doubling my transfere rate.
Also, I can connect any of my other networked Machines to route traffic by setting their Default Gateways and DNS Server IPs to this 192.168.122.66

However, My Question is - Is there a better way of getting my traffic in to the VM other than setting the Host NIC to the LAN IP address of PFSence???

don't disable firewall "features"

add rules like this
WANTAB: pass | src wan_subnet | dest opt1_subnet
OPT1TAB: pass | src opt1_subnet | dest wan_subnet

also if your wan interface is not really a wan interface, you might consider disabling NAT

check the monitor IP of the gateway you are using for WAN interface. Or disable monitor IP for GW on WAN.

@anomaly0617:

All,

…

Perhaps it's frustration talking, but I'm at a loss. Any ideas what I'm missing?

Solved it. It took talking to another BSD nerd to figure it out.

The problem was under Interfaces -> WAN.

Uncheck this: Block private networks. When set, this option blocks traffic from IP addresses that are reserved for private networks as per RFC 1918 (10/8, 172.16/12, 192.168/16) as well as loopback addresses (127/8).  You should generally leave this option turned on, unless your WAN network lies in such a private address space, too.

Then traffic will be permitted to return to the 192.9.201.0 network.

i was able to add a static route, then gain access to the firewall. However now all traffic appears to be getting blocked by the firewall.
when i select a blocked packet i get this pop up
@1Scrub in on rl0 (lan interface) all fragment reasemble.

I am also seeing allot of collisionson that interface. Any help would be great.

Thanks

update
collision issues has been solved. Needed to put the cisco interface to auto duplex and collisions stoped accuring. I am however still getting blocked traffic even though i have any any statements on both the WAN and LAN interface. Still have no clue as to why.