Help on load balancing arplookup 210.213.215.254 failed: host is not on local ne

cheeky

I've updated to the latest snapshot (2-02-2007) and the following error logs appears for my load balancing. 210.213.215.254 is the gateway of WAN.
Cannot surf using the WAN gateway, but ok using the WAN2 gateway.

WAN - PPPoE
OPT1 (wan2) - static IP

Help please.

Feb 4 10:43:07 kernel: arpresolve: can't allocate route for 210.213.215.254
Feb 4 10:43:07 kernel: arplookup 210.213.215.254 failed: host is not on local network
Feb 4 10:43:07 kernel: arpresolve: can't allocate route for 210.213.215.254
Feb 4 10:43:07 kernel: arplookup 210.213.215.254 failed: host is not on local network
Feb 4 10:43:01 kernel: arpresolve: can't allocate route for 210.213.215.254
Feb 4 10:43:01 kernel: arplookup 210.213.215.254 failed: host is not on local network
Feb 4 10:42:58 kernel: arpresolve: can't allocate route for 210.213.215.254
Feb 4 10:42:58 kernel: arplookup 210.213.215.254 failed: host is not on local network
Feb 4 10:42:57 kernel: arpresolve: can't allocate route for 210.213.215.254
Feb 4 10:42:57 kernel: arplookup 210.213.215.254 failed: host is not on local network
Feb 4 10:42:57 kernel: arpresolve: can't allocate route for 210.213.215.254
Feb 4 10:42:57 kernel: arplookup 210.213.215.254 failed: host is not on local network
Feb 4 10:42:54 kernel: arpresolve: can't allocate route for 210.213.215.254

sullrich

Double check the subnet masks on all interfaces.

hoba

I guess you are the first one to try loadbalancing with PPPoE. For PPPoE the gateway is the same IP as the interface has itself. Subnetmaks is /32. Guess that causes some problems with the current loadbalancing code.

sullrich

@hoba:

I guess you are the first one to try loadbalancing with PPPoE. For PPPoE the gateway is the same IP as the interface has itself. Subnetmaks is /32. Guess that causes some problems with the current loadbalancing code.

Heh. Yeah, that will cause a problem I suspect.

cheeky

But i've been using the load balancing feature since release 1.0 up to initial 1.0.1, no problem using PPPoE on WAN.
After updating to snapshots with failover capability, i encountered this problem.
Any workaround on this, please. Thanks.

hoba

So this is an updated system? You need to recreate the pools due to the changes in poolcreation. Edit the pools and delete all the poolmembers. Then add them back with the new gui code. Does that fix it?

cheeky

Actually i have reinstalled the whole system using the latest snapshot ISO, reconfigure from scratch and created new pools for load balancing. I want the failover feature in the latest snapshots. ;D

Now, i've reformatted again my pfsense using the orig 1.0.1 installer, load balancing works perfectly.

hoba

So to get this right: Using the latest snapshot doesn't work with a PPPoE WAN as poolmember but using original 1.0.1 release does work?

cheeky

Yes, hoba. WAN via PPPoE causes problems for load balancing pools using latest snapshots but it is functional using the original 1.0.1.
This must be the cause of my sleepless nights trying to use the latest snapshots in order for my pfsense to have failover capability. ???

hoba

What do you manually assign as gateway when using 1.0.1 releaseversion for the pppoe link?

cheeky

The gateway that i assign for the WAN poolmember is the gateway that i see from the status\interfaces of WAN (assigned automatically by my ISP via PPPoE).

hoba

I guess gateway and interface IP is the same and it is a /32, right?

cheeky

WAN IP is 210.213.171.2xx subnet 255.255.255.255
Gateway IP 210.213.170.x

hoba

Ok, I guess the code is too smart to accept a /32 and a gateway outside this subnet atm. Time to revisit it ;)

cheeky

Any update on this problem? Have this been resolved by the new snapshots?
Thank you very much.

databeestje

please give me the output from the routing table from Diag -> Routes.

I also require the relevant rules from your rules.debug.
using the command page execute the following.
grep route-to /tmp/rules.debug

I do not have a working pppoe test rig so I'm working in the dark here.
Are you willing to provide me access to the box?
I also commited another load balancer fix this evening. Check if that one outputs any messages.

cheeky

I'll try loading the new snapshot later, i'm currently using now the original 1.0.1 iso. So far no problem on my load balancer using PPPoe on WAN but no failover.
I'll post my results immediately.
Yes, i'm willing to open my pfsense firewall for you to debug inside.
Thanks.

dotdash

I am also having this problem. If it helps any, here is some data on my setup:
lan=fxp0
wan=fxp1 (pppoe)
opt1=fxp2 (WAN2 static IP)

wan 100.200.200.110
subnet mask 255.255.255.255
gateway 100.200.200.254

1.0.1 release
worked fine, but ISP turned off ICMP on the next-hop router.
Started monitoring the dns server, but it frequently timed-out and went offline.

Updated using
pfSense-Full-Update-1.0.1-SNAPSHOT-02-14-2007.tgz
Deleted and re-configured balancer.
Started getting messages in system log and no traffic was going out WAN
kernel: arpresolve: can't allocate route for 100.200.200.254
kernel: arplookup 100.200.200.254 failed: host is not on local network

updated to
pfSense-Full-Update-1.0.1-SNAPSHOT-02-18-2007.tgz
Slightly diffferent messages in log

kernel: arpresolve: can't allocate route for 100.200.200.254
kernel: arplookup 100.200.200.254 failed: could not allocate llinfo
kernel: arpresolve: can't allocate route for 192.0.2.113
kernel: arplookup 192.0.2.113 failed: host is not on local network

For now, I have pulled the WAN from the balancer pool
(only has opt1)
The default gateway for my LAN is still the balancer.
I CAN however, create a rule for a specific IP pointing to gateway default
and send that machine out the WAN.

Here is a partial dump of the current routing table
Destination Gateway Flags Refs Use Mtu Netif Expire
default 100.200.200.254 UGS 0 45115 1492 ng0
100.200.200.110 lo0 UHS 0 0 16384 lo0
100.200.200.254 100.200.200.110 UH 1 105 1492 ng0

dotdash

Just got a chance to try with 1.0.1-SNAPSHOT-02-27-2007 (via update firmware)
Still getting 'host is not on local network' error.

hoba

We are missing a testbed to fix this bug atm. We'll work on it soon. Stay tuned.

sai

dotdash

@dotdash:

wan 100.200.200.110
subnet mask 255.255.255.255
gateway 100.200.200.254

Its a bit early in the morning here, but your subnet mask looks way wrong. The WAN and its gateway should be on the same subnet, and so the arplookup error seems to be reasonable. Try a subnet of 255.255.255.0

hoba

@sai:

dotdash

@dotdash:

wan 100.200.200.110
subnet mask 255.255.255.255
gateway 100.200.200.254

Its a bit early in the morning here, but your subnet mask looks way wrong. The WAN and its gateway should be on the same subnet, so try a subnet of 255.255.255.0

No, pppoe just works this way.

cheeky

@hoba
Still waiting for the final resolution to this problem, i'm still using now the original 1.0.1 with no problem.
Thank you very much.
:D ;D ::)

ndelong

If it helps, I'm in a similar situation (fail-over not working due to arp messages):

WAN1: PPPoE w/"static" DHCP /32
WAN2: Static

Running 1.0.1-SNAPSHOT-03-08-2007

Getting these messages:


kernel: arplookup 204.213.240.xxx failed: host is not on local network
kernel: arpresolve: can't allocate route for 204.213.240.xxx

Anything I can do to help? Send logs, etc?

sullrich

Try adding static routes for these hosts or check the routing table to ensure they are correct.

netstat -rn

Also I recall LB not working with pppoe but I could be wrong here.

databeestje

that would be a yes
I think we add a route for the gateway on the underlying interface instead of the ng0 device.

Which would result in this iirc.

ndelong

Our firewall started off with PPPoE on the WAN interface connected to an old Westel modem. We added a cable connection w/static IP months later and are currently testing LB/FO. Both connections work fine independently of each other - even with the load balancing pools created. As soon as we create any rules on the LAN interface that reference the LB or FO gateways, we get intermittent connectivity and those arp messages. Unfortunately, I'm not on that customer's site, so my netstat -rn is shows the routes with LB/FO rules disabled:


Destination        Gateway            Flags    Refs      Use  Netif Expire
default            204.213.240.129    UGS         0  7510236    ng0
74.92.60.116/30    link#4             UC          0        0    em3
74.92.60.118       00:13:f7:22:ae:b0  UHLW        1   432194    em3   1043
127.0.0.1          127.0.0.1          UH          0        0    lo0
192.168.1          link#3             UC          0     3236    em2
204.213.240.253    lo0                UHS         0        0    lo0

Would a temporary workaround be to connect the Westel modem to a Linksys router (to handle the PPPoE) and configure static RFC1918 IP's between the pfSense box and the Linksys?

dotdash

A similar situation was discussed here: http://forum.pfsense.org/index.php/topic,4100.0.html
Quick summary:
You could use 1.0.1 release and PPPoE on pfSense's WAN.
You could double nat, but that sucks.
If you have multiple publics (static block), have a router (another pfSense with LAN bridged to WAN??) do the PPPoE and configure pfSense with a straight public IP.

databeestje

Setup a modem with pppoe bridging so you can just use dhcp or static with a public ip on the pfsense box.

Still have not found time to troubleshoot this.

ndelong

My main issue with reverting back to straight 1.0.1 is that the traffic I need to handle requires FO rather than LB. The traffic is such that if the recipient detects even one packet coming from a different source IP, they'll can the entire connection.

At a different site I have a Westell 6100 modem/router that appears to handle the pppoe & pass the static info on to the pfsense box. I'm going to see if I can replicate that configuration at this particular site.

databeestje - If you don't have time to troubleshoot, keep tossing ideas my way & I'll do my best to see what information I can gather for you. It would be awesome if we can get native PPPoE + Cable LB/FO working - poorman's enterprise!

databeestje

as of 22-03-2007 pppoe is still not supported/fixed.

Considering my vacation next week this is unlikely to change soon.

You can however use the old style configuration as a workaround. e.g. Not the interface name, but the gateway address. This requires manual config mangling and a filter reload.
so replace "wan" with you wan gateway IP in the relevant config section, etc. Please confirm that works with a pppoe connection before we troubleshoot this any further.

e.g. edit file config.xml, search for the load balancer section. Gateway goes left, monitor ip goes right.
Save that, reboot, see if it works.

Old style configuration will trigger some logging warnings though.

cheeky

Have tested using the above procedure, duan wan load balancing now working using PPPOE account in WAN.
;D ;D ;D

databeestje

i'm one week further now. And i have a hunch.

Can you give me the contents of /var/etc/slbd.conf for the working and the non working configuration?

You will also need to clarify the ip configuration. E.g. which IP is the local address, which is the gateway and which is the monitor ip.

This would really help tracking this problem down, thank you.

rafael.cardoso

databeestje, load balance with ppoe works now?
Thank you!

databeestje

Not untill Mr Cheeky or someone else with a not working pppoe config sends me the items I specified in my previous post.

I lack a broken PPPoE connection to test this. Because I have tested this with one connection I had access too and it just worked.

techatdd

@databeestje:

Not untill Mr Cheeky or someone else with a not working pppoe config sends me the items I specified in my previous post.

I lack a broken PPPoE connection to test this. Because I have tested this with one connection I had access too and it just worked.

I have the same issue.
Can you please give me the exactly way for the work around. I will provide you with the informations you needed then.

techatdd

@techatdd:

@databeestje:

Not untill Mr Cheeky or someone else with a not working pppoe config sends me the items I specified in my previous post.

I lack a broken PPPoE connection to test this. Because I have tested this with one connection I had access too and it just worked.

I have the same issue.
Can you please give me the exactly way for the work around. I will provide you with the informations you needed then.

I just read the other post: http://forum.pfsense.org/index.php/topic,4542.0.html
Is this already fixed?

cheeky

This problem still exist using the latest 1.2 snapshots.
You have to manually edit the lood balancing pools (replace wan, opt1, etc with the real ip address of the gateway. :( :( :(

sullrich

LB works fine now with PPPoE.

cheeky

Have tried the latest ISO of Pfsense-Beta-1-testing, this problem on LB using PPPOE on WAN still exist. Using the WAN gateway as a default gateway (single gateway), no problem. But if you include it on any pool that is LB or failover, error appears.
Again, manually editing the pools replacing WAN with the real IP corrects the problem.