Huray, all is working.  Viva la pfsense. :D

@vanhaakonnen:

Thanks!

I diabled NAT and add a firewallrule for the wan interface. Now I can access clients from both networks :
Proto * | Source WAN set | Port * | Destination LAN set | Port * | Gateway * | …

The Clients in the 10.x.x.x and 192.x.x.x network should talk without any firewallrule to eachother. But the "real" wan (internet) comes also from a 10.0.0.1 (Fritzbox). Is this rule a good idea?

I think you are right.

Internet–--fritzbox--------pfsense------LAN

when you set a rule on the pfsense interface which is connected to fritzbox like this than traffic FROM Internet is blocked but from netwrok between fritzbox and pfsense is allowed:

Source port: any
source ip: WAN subnet
destination IP: any
destination port: any

currently you have 1 gateway group that has both gateways with the same Tier to achieve loadbalancing?
if yes –> create a second gateway group with same gateways but set them at a different Tier to achieve failover
        ---> create an additional firewall rule with destination port HTTPS, select the New gateway group as gateway
        ---> done

Checking the box "bypass fireall rules for traffic on same interface did it". Thanks again for your help.

I now have it working although I thought that setting marker and reading marker were reversed…...
I'm still not sure if everything is as it should, but in plain english I want to do this:

I want to mark all TCP-packets going to port 80 coming from 127.0.0.0/8 (the localhost IF)
These packets would arrive on the default gateway.
On that interface I want to read that marker and then choose the loadbalancing gateway....

I still don't understand what that 2nd rule is doing?

pfctl -sr | grep fmh pass out quick inet proto tcp from 127.0.0.0/8 to any port = http flags S/SA keep state label "USER_RULE" tagged fmh pass out log on dc0_vlan10 proto tcp from any to <negate_networks>flags S/SA keep state label "NEGATE_ROUTE: Negate policy routing for destination" tag fmh pass out log on dc0_vlan10 route-to { (pppoe0 217.16.40.239), (dc0_vlan13 89.250.180.1), (dc0_vlan10 89.250.179.1) } round-robin inet proto tcp from any to any port = http flags S/SA keep state label "USER_RULE: For Squid" tag fmh</negate_networks>

I tested it with pfsense or windows because it did not work with remote pptp server. I want to use it with ISP but it fails with kernel: Loop detected on pptp0.
What is the technical problem. because I think if you support PPTP connection on WAN. Other things are layer 3 or above issues and the packet is delivered to PPTP server. Is it about mpd insuffisance or adding some configuration in GUI can solve it.

I appreciate you if letting me know what is the underlying problem about.

I really want to solve it and send a patch.

Best regards

Update:

After testing a while I do have issues with client FTP to an outside FTP server on the internet.

LAN interface works perfect.

I have tride a couple different FTP clients and they hang at hang at LIST or MLSD command logging into different different FTP servers and time out.

I have set the FTP Proxy to look at all interfaces, even Bridge and still does not make a difference.

This issue was resolved when LAN -> Bridge0 all client FTP worked perfectly and including WLAN.  You can read in the previous posts I could not see my LAN client machines with bridge on LAN.

So used the guide and set OPT5  -> Bridge0 and did the other settings in the guide.  FTP works intermittently on all Bridged interfaces, excetp LAN it works perfect.

Another thing I have noticed when set to LAN -> Bridge0 I could set my Firewall rules on this interface alone and it effected all other interfaces in the bridge.  But with OPT5 -> Bridge0 I have to set firewall rules on all interfaces because Bridge0 will not effect any other interface if rules are set on it.  This might be the nature of the beast not sure.

I have tried all the pfense help guides on this issue and still no go.  Does any body have any ideas…. Thanks again for all your help!!!!!!!

That's what happens if you're not actually monitoring your connection status, but using your modem's IP or similar as your monitor IP. Use something on the Internet for that.

Hi
After some investigations on traffic I think it PPTP link does not correctly work. it become connected but does not encapsulate packets to remote gate way in GRE packets. It shows that some loop in kernel prevent it from working. I appreciate any one describing me why and giving me any solution.

thanx

Okay I switched to tier 1 and 2 only.  These still aren't working optimally.
I've attached a picture that is very descriptive of the situation.
All the traffic is on WANSHAW1 almost all time, except for a brief period when WANSHAW2 will be used.  It's not being balanced as expect.
Presently about 95% of traffic will go to WANSHAW1.  Usually when I look at WANSHAW1 it is using <10 Kbps.

Note: I have sticky connections enabled.

Any advise is greatly appreciated.  I also bought the old pfsense book a while ago so if there is something in there that I should be pointed towards, then let me know.

Untitled2.png
Untitled2.png_thumb

I don't think load balancing across 2 pfsense servers is possible. You can do that in 1 pfsense machine. You might be able to if that wire connecting the two firewalls is a WAN on the old pfsense … but it would not be able to load balance the other way though.

because it's a very critical system in a very specific production environment and it took me ages to set it up right.
I don't have the time to go through that again right now.

Make sure that the default gateway option is un-checked for both gateways in Routing

The floating rule needs to be set with both WANS selected and also make sure that you change the gateway in the rule to the load balancing gateway group NOT the default setting.

@xarope:

The reason I say it is strange, is that to resolve this issue, I turn off that pfense advanced rule to not check rules between subnets on the same interface.  Which, although I don't have this situation now, in future I may need to block say vlan2/subnet2 from vlan3/subnet3.  So it's just a stopgap whilst I figure this out.

When you need to filter between vlans, tag second vlan to pfSense firewall and configure everybody gateway to pfSense respective vlan ip.

It turns out pfsense with only 2 NICs is limited with regard to my implementation.

Now I can't spoof my MAC for a certain VLAN interface without putting my parent interface in promiscuous mode, I can't use anymore DHCP-lines from that same ISP.

Is this only on my hardware?
Does anyone for a fact know it should be working?
Anyone knows how to get it working again?

Will there be a fix in the webif?

Interfaces > (assign), wireless tab.

Add your additional access point instances there, then assign them under Interfaces > (assign)

Some settings are shared across all SSIDs, others are specific to each instance. They are grouped or noted on the pages as appropriate.

What version are you running?

That looks like something that was fixed before 2.0 was released, where a missing or disabled interface (or perhaps it was rules?) were finding their way into the ruleset somehow.

It could be something else, but that looks familiar.

It would help to see a full copy of your /tmp/rules.debug file when this error happens.