How to split LAN into two? - Jikjik101's network

[wallabybob]

@jikjik101:

I'm confused with Number 2. Please see attached picture if I understand correctly your suggestion sir.

You understand.

You diagram suggests group G has an access restriction schedule while group E doesn't. It could be convenient to put group E and group G on separate interfaces so you can use firewall rule schedules on group G.

[jikjik101]

@wallabybob:

This might be a good time to recall the proverb "the devil is in the detail". The costs will be significantly affected by the number of computers in group E. If its two then a cheap 5 port VLAN capable switch will be sufficient. If its 24 then you will need a rather more expensive VLAN capable switch. The way you have drawn the diagram suggests there might be more switches than you have shown in which case implementing either solution might require new cabling which might be a non trivial installation task.

Just some clarifications sirs. If for example I have 20 computers in group E and another 20 computers in group G, does it mean I need atleast a 40-port VLAN switch? Can't I use a normal switch to connect all my clients behind Group G and E?

Thanks for all your input. Maybe I am going to change the title of this thread to jikjik101's network, because I think I need more of your expertise to help me build my network in a sound technique and more appropriate methods.

Every now and then, some problems arise in my network and I am going to post it here so that I can access it easily. I hope the moderators don't mind if I am going to "own" this thread. ;D

.jpg)
.jpg_thumb)

[jikjik101]

This is ONE of the concerns of my network.  ;D

As you can see, I have three ISPs and they are in load balance mode. But I cannot "stabilize" my ISP1 and ISP3. The connections are so erratic that it becomes so hard to connect to the internet. Unlike my ISP2, the connection is so stable and so I just use the failover mode with ISP2 in tier 1 and both ISP1 and ISP3 in tier2.

If I assign my whole network in using just one ISP, it is stable.

I already tried the following:
1. Assign an ISP as default gw
2. Not assigning a default gw
3. Check the "allow default gw switching"
4. Uncheck the "allow default gw switching"
5. Set each GW with maximum and minumum latency % base on its RRD.

I have squid, squidguard, lightsquid, lusca-cache, havp, vnstat2 and bandwidthd.

I cannot fully utilize all my ISPS, it seems ISP2 is doing the hardwork. >:( and the rest are just easy-go-lucky ISPs. :-X


[stephenw10]

With those rules and gateways I would expect almost all traffic to using be using ISP2, and it is.
This is because most of your traffic is caught by the first rule as it's web traffic. Only non-webtraffic is reaching the second rule where it is shared between ISP1 and 3 as they are both in the same tier.
You need to change the gateway to loadbalance on the first rule if you want to see the traffic more evenly distributed.

There is no need (or harm) to use tier5 in your load balancing rule. The importance of each connection is relative within the gateway and not related to the other gateways. If you had all three at tier1 it would be the same. The same applies to your failover3 gateway.

You can use a normal switch behind your VLAN switch to connect your clients.

Steve

[wallabybob]

@jikjik101:

Maybe I am going to change the title of this thread to jikjik101's network, because I think I need more of your expertise to help me build my network in a sound technique and more appropriate methods.

Every now and then, some problems arise in my network and I am going to post it here so that I can access it easily. I hope the moderators don't mind if I am going to "own" this thread. ;D

You might find it more workable to have as few topics as possible per thread: that is make a new thread for a new issue.

@jikjik101:

If for example I have 20 computers in group E and another 20 computers in group G, does it mean I need atleast a 40-port VLAN switch? Can't I use a normal switch to connect all my clients behind Group G and E?

Maybe I've missed something. The 40 is from 20 in group E + 20 in group G? But your diagram shows group G as WiFi clients connecting to an Access Point. The AP would use one port on a switch (VLAN or non-VLAN). The 20 computers in group E would use 20 ports in a switch (because they are shown as using wired connections) unless they are connected to another switch.

[jikjik101]

@stephenw10: sorry for not being clear. The fw and gw pictures that I posted are my current setup as a solution to my erratic loadbalance on ISP1 and ISP3. That is only to utilize my my ISP1 and ISP3 because they are not stable if I use the LoadBalance gw.

You are correct that my webbrowsing enters ISP2 and the rest to my Failover GWs. That is my current setup. i forgot to disable the first two rules when I took the screenshots.

But the erratic connection that I am talking about is when I use the LoadBalance gw and disabling the first two fw rules leaving this one alone active:

• • • • • LoadBalance none

I only use the default LAN rule with LoadBalance as gw, disable/remove other rules except the Anti-lockout but same results.

@wallybob:

The way you have drawn the diagram suggests there might be more switches than you have shown in which case implementing either solution might require new cabling which might be a non trivial installation task.

I remove some switches in the diagram because I don't find it essential for the network diagram. sorry, my bad.

The 40 is from 20 in group E + 20 in group G? But your diagram shows group G as WiFi clients connecting to an Access Point. The AP would use one port on a switch (VLAN or non-VLAN). The 20 computers in group E would use 20 ports in a switch (because they are shown as using wired connections) unless they are connected to another switch.

Actually I have more than 200 computers behind Group E and Group G. Group G atleast 100 WiFi clients and Group E another 100 both Wired And WiFi clients.

You can use a normal switch behind your VLAN switch to connect your clients.

I think stephenw10 already answered my clarification regarding a LAN switch behind a VLAN switch.

I'll go back to my concern, I use the default LAN fw rule with LoadBalance as my gw, leaving my LAN rule as follows:
*  *  *  LAN Address  22 * * Anti-Lockout Rule
                                                80
  *  *  *  *  *  LoadBalance  none    Default allow LAN to any rule

But I cannot utilize my ISP1 and ISP3 because they just suddenly drop my connection, or worse they cannot get the maximum bandwidth even if I bombarded my system with lots of video streaming.

[stephenw10]

Your ISPs 1 and 3 both use a wireless connection. They are likely to have high latency. When you are trying to loadbalance all three what you see in the logs?
It's very possible that they are being removed from the loadbalancing gateway due to the latency becoming too high or packet loss.

Steve

[jikjik101]

All my ISPS are wireless.
ISP1 is using a grid antennae
ISP2 is using radio tower
ISP3 is using satellite dish

My syslog is clear of any disconnections from any ISP except if there is a high network utilization.

What I don't understand is for example I'm downloading a torrent, I can see all my ISPs as being used due to high traffic. The only difference is that my ISP2 has a steady traffic, but my ISP1 and ISP3 shows very minimal or "erratic" connection. See the attached picture please.

[stephenw10]

In your diagram you show some infrastructure in the network of ISP2 between you and the radio link section. I would guess that ISP2 is limiting your connection speed in that infrastructure such that you are never seeing the limit of what the radio link can achieve and hence any variation in speed. The other connections, ISP1 and 3, are subject to error checking and additional network overhead caused by a wireless connection.

Steve

[jikjik101]

I don't know if you're a magician or an x-men with telekinesis power but I think you got it right. :o
The diagram actually only showed a building with ISP2, but as a matter of fact, all my ISPs are in the same roof with 10-20meters in-between distance. The only difference is that ISP2 has a router in-between my pfsense box and the rest has none.

But I think you're right when you say that ISP2 is limiting my connection speed. It looks like ISP2 had put a cap on my download speed so it looks like very stable.

[jikjik101]

And I think I got my multiwan to work with loadbalance with squid.

What I did is just put floating rule and the manual NAT.
In the floating rule, I select WAN1 as the interface source, loadbalance gw as the gateway. And that's it.

• WAN1 address * * * LoadBalance none
  My loadbalance with squid is now working (I think) :o

I read this somewhere but I forgot the link. I don't know why it is needed to put WAN1 Address as the source and selecting WAN1 interface only in the floating rule.
I found the link: http://forum.pfsense.org/index.php/topic,38882.0.html


[jikjik101]

kernel: pid 26983 (imspector), uid 0: exited on signal 11 (core dumped)

What is this? This is with imspector-wip. I tried to reinstall but failed.
When I looked at my installed packages, imspector didn't show up. When I go to packages, I didn't find the package anymore. So I uninstall it and try the imspector, the other version.

[stephenw10]

@jikjik101:

I don't know if you're a magician or an x-men with telekinesis power

Hmmm, X-man!  ;D

I can't help you with imspector though.

Steve

[jikjik101]

I thought you can solve anything.  ;D

I have another problem, the loadbalance works. But sometimes when an ISP goes down, it doesn't reconnect. I have to edit its gateway and save just to "wake" it up. Any thoughts?

[jikjik101]

Now this make my head aches!  >:(

It seems that LoadBalance doesn't work properly.
Default GW: None
Unabled: Use sticky connections
Unabled: Allow default gateway switching

NAT: Manual
WAN1 * * * * * * NO
WAN2 * * * * * * NO
WAN3 * * * * * * NO

LoadBalanceGW is the gateway group for the 3 ISPs under same tier.

Firewall Rules
LAN:

• • • • • LoadBalanceGW None
          Floating:
• • • • • LoadBalanceGW None

Packages: squid, lusca-cache, squidguard, lightguard, havp, bandwidthd, vnstat2 and imspector
Squid is transparent mode.

WAN1 = Dynamic, 2 DNS
WAN2 = Static, 2 DNS
WAN3 = Static, 2 DNS - 1 is the Google DNS (8.8.8.8)

Problem1: It seems that round-robin doesn't work correctly. It seems that one ISP gets the heavy work while the other two are just simply "sleeping". But when I check the http://pfsense.org/ip.php, I get all the public IPs of the three ISPs. When I checked the firewall logs, it seems that an ISP gets a weight twice the other two. I can see that ISP1 for example is being used twice before it connects to ISP2 then ISP3 then goes back to ISP1 then another ISP1 then ISP2 and so on and so forth.

Problem2: If one ISP gets down, it doesn't reconnect. I manually edit the router page or unplug/plug the cable just to get it online. The status either stays OFFLINE or Gathering Data, it never goes back to ONLINE.

Please help again. TIA.

[stephenw10]

I assume you have made most of those settings because you are running Squid with loadbalancing? I've never tried to do that I'm afraid.
I think you would be better asking a new question in the loadbalancing section of the forum. This thread is now long enough to put people off reading it!  ::)

Steve

[jikjik101]

I cannot get the loadbalance to work so I go to policy routing with squid in transparent.

Although I am disappointed with my setup since I cannot fully utilize all my ISPs but I have to be contented with it.
I manually balance the load to the 3 different ISPs.

GroupA(high priority) - ISP1
GroupB(medium priority) - ISP2
GroupC(low priority) - ISP3

I just add the following in my custom options of squid to make it work (IPs are just for example):
acl GroupA src 192.168.100.1/24;
acl GroupB src 192.168.101.0/24;
acl GroupC src 192.168.102.0/24;

tcp_outgoing_address 10.10.10.1 GroupA;
tcp_outgoing_address 10.10.10.2 GroupB;
tcp_outgoing_address 10.10.10.3 GroupC;