Logs to check multi-WAN load balancing, and configure persistency or sessions?
I looked at the firewall logs, and I see no information about which WAN the traffic is going through. I have multi-WAN, configured for load balancing, but it appears only one of the WAN's is being used. There are quirks about my setup that might account for that, and it does appear that a tiny amount of traffic might be going through the other WAN, but the only info I have is in the traffic charts. In particular, it might be that only a few high bandwidth connections are coincidentally going on one of the WAN's, while the other low bandwidth connections are going on the other one, but I have no data to be sure that's what is happening.
It appears that the high bandwidth connections are "sticking" to one WAN, and that probably has something to do with the states, sessions, or persistency, or something like that. Is there a way to control how and when connections are moved around to the different WAN's? In my use case, it's probably not important to maintain connections on a single WAN, and it would be better to just shuffle everything around more evenly. Is it possible to do that?
Once a state is created then that state will always be out the WAN it got allocated to. There is no feature to shuffle around existing states, because mostly that would break the communications anyway - the TCP packets back and forth would suddenly have a different public IP.
If there are many clients doing lots of things, then the mixed allocation of client states to WANs in a gateway group means that on a rough average the WANs will have load balanced as per the weights given.
If there are only a few clients doing things that use a lot of bandwidth, then those clients might happen to get allocated to the same WAN. If it is just vanilla downloads, then if the clients use a download manager that creates multiple parallel download streams that will really help them to use the available bandwidth on multiple WANs. If the clients use special applications that move lots of data across a single TCP/UDP connection then it is harder - for that you might know which are the client and server ends and be able to have more specific rules that put certain clients on 1 WAN and other clients on another WAN to purposely create a kind of manual load-balancing.
Thank you very much for the detailed assessment of my situation and options. You guessed correctly that there are only a few clients, and they are moving a lot of data across a single TCP/UDP connection. I really like your idea of purposely creating a kind of manual load balancing, but after some thought, I don't think this would work until the states expired.
I really need to shuffle states, because all traffic goes out of very long lived connections, like Tor and VPN. In both of those cases, either the IP is already intentionally expected to be changing randomly, or it doesn't matter because it will be masked by the VPN which presents the same IP to servers that might complain about the changing IP. So, I guess there really needs to be a feature that shuffles states around. Is this possible somehow? I'm thinking about how to make a feature request that makes sense, and is actually doable. What do you think?