Noob in a bad network
So I have a simple, but badly designed network.
I have a DSL router for a 3071/509 kbps connection, which does NAT on my internal network, say 10.76.78.1/24
I have a Linux box with a single network interface, 10.76.78.2. I used to route traffic through this box, doing NAT itself on the single interface using Wondershaper. This did not work too bad, but it was confusing and bandwidth measuring tools such as iftop/ntop got confused by the double NAT, adding new rules was a chore and well, lately the GF was complaining that P2P was hurting her internet quality.
So given that I have just a Linux box with a single NIC, I try the following. Create a VM with two virtual NICs (10.76.78.8/24 WAN, gw 10.76.78.1/24 and 10.76.78.9/24 LAN), install PFSense on it (the VM is a KVM VM, KVM sets up a br0 bridge and adds eth0 to it).
It works- I have a few issues setting it up, but it works and does not seem to affect connection quality at all.
But when I use the single link shaping wizard, just entering the bandwidth characteristics of my DSL connection and not shaping any traffic, leaving everything as default, speed tests suddenly suck (say 0.3mbps download…) and internet quality sucks heavily. Removing the shaping immediately fixes things.
What gives? Is my setup completely incorrect? Can I "fix" it within my constraints?
Create a VM with two virtual NICs (10.76.78.8/24 WAN, gw 10.76.78.1/24 and 10.76.78.9/24 LAN)
I'm a little surprised this works - in fact, I'm not sure if you're actually passing traffic through the pfSense at all. For starters, your LAN and WAN networks are the same, so your routing is going to be all to cock. You can't have your LAN and WAN on the same 10.76.76.x/24 networks - start by changing one of them so they're completely different (eg: LAN 192.168.0.x and WAN 10.76.78.x). That way you won't get confused when trying to set up rules and your firewally will have an easier time routing from one side to the other.
As muswellhillbilly says, I would focus on more obvious problems before concerning myself with traffic-shaping.
If possible, limit the p2p traffic's bitrate qt the p2p client. That is more effective than traffic-shaping at the gateway router.
Yeah, the traffic is being routed correctly and everything seems to be OK. I'm doing this weird setup to avoid having to get a dedicated two-NIC box for running Pfsense (on top of the ISP DSL router which is not supported on bridge mode)- if I need a dedicated router it would make more sense to upgrade my internet connection to fiber and forget about having to shape. So yeah, I'm being CHEAP :)
Setting up a codelq for the WAN interface seems to have helped somewhat, I'll look into bandwidth limiting my P2P VM. I'd rather not limit it at the application level- I run two separate two P2P protocols and that wouldn't work too well. I'd ideally like that P2P could use all "idle" bandwidth, so I could get the best P2P throughput without impacting regular use of the network- although I understand that might be tough to achieve. A simpler alternative would be to have P2P run just on off-hours, but I'd prefer the former solution if possible, of course.
Whatever you think may be working, if your LAN and WAN occupy the same subnet then your firewall won't know how to apply your rules, nor will it know where to route traffic to/from. How can you set up a rule from your LAN to WAN, for instance, if the LAN and WAN are the same network? You can run your WAN/LAN from the same interface if you like, but you'll need to give each 'virtual' interface a VLAN ID and separate address space so the firewall will function correctly.
Im with muswellhillbilly here.. Not sure what you think you are doing but running same network on both your wan and lan is BROKEN!!! And it not going to work!