Well, i know my replies have been lacking here and i hate leaving topics unresolved, so some further information that might help others. As it stands, there are two known issues. 1). Not causing my latencies or packet loss, but still needs fixing - my upload channels, i have 2 bonded channels, i should have 4 - the ISP is investigating the cause of this, the problem this causes me is two-fold. First of all i cannot reach my 36Mbps upload speed because i only have half the channels bonded, this in turn cases me an unstable connection during high uploads - I have temporarily fixed this by putting a limiter on my WAN uploads. 2). The ISPs router - the SuperHub 3 has apparently a bug in the FW which causes high latencies and random spikes (seems to have been rolling out since Feb/March), but only when used in Modem only mode and mostly noticeable with PfSense (since this is a common choice for enthusiasts) - I'm linking a topic relating to others and this issue. While this is likely to be a router specific issue, PfSense or BSD doesn't appear to help with this, if you leave the ISP modem in modem only mode and use another firewall, even a consumer grade router the issue doesn't appear to exist or at least not as often, changing the router back to router mode and double-NATting also doesn't show the symptoms, but this is not really a workable solution. So, back to my original question, is there something or somewhere in PF i can log anything else that would be useful to anyone, to see if the OS or firewall itself is adding to this, perhaps a driver issue? (I am guessing, but trying to offer help at the same time). Side note - last night i rebooted my router and i've yet to have the same issues, based on posts in the link below, this can sometimes be true for up to about a week. This suggests the modem is getting full and unable to clear itself, in modem only mode there is limited options, in fact after about an hour, i lose access to it completely on the modem IP. While i am more convinced this is all related to a FW issue in the router after additional findings, is there anything else i can do or capture to see if anything in PF or BSD is related to this? I've read a few articles that say this issue does not seem to be in 2.5, but i'll have to try and relocate those links again. The link to one of many of the ISPs FW bug, for those interested; https://community.virginmedia.com/t5/Networking-and-WiFi/Did-anything-change-with-Superhub-3-0-firmware-recently/td-p/4192831 Post 4 in the above links multiple other articles. For anyone reading in the UK, this seems to affect SH3, with FW 9.1.1811.401 and possibly the SH4 for those on Gig Both the SH3 and SH4 use Intel Puma 6 and 7 chipsets respectively, which have their own known issues, unfortunately in the UK we are restricted by the kit supplied by the ISP and have no options to buy our own modems. If nothing else i wanted to keep people informed of the progress on my issues.

@JKnott I use KVM. vtnet2 is virtual interface that is directly attach to real interface using passthrough mode (KVM). And I also check MAC table on switch. This arp is come from vtnet2, not other port.

Oh I've felt that pain!

Where? You create a connection on pfsense and your other router on some network you come up with as your "transit" 172.16.0/30 as example... What are you not getting about this.. You create your vlan right.. Create another.. Connect 2nd router to this vlan.. How you connect them physically is up to you - be it via interfaces on each device via vlan, etc. doesn't really matter. I drew it up for you - Not getting what is difficult to understand from the drawing? Replace whatever networks you have the samples I put in, add more if you want, etc.. But your drawing your trying to use this 192.168.2 as your transit.. Which you have hosts on - so yeah that not going to work!! If you want to use 192.168.2 like you have drawn.. "EVERY" device on this 192.168.2 vlan would need default route that you want it to use .2 or .1, and then a route to this 10.5.20 network pointing to the pfsense 2.2 IP if you want these devices to talk to back and forth with 10.5 network..

Almost impossible to say without seeing the routing table. You might have something tied to a MAC address maybe? The fact inbound connections work and outbound don't can really only be a firewall rule on the internal interface, a routing issue or no outbound NAT rule for internal clients. Hard to say why that would be any different between the two instances. Steve

Ah, Ok. Yeah Client ID is not required and is only shown as a column if any clients are sending it. You can probably remove it from those static mappings. Steve

@Gertjan so i have internet working throught the vpn but am i supposed to see on my local network when i google "whats my ip" should bsee my ISP address or should i be seeing the 192.168.110.5 for the landon Openvpn account.. and when i run the dnsleaktest same ip as the ISP and cloudfare is the dns and using cell internet doesnt point to the 192.168.110.x could you happen to send me to a document or video that shows how to set it up. i read 1 article guy didnt know how he got it working.. but if there is a document or video be good or do i need to start a new topic..

@JKnott i know it took a while, but i didnt have another client that i could hook over lan on it, so i bought a usb-to-rj45 connector. with that said im getting similiar results. i go into the console and start to asign interface, but then back out of the command using ctl c. I then connected for a minute or 2. i must have some conflicting setting in my GUI but have no idea where to begin. i may reset to factory and see if it works without reloading my config.

@heper That did it. Tnx.

WAN failover is described here: https://docs.netgate.com/pfsense/en/latest/book/multiwan/index.html In addition to using policy routing you can now also set the system default gateway to a failover group which will also route traffic from the firewall itself. You can, and probably should, have both WANs set as DHCP. It may be showing as unknown if the service is not started or there is a subnet conflict perhaps. More likely the latter. You IoT/Guest AP should be in a different subnet to the LAN. That interface should be separate. The pfSense interface IP would commonly be .1 and the AP IP should be either statically set outside the dhcp range on that interface or set via a static dhcp mapping so you always know where to access it. IPv6 should probably be set to track PPPoE unless you have a static IPv6 address. To be able to have guest and regular users connected to either AP you want to set them up with multiple SSIDs on different VLANs and separate the traffic that way. Both APs should connect back to pfSense on the same port carrying all VLANs. VLANs have nothing to do with connecting between subnets, each VLAN would be a separate subnet. Traffic from the IoT/Guest subnet will be able to reach resources on LAN as long as there are firewall rules allowing it. Conversely you should have firewall rules blocking access to the LAN for most guest clients. Multicast/broadcast services, like Chromecast, are a different matter. They are not intended to be used across subnets and additional measures are required (igmpproxy/pimd). Anything that should be in the same subnet should be on the switch. Interfaces on the SG-5100 are not switch ports and though they can be bridged to act like switch ports moving traffic between them requires valuable CPU cycles they could be used elsewhere. The lag introduced going through a switch is negligible. IP cameras would normally be considered an IoT device. Commonly found with known firmware vulnerabilities and no updates from the manufacturer. That would denote they are put on a separate subnet with very limited access to anything else. However they also generally generate a lot of traffic which will all have to be routed by pfSense if they are separated from the NAS like that. The decision is yours! You can apply a basic priority based shaper to prioritise traffic from the xbox IP. It will need a static dhcp mapping to do so. However I would not do that unless you are actually seeing latency issues. Adding traffic shaping often introduces more problems that is solves. Steve

@thompsonm Screenshots of your rules on the two interfaces?

Unless you do dynamic assigned vlans, yes you assign vlan X to ssidX and vlan Y to ssidY be it they run on 2.4 or 5 band or both doesn't matter.

@stephenw10 said in Problem with pppoe over vlan: Hmm, so even though you no longer have vlan7 assigned it still gets rebuilt when config changes are made? YES! (I checked a few times on both machines)

Nice.

@skybri100 Thank you very much and greetings.

Bingo! I reset it while connected and started getting console output of the boot sequence. It was getting stuck on "Starting DNS Resolver". Quick google lead me to a Reddit post below. Basically delete this "/var/unbound/pfb_dnsbl.conf", recreate the file, and restart. Back in business! You help was very much appreciated John! Thanks, Moon https://www.reddit.com/r/PFSENSE/comments/89gt37/stuck_on_starting_dns_resolver_on_reboot/

Then make sire rules in place at site 2 allowing the traffic from the tunnel subnet the client is in. If the client is not redirecting all traffic over the VPN then they will need to be passed a route to the site 2 subnet via the VPN. Add it as a local network in the remote access server at site 1. Steve

@stephenw10 Great ! Its as I expected, Thank you very much for your answers ! Farisse